###################################################################### Rose::DBx::Object::InternalPager 0.02 ###################################################################### NAME Rose::DBx::Object::InternalPager - Throttle Rose DB Iterator Fetching SYNOPSIS use Rose::DBx::Object::InternalPager; my $pager = Rose::DBx::Object::InternalPager->new( class_name => "Namespace::Author", manager_method => "get_authors", manager_options => { query => [ published => 'yes' ], require_objects => [ 'city_of_birth' ], sort_by => 'last_name', }, ); while(my $author = $pager->next()) { print $author->first_name(), " ", $author->last_name(), " ", $author->city_of_birth->string(), "\n"; } DESCRIPTION "Rose::DBx::Object::InternalPager" is a 3rd party module for "Rose::DB" iterators to work around MySQL client's limited control over how many rows are fetched from the database at a time. "Rose::DBx::Object::InternalPager" provides a hack to limit the number of fetched records and prevents programs from running out of memory. The pager creates an iterator object, similar to the Rose "Manager"'s "get_xxx_iterator()", method. Except, behind the scenes, the pager makes sure to never fetch more than a preset number of records from the database at a time. To accomplish this, it uses LIMIT to limit the number of records retrieved, and OFFSET to fetch the next batch. This approach might lead to anomalies when the database gets modified while the pager is at work, and this is the reason why this module has been released *outside* of the "Rose::DB" realm as a 3rd party module. While normally, you would call my $itr = Namespace::Author::Manager->get_authors_iterator(...); to get an iterator object which offers a "next()" method to get from one database record to the next, with "Rose::DBx::Object::InternalPager", you call my $pager = Rose::DBx::Object::InternalPager->new( class_name => "Namespace::Author", # Note: no 'Manager' manager_method => "get_authors", # Note: no 'iterator' # ... ); which returns a pager object that can be used to iteratate over all database records found via while(my $author = $pager->next()) { # ... } Just as the manager's "get_xxx_iterator()" method offers ways to modify the query with "query", "sort_by" and other parameters, these parameters can be set with the pager by using the "manager_options" parameter: my $pager = Rose::DBx::Object::InternalPager->new( class_name => "Namespace::Author", # Note: no 'Manager' manager_method => "get_authors", # Note: no 'iterator' manager_options => { query => [ published => 'yes' ], require_objects => [ 'city_of_birth' ], sort_by => 'last_name', }, ); By default, the pager fetches 50 records at a time. This value can be modified by setting the "per_page" parameter in the optional "pager_options" hash: my $pager = Rose::DBx::Object::InternalPager->new( # ... pager_options => { per_page => 100, }, ); WHY THIS MODULE? Even with "mysql_use_result" set, I've found that with large database tables, clients run out of memory when they want to iterate over all records of a table. At the cost of eventually creating anomalies, the pager provides fine-grained control over the amount of memory used by the database client application. DISCLAIMER Note that while this module uses "Rose::DB", it was released and will be maintained *separately* from John Siracusa's project. LEGALESE Copyright 2007 by Mike Schilli, all rights reserved. This program is free software, you can redistribute it and/or modify it under the same terms as Perl itself. AUTHOR 2007, Mike Schilli