#complete test:
dropdb -U postgres comma ; createdb -U postgres -O comma comma; \
make distclean ; perl Makefile.PL && make && make test && sudo make install

THINK:
	is the indexing iterator only implementation of lazy doc eval
	performance-errific? or is it reloading a doc each time?

things to add to guide.html:
	storage iterator overloading
	virtualdocs / to_array
		notes: refresh() not called for you, docs are r/o - call 
		get_lock() if you want it otherwise

dave ideas:
  2 - (not sure if this is something we should do via comma, but to 
    fight the "SQL resistance" at MMFA):
  this common construct is reaaaaaly slow:
        for ( @{ $doc->issue_ptr } ) {
           push @issues, $index->single( where_clause => "doc_id = $_" )
        }
        foreach my $issue (@issues) {
          ...
        }
  you hit the DB N times. you can hit it just once by building a gigantic
  where clause, e.g.
        my $wc = join ("OR", map { "doc_id = $_" } @{ $doc->issue_ptr });
        my $it = $index->iterator(where_clause => $wc);
        while(++$it) { 
          ...
        }
  or even better, using an IN statement:
        my $wc = "doc_id IN (".join (",", @{ $doc->issue_ptr }).")";
        my $it = $index->iterator(where_clause => $wc);
        while(++$it) { 
          ...
        }


Reported by tim, 2/15/2007

This doesn't work:
$ perl -MXML::Comma -e 'use strict ; my $def = XML::Comma::Def->read( name=>"AllAfrica_Publisher" ); foreach my $el ( $def->def_sub_elements() ) { my $tag = $el->name(); print $tag . "\n" if $el->element_is_plural($tag) ; }'

This does:
$ perl -MXML::Comma -e 'use strict ; my $def = XML::Comma::Def->read( name=>"AllAfrica_Publisher" ); foreach my $el ( $def->def_sub_elements() ) { my $tag = $el->name(); print $tag . "\n" if $def->is_plural($tag) ; }'

This is an API bug. See if the other is_* have this problem as well.

you shouldn't have to say $el->element_is_plural($tag)
$el->element_is_plural() shuold be enough


Test::More stuff:
- standardize/clean up the order of "use" statements, etc.
- nominally using Test::More (numbered tests via ok instead of descrip):
    order.t, indexing.t, parser.t, storage.t, validation.t


with postgres, we get lots of spurious warnings starting "NOTICE"
	these can be ignored (i think/hope)
	the Pg code in general needs a general cleanup too.

with postgres, we fail t/multi_store_doctype.t 2..29, yikes.
pg for dummies:
  To get postgres up and running on my machine, for use with comma, I
  just did an ubuntu install, then:
    apt-get install postgresql-8.1 postgresql-server-dev-8.1
    sudo cpan -i DBD::Pg
    sudo bash
     su postgres
      createuser comma #for now make a superuser
        #there is a bug we need to fix with COPY commands with Pg
        #relevant tests are in t/indexing.t
      createdb -O comma comma
      exit #shell as postgres user
     $EDITOR /etc/postgresql/8.1/main/pg_hba.conf
     ## change:
     #  local  all  postgres  ident sameuser
     ## to:
     #  local  all  all       password
     su postgres
      echo "alter user postgres with password 'random_1';" | psql
      echo "alter user comma with password 'random_2';" | psql
      exit #shell as postgres user
     /etc/init.d/postgresql-8.1 restart
     psql -U comma comma #make sure you can connect
     psql -U postgres #make sure you can connect
     #edit Configuration.pm accordingly
#TODO: must be some way to say that passwords only matter for comma,
#in all other cases use the default per-user ident thing

release notes:
NOTE!
  comma.log CANNOT be on NFS or you will lose messages (flock limitation)

- http://www.mail-archive.com/dbi-users@perl.org/msg28459.html
    this is harmless, but if you don't like it, upgrade to
    DBD::mysql > 3.0007
  prepare_cached(
          SELECT table_name 
            FROM index_tables 
            WHERE doctype  = ? 
            AND textsearch = ? 
              ORDER BY table_type
        ) statement handle DBI::st=HASH(0x8b2b390) still Active at 
  .test/lib//XML/Comma/SQL/Base.pm line 755

2.0 RELEASE TODO:
- document the remaining 1.2x -> 2.x changes, esp in guide.html
- make sure perl Makefile.PL && make && make test && sudo make install
  works out of the box w/o problems or errors w/ either mysql or postfix

LATER:
- element-specific locking for when whole body lock is unnec. and too
  heavy weight?
- maybe use ExtUtils::MakeMaker::prompt for automated testers
- it'd be nice to have "iterators" on nested plural elements
- test cases for $blob_el->{append|set}() combinations
- make $blob_el->set() more efficient by nuking the fcopy and
  instead keep the appendage and the original data seperate
  use _set_was_called for this
- audit for SQL queries => parameterized queries, e.g.:
  my $sth = $dbh->prepare_cached ( "SELECT bla FROM foo WHERE key = \"$key\"");
  $sth->execute();
becomes:
  my $sth = $dbh->prepare_cached ( "SELECT bla FROM foo WHERE key = ? ");
  $sth->execute( $key );
this is MUCH more secure and less error prone. it appears most of the
SQL code we have (if not all) already does this, but since I just 
discovered it, I want to make sure it ALL does...

- t/indexing.t sometimes fails because of an incomplete previous run, or
	cruft in the database. make it more robust.
	### step 5. - STORE_ERROR/POST_STORE_ERROR/DOC_INDEX_ERROR (0001) 
	think: table to drop, delete entryy in index_tables, delete any comma_locks (dangerous)

- on call to read/retrieve(), but on
  subsequent call to $doc->somefield, grab from db if poss. else grab from disk for 
  speed reasons. allow old behavior with read(slow => 1), but I'm not sure why you'd
  need that except debugging)
  this will also allow things like my @docs = $it->all() - right now you couldn't do
  that without calling a read_doc on each item, which is verrrrry slow...
- it'd be nice if we could do some SQL-ish things with storage iterators... but best to
  read up on the progress (or is that lack of progress) with SQLite first...

