Permalink
Commits on May 15, 2012
  1. Reorganize copy denormalizations to properly support multiple-column …

    …copy.
    
     * pyrseas/extend/__init__.py (DbExtension.__init__): Allow all
       attributes.
     * pyrseas/extend/audit.py (CfgAuditColumn.apply): Call add_func at
       the database rather than schema level.
     * pyrseas/extend/denorm.py (ExtDenorms): New helper class.
       (ExtCopyDenormColumn.add_trigger_func): Deal with both single or
       multi-item translation tables.  (ExtCopyDenormColumn.apply):
       Refactor out column-level code to ExtDenorms.  Create single and
       multi-item translation tables.  (ExtDenormDict.from_map): Accept
       new input map structure and use new internal structures.
       (ExtDenormDict.link_current): Adjust loop to new internal
       structures.
     * pyrseas/extend/function.py (CfgFunctionSource, CfgFunctionSegment,
       CfgFunctionTemplate, CfgFunctionSourceDict): New classes to support
       more flexible transformations of function sources.
       (CfgFunction.apply): Replace source patterns using both single and
       multi-item tables.
     * pyrseas/extend/table.py (ExtTable.apply): Call denorms.apply
       instead of processing columns individually.
       (ExtClassDict.link_refs): Columns are now an attribute of an
       ExtDenorm.
     * pyrseas/extenddb.py (ExtendDatabase.__init__): Initialize funcsrcs.
       (ExtendDatabase.add_func): New method to add functions at database
       level.  (ExtendDatabase._from_cfgmap): Process function templates
       and segments.
     * tests/extend/test_denorm.py
       (CopyDenormalizationTestCase.test_copy_column): Reorganize
       according to new input map.
       (CopyDenormalizationTestCase.test_copy_columns_cross_schema): New
       test to verify changed functionality.
    jmafc committed May 15, 2012
Commits on May 1, 2012
  1. Document changes regarding copy, aggregate and calculated denormaliza…

    …tions.
    
     * cfgobjects.rst: Add 'Extension' to title.
     * extenddb.rst: Amend description of "dictionaries" in ExtendDatabase.
     * index.rst: Move predefext section ahead of tools. Order tools in
       alphabetical order.
     * overview.rst: Add section on Schema Augmentation.
     * predefext.rst: Add sections for Copy Denormalizations, Aggregate
       Denormalizations and Calculated Columns.
    jmafc committed May 1, 2012
Commits on Apr 27, 2012
  1. Add trigger/function to force cascade denormalization when parent cha…

    …nged.
    
     * pyrseas/extend/denorm.py (ExtCopyDenormColumn.add_trigger_func):
       New method to be called for each trigger/function.
       (ExtCopyDenormColumn.apply): Refactor code to new function, add
       'copy_cascade' trigger/function.
     * pyrseas/extend/function.py (CFG_FUNCTIONS): Add definition for
       'copy_cascade' and change name/description for 'copy_denorm'
       function.
     * pyrseas/extend/trigger.py (CFG_TRIGGERS): Add 'copy_cascade'
       trigger and change name of 'copy_denorm' procedure.
     * tests/extend/test_denorm.py
       (CopyDenormalizationTestCase.test_copy_column): Add parent
       table/function assertions.
    jmafc committed Apr 27, 2012
Commits on Apr 24, 2012
  1. Document and reorganize new dbtoyaml options.

     * AUTHORS: Add Roger Hunwicks, Andrey Popp.
     * docs/dbtoyaml.rst: Document -N and -T options and add info to -n
       and -t.
     * pyrseas/dbtoyaml.py: Group -n/N/t/T options in a separate section
       and editorialize help text and meta variables to resemble pg_dump.
    jmafc committed Apr 24, 2012
Commits on Apr 23, 2012
  1. dbtoyaml: add -N/--exclude-schema, -T/--exclude-table

      * -N/--exclude-schema command line option allows to specify several schemas
        (via repeating option declaration) to exclude from output
    
      * -T/--exclude-table command line option allows to specify several tables
        (via repeating option declaration) to exclude from output
    
      * -n/--schema now allows to specify several schemas to include in output (via
        repeating option declaration).
    
      * Database.to_map now doesn't treat schemas=[None] as special case
    andreypopp committed Apr 23, 2012
Commits on Apr 20, 2012
  1. Correct handling of closed connections.

     * pyrseas/lib/dbconn.py (DbConnection.close): Always set conn to None.
       (DbConnection.execute): Test Psycopg2 closed status too.
    jmafc committed Apr 20, 2012
Commits on Apr 19, 2012
  1. Make DbConnection more generic and move into pyrseas.lib.

     * docs/database.rst: Explain new CatDbConnection class.
     * docs/dbconn.rst: Explain generic usage of extended DbConnection.
       Add new automethods close, commit, rollback, execute and remove
       autoattribute.version.
     * docs/{dbobject,index}.rst: Update links.
     * pyrseas/database.py (CatDbConnection): New class derived from
       DbConnection.  (Database.__init__): Change arguments and initialize
       dbconn here.
     * pyrseas/dbconn.py: Deleted/moved to pyrseas/lib.
     * pyrseas/dbobject/__init__.py (DbObjectDict.fetch): Remove connect
       but add rollback.
     * pyrseas/dbobject/operclass.py (OperatorClassDict._from_catalog):
       Add rollback's after fetchall's.
     * pyrseas/dbobject/table.py (ClassDict._from_catalog): Add rollback
       after fetchall.
     * pyrseas/dbtoyaml.py (main): Initialize Database directly. Get
       password here.
     * pyrseas/lib/__init__.py: New submodule.
     * pyrseas/lib/dbconn.py: Moved from pyrseas. Made more generic so
       that it can be used for other purposes.  Renamed _execute to
       execute, added close, commit and rollback methods.  Don't rollback
       after fetchone and fetchall.  Exclude set search_path calls.
     * pyrseas/testutils.py (PyrseasTestCase.database): Initialize
       Database directly.
     * pyrseas/yamltodb.py (main): Initialize Database directly. Get
       password here. Use DbConnection execute, rollback and commit
       instead of Psycopg.
    jmafc committed Apr 19, 2012
Commits on Apr 3, 2012
  1. Correct handling of function names in denormalizations.

     * pyrseas/extend/denorm.py (ExtCopyDenormColumn.apply): Call new
       adjust_name method to get correct new name before searching in
       current functions.
     * pyrseas/extend/function.py (CfgFunction.adjust_name): Adjust the
       function name if necessary.
     * tests/extend/__init__.py: Module to invoke all extender functions.
    jmafc committed Apr 3, 2012
  2. Refactor ExtendDatabase to eliminate CfgDicts.

     * pyrseas/extend/audit.py (CfgAuditColumn.apply): Eliminate db
       argument and rename cfgdb to extdb.  Call new add_func and add_lang
       functions.
     * pyrseas/extend/denorm.py (ExtDenormColumn.apply): Eliminate db
       argument and rename cfgdb to extdb.  Call new add_func and add_lang
       functions.
     * pyrseas/extend/schema.py (ExtSchema.apply): Eliminate db argument
       and rename cfgdb to extdb.  (ExtSChema.add_func): New method to add
       a function to a schema.
     * pyrseas/extend/table.py (ExtTable.apply): Eliminate db argument and
       rename cfgdb to extdb.
     * pyrseas/extenddb.py (ExtendDatabase.ExtDicts.__init__): Add
       initializations from removed CfgDicts.
       (ExtendDatabase.ExtDicts._link_current): Add pointer to current
       database.  (ExtendDatabase.ExtDicts.add_lang): New method to add a
       language if not present.  (ExtendDatabase._from_cfgmap): Replace
       self.cdb by self.edb.  (ExtendDatabase.apply): Remove CfgDicts
       initialization. Don't pass current database to schemas.apply.
    jmafc committed Apr 3, 2012
Commits on Mar 30, 2012
  1. Implement copy denormalizations.

     * pyrseas/extend/audit.py (CfgAuditColumnDict): Correct comment.
     * pyrseas/extend/denorm.py: New module for denormalizations.
     * pyrseas/extend/function.py: Add FUNC_COPY_DENORM template and
       copy_denorm to CFG_FUNCTIONS.  (CfgFunction.apply): Generalize text
       substitution to handle both audit columns and denormalizations.
       (CfgFunctionDict.__init__): Allow the name of a function to differ
       from the key used in the dict.
     * pyrseas/extend/schema.py (ExtSchemaDict.from_map): Add extdb as
       argument to tables.from_map.
     * pyrseas/extend/table.py (ExtTable.apply): Handle denormalizations.
       (ExtClassDict.from_map): Add extdb as argument.  Handle
       denormalizations.  (ExtClassDict.link_refs): New method to connect
       denormalizations to the respective tables.
     * pyrseas/extend/trigger.py: Add copy_denorm to CFG_TRIGGERS.
       (CfgTrigger.apply): Translate table_name in trigger name.
     * pyrseas/extenddb.py (ExtendDatabase.ExtDicts.__init__): Add
       denorms.  (ExtendDatabase.ExtDicts._link_refs): Call
       tables.link_refs to link denorms.
       (ExtendDatabase.ExtDicts._link_current): Call denorms.link_current
       to link to constraints.
       (ExtendDatabase.{from_extmap,_from_cfgmap): Python 3 fixes.
     * tests/extend/test_audit.py: Correct key in test_unknown_table.
     * tests/extend/test_denorm.py: New test module.
    jmafc committed Mar 30, 2012
Commits on Mar 22, 2012
  1. Add tests for error conditions in extension specifications.

     * pyrseas/extend/table.py (ExtClassDict.from_map): Raise error on
       unidentified attributes.
     * tests/extend/test_audit.py: Three new tests.
    jmafc committed Mar 22, 2012
Commits on Mar 21, 2012
  1. Add ExtensionToMapTestCase classes to streamline extension tests.

    * pyrseas/testutils.py (PostgresDb.process_extmap}): Refactor into
      new class.  ExtensionToMapTestCase: New class to derive test cases.
    * tests/extend/test_audit.py: Refactor tests using the new class.
    jmafc committed Mar 21, 2012
  2. Refactor PyrseasTestCase into two classes to streamline test construc…

    …tion.
    
     * pyrseas/testutils.py (PostgresDb.{execute_and_map,process_map}):
       Refactor to new classes.  (PyrseasTestCase.database): New method to
       instantiate Database object.  DatabaseToMapTestCase,
       InputMapToSqlTestCase: New classes to derive test cases.
     * tests/dboject/test_*.py: Refactor tests using the two new classes.
    jmafc committed Mar 21, 2012
Commits on Mar 20, 2012
  1. Fix dbtoyaml -t to output sequences owned by tables and schema descri…

    …ption.
    
     * pyrseas/database.py (Database.to_map): Add schemas and tables as
       arguments. Move modified code from dbtoyaml.py and adjust it to
       output owned sequences and schema description.
     * pyrseas/dbtoyaml.py: Remove code for trim schemas/tables and pass
       new arguments.
     * pyrseas/testutils.py (PostgresDb.execute_and_map): Add schemas and
       tables arguments and pass them on to to_map.
     * tests/dbobject/test_schema.py: New regression test.
     * tests/dbobject/test_table.py: New regression tests.
    
    Requires further work, but these changes suffice for r0.5.
    jmafc committed Mar 20, 2012
Commits on Mar 19, 2012
Commits on Mar 16, 2012
  1. Schema-qualify table when dropping columns. Fixes #26.

     * pyrseas/dbobject/column.py (Column.drop): Add schema qualification.
     * tests/dbobject/test_column.py: New regression test.
    jmafc committed Mar 16, 2012
  2. Normalize creation/dropping of schemas in tests.

     * pyrseas/testutils.py (PostgresDb.clear): Drop all non-public
       schemas at the beginning. Simplify remaining steps.
     * tests/dbobject/test_table.py: Move test_create_table_within_schema
       here from test_schema.py.
     * tests/dbobject/test_*.py: Remove special-case code to drop schema s1.
    jmafc committed Mar 16, 2012
Commits on Mar 14, 2012
  1. Additional changes to run dbextend under Python 3.

     * pyrseas/extend/*.py: Wrap dict items(), keys() and values() with
       list().
    jmafc committed Mar 14, 2012
  2. Additional changes to make Pyrseas run under Python 3.

     * pyrseas/database.py, pyrseas/dbobject/*.py: Wrap dict items(),
       keys() and values() with list().
     * pyrseas/dbobject/table.py (Sequence.to_map): Refine handling of int
       values.
     * tests/dbobject/__init__.py: Use explicit import statements.
    jmafc committed Mar 14, 2012
Commits on Mar 13, 2012
  1. Schema-qualify trigger functions based on the table's schema.

     * pyrseas/extend/audit.py (CfgAuditColumn.apply): Separate the schema
       from the function name before looking up in the configuration
       functions.
     * pyrseas/extend/trigger.py (CfgTrigger.apply): If the trigger
       procedure's schema is not the same as the table schema, assign the
       latter to the former.
     * tests/extend/test_audit.py: New test to verify fix.
    jmafc committed Mar 13, 2012
Commits on Mar 12, 2012
  1. Additional fix of queries to correctly exclude internal schemas.

     * pyrseas/testutils.py (PostgresDb.clear): Exclude pg_catalog and
       information_schema instead of joining to pg_roles.
    
    See commits 5a6e4bc and 8d92dd8 10 Oct 2011 for original changes.
    jmafc committed Mar 12, 2012
  2. Fix dbconn.py exception handling under Python 3. Fixes #25.

      Author: Chris Mayo
    jmafc committed Mar 12, 2012
Commits on Mar 10, 2012
  1. Update release files for 0.5.0.

    jmafc committed Mar 10, 2012
  2. Changes to foreign tables should use ALTER FOREIGN TABLE.

     * pyrseas/dbobject/column.py (Column.drop): Use objtype instead of
       hard-coding 'TABLE'.
     * pyrseas/dbobject/foreign.py (DbObjectWithOptions.diff_map): Invoke
       super method.
     * pyrseas/dbobject/table.py (Table.diff_map): Use objtype instead of
       hard-coding 'TABLE'.
     * tests/dbobject/test_foreign.py: New test to verify changes.
    
    Main changes by Ronan Dunklau, test by Joe Abbate.
    Ronan Dunklau committed with jmafc Mar 10, 2012
Commits on Mar 9, 2012
  1. Add support for modification of OPTIONS of FDW-related objects.

     * docs/foreign.rst: Document new helper class DbObjectWithOptions.
     * pyrseas/dbobject/foreign.py (DbObjectWithOptions): New helper
       class.  ({ForeignDataWrapper, ForeignServer, UserMapping,
       ForeignTable}): Derive from new helper class, add diff_map method
       (except for UserMapping) and change create method.
     * tests/dbobject/test_foreign.py: New tests to verify OPTIONS changes.
    jmafc committed Mar 9, 2012
  2. Correct order of creation/destruction of FDW-related objects.

     * pyrseas/database.py (Database.diff_map): Reorder calls and add
       calls to drop servers and FDWs.
     * pyrseas/dbobject/foreign.py (ForeignDataWrapperDict.diff_map): Mark
       wrappers to be dropped.  (ForeignDataWrapperDict._drop): New method
       to actually drop the wrappers.  (ForeignServerDict.diff_map): Mark
       servers to be dropped.  (ForeignServerDict._drop): New method to
       actually drop the servers.
     * tests/dbobject/test_foreign.py: New tests to verify order of
       multiple object creation/destruction.
    jmafc committed Mar 9, 2012
  3. Change user mappings to be listed under their servers in YAML output.

     * docs/foreign.rst: Add new automethods.
     * pyrseas/database.py (Database.link_refs): Link user mappings to
       their servers.  (Database.from_map): Remove mapping of input user
       mappings.  (Database.to_map): Remove mapping of user mappings.
     * pyrseas/dbobject/foreign.py (ForeignDataWrapperDict.from_map): Add
       newdb argument to downstream call.  (ForeignServer.to_map): New
       method to output user mappings as part of their servers.
       (ForeignServer.from_map): Handle user mappings as this level.
       (ForeignServerDict.link_refs): New method to link mappings and
       servers.  (ForeignServer.extern_key): Change output format.
       (Foreign.UserMappingDict.query): Add join to
       pg_foreign_data_wrapper and change sort order.
       (UserMappingDict.from_map): Change arguments and reorganize for new
       input.  (UserMappingDict.diff_map): Change key tuples.
     * tests/dbobject/test_foreign.py: Adjust tests to new input and
       output.
    jmafc committed Mar 9, 2012
Commits on Mar 7, 2012
  1. Change foreign servers to be listed under their wrappers in YAML output.

     * docs/foreign.rst: Add new automethods.
     * pyrseas/database.py (Database.link_refs): Link servers to their FD
       wrappers.  (Database.from_map): Remove mapping of input servers.
       (Database.to_map): Remove mapping of servers.
     * pyrseas/dbobject/foreign.py (ForeignDataWrapper.to_map): New method
       to output servers as part of their wrappers.
       (ForeignDataWrapperDict.from_map): Handle servers as this level.
       (ForeignDataWrapperDict.link_refs): New method to link servers and
       wrappers.  (ForeignServer.identifier): New method.
       (Foreign.ServerDict.query): Order by FDW first.
       (ForeignServerDict.from_map): Use FDW as part of key.
     * tests/dbobject/test_foreign.py: Adjust tests to new input and
       output.
    jmafc committed Mar 7, 2012
Commits on Mar 6, 2012
  1. Add support for PG 9.1 foreign tables.

     * docs/foreign.rst: Document new features.
     * pyrseas/database.py (Database.Dicts.__init__): Add dictionary for
       foreign tables. (Database.link_refs): Add foreign tables to
       schemas.link_refs and add ftables.link_refs. (Database.diff_map):
       Call diff_map for foreign tables.
     * pyrseas/dbobject/column.py (ColumnDict.query): Retrieve foreign
       table columns.
     * pyrseas/dbobject/foreign.py: Add ForeignTable and ForeignTableDict
       class to implement new features.  Add options_clause helper
       function to abstract creation of OPTIONS clauses.
     * pyrseas/dbobject/schema.py (Schema.to_map): Add ftables to
       objtypes.  (SchemaDict.from_map): Process foreign tables in input.
       (Schema.link_refs): Link foreign tables to their schemas.
     * pyrseas/testutils.py (PostgresDb.clear): Drop foreign tables if
       they exist.
     * tests/dbobject/test_foreign.py: Add new test cases for foreign
       tables.
    jmafc committed Mar 6, 2012
Commits on Mar 5, 2012