Skip to content
This repository

Oct 09, 2013

  1. Joe Abbate

    Update docs for previous changes for data_export/import [ci skip].

    authored October 09, 2013

Oct 08, 2013

  1. Joe Abbate

    Second phase of outputting static data files under a repository tree.

     * pyrseas/database.py (Database._link_refs): Pass datacopy config to
       schema link_refs; also, pass dictionary of dictionaries instead of
       individual ones.  (Database.to_map): Do not set opts.datacopy.
       (Database.diff_map): Call schemas.data_import of
       tables.data_import.
     * pyrseas/dbobject/__init__.py (DbObject.extern_filename): Add file
       extension argument, default 'yaml' to also support 'data'
       extension.
     * pyrseas/dbobject/schema.py (Schema.to_map): Handle datacopy by
       calling tables data_export.  (Schema.data_import): New method to
       handle datacopy in other direction.  (SchemaDict.link_refs):
       Refactor by accepting dictionary of dicts; also process new
       datacopy argument.  (SchemaDict.data_import): New method to iterate
       over schemas/tables to import.
     * pyrseas/dbobject/table.py (Table.data_export): New method to copy
       table data out to a file.  (Table.data_import): Add dirpath
       argument and expect table in specific path.  (ClassDict.link_refs):
       Refactor processing wih a local link_one function; remove datacopy
       handling.  (ClassDict): Removed data_export and data_import
       methods.
     * pyrseas/testutils.py (DatabaseToMapTestCase.to_map): For
       multiple_files and datacopy, merge metadata and data repo info.
       (DatabaseToMapTestCase.yaml_load): Read file from metadata_path.
       (InputMapToSqlTestCase.to_sql): For datacopy, merge data_path info.
     * tests/dbobject/test_static.py: Refactor tests to output/expect
       files in data_path config location.
    authored October 08, 2013

Sep 30, 2013

  1. Joe Abbate

    First phase of outputting static data files under a repository tree. …

    …[ci skip]
    
     * pyrseas/cmdargs.py (parse_args): Set config data_path.
     * pyrseas/database.py (Database.to_map): Move mkdir_parents higher so
       that it's available for multiple_files and datacopy.  Set up
       metadata_dir and datacopy under opts, so it's available downstream
       without having to pass them independently.
     * pyrseas/dbobject/__init__.py (DbObjectDict.to_map): Remove
       metadata_dir argument and use opts item instead.
     * pyrseas/dbobject/schema.py ({Schema, SchemaDict}.to_map): Remove
       metadata_dir argument and use opts item instead.
     * pyrseas/dbobject/table.py (Table.data_import,
       ClassDict.data_export): Add 'table.' prefix to file names.
     * pyrseas/testutils.py (DatabaseToMapTestCase): If datacopy in config
       arg, merge the TEST_DIR as files:data_path.
     * tests/dbobject/test_static.py: Add 'table.' prefix to FILE_PATH.
       In test_copy_static_table, expect the file in the corresponding
       schema directory under the repository tree.
    
    Skipping CI since the latter test will only be addressed in the second
    phase.
    authored September 30, 2013

Sep 25, 2013

  1. Joe Abbate

    Rename datacopy "load" to data_import and "copydata" to data_export.

      This is both for clarity and consistency and to avoid confusion
      between YAML load and data "load" (import).
    authored September 25, 2013
  2. Joe Abbate

    Remove metadata files after running functional autodoc test. [ci skip]

    authored September 25, 2013

Sep 24, 2013

  1. Joe Abbate

    Rename --directory to --multiple-files and add --repository command o…

    …ption.
    
     * config/config.yaml: Add default values for repository items.
     * docs/{cmdargs, config, dbtoyaml, yamltodb}.rst: Document new and
       changed options.
     * pyrseas/cmdargs.py (_repo_path): New helper function.
       (cmd_parser): Add --repository option.  (parse_args): Add helper
       function to refactor common code; store path to metadata under
       'files'.
     * pyrseas/database.py (Database.map_from_dir): Eliminate directory
       argument and and instead use metadata path.  (Database.to_map): Use
       metadata path instead of opts.directory and pass it downstream.
       Ensure we remove old schema directories.
     * pyrseas/dbaugment.py: Rename --directory to --multiple-files (still
       a TODO).
     * pyrseas/dbobject/{__init__, schema}.py ({DbObjectDict, Schema,
       SchemaDict}.to_map): Add metadata_dir argument and use it instead
       of opts.directory.
     * pyrseas/{dbtoyaml, yamltodb}.py: Rename --directory to
       --multiple-files.
     * pyrseas/testutils.py (DatabaseToMapTestCase.to_map): Rename
       directory argument to multiple_files.
       (DbMigrateTestCase.create_yaml): Merge with create_yaml_dir.
       (DbMigrateTestCase.migrate_target): Merge with migrate_target_dir.
       (AugmentToMapTestCase.to_map): Rename directory argument to
       multiple_files.
     * tests/dbobject/test_extern_file.py: Rename directory parameters to
       multiple_files.
     * tests/functional/test_autodoc_dir.py: Instead of saving empty
       metadata files, in the second phase, drop all schemas from source
       database and use that to migrate changes to target.
     * tests/test_config.py: Test repository items.
    authored September 24, 2013

Sep 11, 2013

  1. Joe Abbate

    Add test for specifying repository path in the user config file.

      Also, change other test_config tests to use 'dataload' key
      instead of schema/table approach.
    authored September 11, 2013
  2. Joe Abbate

    Allow the repository config path to be specified in the user config f…

    …ile.
    authored September 06, 2013
  3. Joe Abbate

    Add basic support for copying tables out to files.

     * pyrseas/database.py (Database.to_map): Call tables.copydata() if
       applicable.
     * pyrseas/dbobject/table.py (ClassDict.copydata): New method to
       iterate over tables and copy their data.
     * pyrseas/lib/dbconn.py (DbConnection.copy_to): New method to execute
       COPY TO commands.
     * pyrseas/testutils.py (DatabaseToMapTestCase.to_map): Add config
       parameter, after stmts, for symmetry with
       InputMapToSqlTestCase.diff_map.
     * tests/dbobject/test_{schema, table}.py: Changes to adjust for new
       parameter to to_map, above.
     * tests/dbobject/test_static.py: New test to verify this
       functionality.
    authored September 05, 2013
  4. Joe Abbate

    Rename 'dataload' config item to 'datacopy'.

    authored September 04, 2013
  5. Joe Abbate

    Add basic support for data loading in yamltodb.

     * pyrseas/database.py (Database._link_refs): If present, pass data
       loading configuration to tables.link_refs.  (Database.diff_map):
       Call tables.load if dataload present in config.
     * pyrseas/dbobject/table.py (Table.load): New method to generate SQL
       to load data.  (ClassDict.link_refs): Add dataload parameter and
       process it.  (ClassDict.load): New method to iterate over tables to
       be loaded.
     * pyrseas/testutils.py (InputMapToSqlTestCase.to_sql): Add config
       param and merge it with test configuration.
     * tests/dbobject/test_static.py: New test.
    authored August 30, 2013

Sep 10, 2013

  1. Joe Abbate

    Consolidate various Python 2/3 compatibility tests in pyrseas.lib.pyc…

    …ompat.
    
       pyrseas/lib/pycompat.py: Defines PY2 and strtypes.  PY2 is used to
       normalize various tests for Python 2.x.  strtypes is moved from
       pyrseas/dbobject/function.py so that it can be used elsewhere.
    
       Note: This breaks the autodoc and pagila functional tests under
       Python 2.x, due to use of yaml.safe_dump in change 1e4fa91.
    authored September 10, 2013

Sep 04, 2013

  1. Joe Abbate

    Some Unicode tweaks for Python 2.x.

     * pyrseas/dbobject/function.py (ProcDict.diff_map): Under Python 2,
       compare stmt to both str and unicode.
     * pyrseas/lib/dbconn.py: Under Python 2, register psycopg type UNICODE.
     * pyrseas/yamlutil.py: Under Python 2, use safe_dump for yamldump.
     * tests/relation/test_relvar.py: Import unicode_literals for Python 2.
    authored September 03, 2013

Aug 23, 2013

  1. Joe Abbate

    Handle a configuration file specified on the command line.

     * pyrseas/cmdargs.py (cmd_parser): Initialize global _cfg here.
       (parse_args): If config file specified, load it and merge it.
     * pyrseas/config.py (Config.__init__): Use new merge method.
       (Config.merge): New method to merge configuration dicts.
     * tests/test_config.py: New test.
    authored August 23, 2013

Aug 22, 2013

  1. Joe Abbate

    Additional configuration file tests.

    authored August 22, 2013

Aug 14, 2013

  1. Joe Abbate

    Handle column renaming in generated function source.

     * docs/dbaugment.rst: Add example of column renaming.
     * pyrseas/augment/column.py (CfgColumnDict.from_map): Regenerate the
       column translation table if column renames seen.
     * tests/augment/test_audit.py: Rework test_rename_column to alsoe
       verify the generated function map.
    authored August 14, 2013

Aug 13, 2013

  1. Joe Abbate

    Remove unimplemented dbaugment options (merge-spec and merge-config).…

    … [ci skip]
    authored August 13, 2013
  2. Joe Abbate

    Fix test_audit.py to conform to change made to config/config.yaml.

    authored August 13, 2013
  3. Joe Abbate

    In AugTable.apply, raise KeyError for nicer error reporting in dbaugm…

    …ent.
    authored August 13, 2013
  4. Joe Abbate

    Correct dbaugment example spec to use current audit_columns value. [c…

    …i skip]
    authored August 13, 2013
  5. Joe Abbate

    Add blank lines in function templates for nicer formatting of final s…

    …ources.
    authored August 13, 2013
  6. Joe Abbate

    Catch KeyError in dbaugment and exit with error message.

      The likely source of the KeyError is from a schema (AugSchemaDict) or
      table (AugClassDict) link_current method reporting that an object in
      the configuration file does not exist in the target database.
    authored August 13, 2013

Aug 12, 2013

  1. Joe Abbate

    Correct CfgFunctionSourceSegment to CfgFunctionSegment. [ci skip]

    authored August 12, 2013
  2. Joe Abbate

    Update Augmenter predefined augmentations and config objects. [ci skip]

       Also, remove delimited identifier issue.
    authored August 12, 2013

Aug 06, 2013

  1. Joe Abbate

    Add sys_only option to Config constructor to facilitate testing configs.

    authored August 06, 2013

Aug 05, 2013

  1. Joe Abbate

    Refactor connecting and clearing databases in testutils.py.

    authored August 05, 2013

Aug 01, 2013

  1. Joe Abbate

    Add PyrseasTestCase.config_options method to simplify derived classes.

    authored August 01, 2013

Jul 30, 2013

  1. Joe Abbate

    Change film functional tests to use 'genre' instead of 'category'.

    authored July 30, 2013

Jul 27, 2013

  1. Joe Abbate

    Restore 'script' line in .travis.yml (inadvertantly removed)

    authored July 27, 2013
  2. Joe Abbate

    Try locale-gen --no-archive as suggested in GH/travis-ci issue 1242.

    authored July 27, 2013
  3. Joe Abbate

    Suggested collation fix for Travis-CI.

      Apparently, when specifying locales or collations, fr_FR.UTF-8 is just as
      valid as fr_FR.utf8.  The latter form is the output from locale -a or
      localedef --list-archive and had worked before on Travis-CI (until June).
      However, it seems Travis-CI (or Ubuntu) now only accepts the former.
    authored July 26, 2013

Jul 26, 2013

  1. Joe Abbate

    Support for CLUSTER, yamltodb side. Fixes #58.

     * pyrseas/dbobject/constraint.py (Constraint.add): Add CLUSTER if
       applicable.  ({PrimaryKey, UniqueConstraint}.diff_map): Add CLUSTER
       or ALTER TABLE SET WITHOUT CLUSTER as needed.
       (ConstraintDict.from_map): Process cluster attribute.
     * pyrseas/dbobject/index.py (Index.create): Add CLUSTER if
       applicable.  (Index.diff_map): Add CLUSTER or ALTER TABLE SET
       WITHOUT CLUSTER as needed.  (IndexDict.from_map): Process cluster
       attribute.
     * tests/dbobject/test_{constraint, index}.py: Several new tests for
       clustering and unclustering.
    authored July 26, 2013

Jul 25, 2013

  1. Joe Abbate

    Support for CLUSTER, dbtoyaml side.

    authored July 25, 2013

Jul 24, 2013

  1. Joe Abbate

    Exclude 'pyrseas' schema from output maps, if present.

    authored July 24, 2013

Jul 23, 2013

  1. Joe Abbate

    Support ALTER COLUMN SET STATISTICS. Fixes #59.

     * pyrseas/dbobject/column.py (Column.to_map): Map statistics if not
       -1.  (Column.diff_map): Issue SET STATISTICS clause.
       (ColumnDict.query): Fetch attstattarget.
     * tests/dbobject/test_column.py: Two new tests.
    authored July 23, 2013
Something went wrong with that request. Please try again.