Permalink
Switch branches/tags
Nothing to show
Commits on May 4, 2010
  1. If using Ruby 1.8, then clobber CSV with fasterCSV. This works since …

    adz committed May 4, 2010
    …in ruby 1.9 CSV *is* fasterCSV and codebase has always used fasterCSV -- which is now "CSV".
Commits on Mar 13, 2010
  1. Cleaned up configuration defaults handling

    Chris DiMartino
    Chris DiMartino committed Mar 13, 2010
Commits on Mar 5, 2010
  1. Extended block processor to be able to accept a config object and pas…

    ryan
    ryan committed Mar 5, 2010
    …s it thru
    
    to the block it calls.
Commits on Mar 4, 2010
  1. Added appropriate caching code to FillRowProcessor. Cleanup up some

    cdimartino committed Mar 4, 2010
    bugs in the implementation
Commits on Mar 3, 2010
  1. - Modified check_unique_processor to use a hash of the values_at value

    cdimartino committed Mar 3, 2010
    instead of the full record.  This should save on memory.
    
    - Added new processor: FillRowProcessor
      This is a row processor that allows for filling an existing source
      row with data merged from another data source.  It has options for
      overwriting the existing data on like columns.  It can also add new
      row columns from the alternate data source.
Commits on Feb 27, 2010
  1. Added new "FillRowProcessor" processor.

    Chris DiMartino
    Chris DiMartino committed Feb 27, 2010
      It's purpose is to fill row values given results from a query against
      a table.
Commits on Feb 23, 2010
  1. Default to MySQL for RunCodeRun.

    thbar authored and cdimartino committed Feb 12, 2010
  2. Trying to run the suite on RunCodeRun.

    thbar authored and cdimartino committed Feb 12, 2010
  3. make database destination table quoting database agnostic

    Darrell Fuhriman authored and cdimartino committed Feb 2, 2010
  4. create scd_required_fields. This allow us to not have to specify the …

    Darrell Fuhriman authored and cdimartino committed Feb 2, 2010
    …scd fields when specifying :unique in a destination, which was also done here
  5. update sql sanitize for newer versions of AR

    Darrell Fuhriman authored and cdimartino committed Feb 2, 2010
Commits on Feb 22, 2010
Commits on Feb 17, 2010
  1. Added back record.rb.

    cdimartino committed Feb 17, 2010
Commits on Jan 28, 2010
  1. Modified Engine:

    cdimartino committed Jan 28, 2010
      - Allow for multiple environments for database config:
        database.<env>.yaml
      - Allow setting of log level by ENV['LOG_LEVEL'] = x.
        0 = Debug
        1 = Info
        etc
      - Added millisecond precision to timestamp method
      - Use UTC for created_at/completed_at for Batch/Job
      - Better handling of paths for control files
      - Beginning (not functional - Syntax broken - Code does not run!!)
        work on multiprocessing transforms
  2. Added new feature to "check_exists" processor - overwrite. This will

    cdimartino committed Jan 28, 2010
    allow already existing records to be removed from the db and readded
    (data will continue into output file)
  3. - Moved from FasterCSV to CSV builtin (Ruby 1.9.1)

    cdimartino committed Jan 28, 2010
    - Fixed minor issues with pathname expecting a string
    - Added require 'forkify' (working on multiprocessing the transforms)
    - Added require 'pidify' (for etl script)
    - Fixed (I think) an issue with "new_records_only" in database_source.
      Was possible race condition (records with timestamp of same second as
      last run, but not yet committed when last run began, so missed)
  4. Added a "work_dir" to allow for output files to be written to a distinct

    cdimartino committed Jan 28, 2010
    location (timestamped).  This allows for concurrent runs against
    differernt databases from the same codebase (with the same named output
    files)
  5. Fixed assumed sort order issue in control/source. Was not finding

    cdimartino committed Jan 28, 2010
    correct previous .trig file in source dir
  6. - Added debugging output to control/destination

    cdimartino committed Jan 28, 2010
    - Modified control/destination to output 1 instead of 'true' for
      scd_latest_version_field
Commits on Dec 30, 2009
  1. add patch from Chris DiMartino

    aeden committed Dec 30, 2009
Commits on Apr 20, 2009
Commits on Jan 19, 2009
  1. just some small release items

    aeden committed Jan 19, 2009
Commits on Jan 15, 2009
Commits on Jan 7, 2009
  1. removing Record model

    mainej committed Jan 7, 2009
    No longer have SCDs, so it isn't relevant.
    
    Table dropped in migration 4 of etl/lib/etl/execution/migration.rb