Skip to content


Subversion checkout URL

You can clone with
Download ZIP
Commits on May 4, 2010
  1. If using Ruby 1.8, then clobber CSV with fasterCSV. This works since …

    …in ruby 1.9 CSV *is* fasterCSV and codebase has always used fasterCSV -- which is now "CSV".
Commits on Mar 13, 2010
  1. Cleaned up configuration defaults handling

    Chris DiMartino committed
Commits on Mar 5, 2010
  1. Extended block processor to be able to accept a config object and pas…

    ryan committed
    …s it thru
    to the block it calls.
Commits on Mar 4, 2010
  1. @cdimartino
  2. @cdimartino
  3. @cdimartino
Commits on Mar 3, 2010
  1. @cdimartino
  2. @cdimartino

    - Modified check_unique_processor to use a hash of the values_at value

    cdimartino committed
    instead of the full record.  This should save on memory.
    - Added new processor: FillRowProcessor
      This is a row processor that allows for filling an existing source
      row with data merged from another data source.  It has options for
      overwriting the existing data on like columns.  It can also add new
      row columns from the alternate data source.
Commits on Feb 27, 2010
  1. Added new "FillRowProcessor" processor.

    Chris DiMartino committed
      It's purpose is to fill row values given results from a query against
      a table.
Commits on Feb 23, 2010
  1. @thbar @cdimartino

    Default to MySQL for RunCodeRun.

    thbar committed with cdimartino
  2. @thbar @cdimartino
  3. @thbar @cdimartino
  4. @thbar @cdimartino
  5. @jlecour @cdimartino
  6. @cdimartino

    make database destination table quoting database agnostic

    Darrell Fuhriman committed with cdimartino
  7. @cdimartino

    create scd_required_fields. This allow us to not have to specify the …

    Darrell Fuhriman committed with cdimartino
    …scd fields when specifying :unique in a destination, which was also done here
  8. @cdimartino

    update sql sanitize for newer versions of AR

    Darrell Fuhriman committed with cdimartino
Commits on Feb 22, 2010
  1. @cdimartino
  2. @cdimartino
  3. @cdimartino
Commits on Feb 17, 2010
  1. @cdimartino

    Added back record.rb.

    cdimartino committed
Commits on Jan 28, 2010
  1. @cdimartino

    Modified Engine:

    cdimartino committed
      - Allow for multiple environments for database config:
      - Allow setting of log level by ENV['LOG_LEVEL'] = x.
        0 = Debug
        1 = Info
      - Added millisecond precision to timestamp method
      - Use UTC for created_at/completed_at for Batch/Job
      - Better handling of paths for control files
      - Beginning (not functional - Syntax broken - Code does not run!!)
        work on multiprocessing transforms
  2. @cdimartino
  3. @cdimartino

    Added new feature to "check_exists" processor - overwrite. This will

    cdimartino committed
    allow already existing records to be removed from the db and readded
    (data will continue into output file)
  4. @cdimartino

    - Moved from FasterCSV to CSV builtin (Ruby 1.9.1)

    cdimartino committed
    - Fixed minor issues with pathname expecting a string
    - Added require 'forkify' (working on multiprocessing the transforms)
    - Added require 'pidify' (for etl script)
    - Fixed (I think) an issue with "new_records_only" in database_source.
      Was possible race condition (records with timestamp of same second as
      last run, but not yet committed when last run began, so missed)
  5. @cdimartino

    Added a "work_dir" to allow for output files to be written to a distinct

    cdimartino committed
    location (timestamped).  This allows for concurrent runs against
    differernt databases from the same codebase (with the same named output
  6. @cdimartino

    Fixed assumed sort order issue in control/source. Was not finding

    cdimartino committed
    correct previous .trig file in source dir
  7. @cdimartino

    - Added debugging output to control/destination

    cdimartino committed
    - Modified control/destination to output 1 instead of 'true' for
  8. @cdimartino
  9. @cdimartino
Commits on Dec 30, 2009
  1. @aeden

    add patch from Chris DiMartino

    aeden committed
Commits on Apr 20, 2009
  1. @sasikumargn
Commits on Jan 19, 2009
  1. @aeden

    just some small release items

    aeden committed
Commits on Jan 15, 2009
  1. @mainej
Commits on Jan 7, 2009
  1. @mainej

    removing Record model

    mainej committed
    No longer have SCDs, so it isn't relevant.
    Table dropped in migration 4 of etl/lib/etl/execution/migration.rb
Something went wrong with that request. Please try again.