EmrEtlRunner: pass etl_tstamp into Hadoop Enrich as an argument

[alexanderdean]

Currently no way of knowing that a given event in Redshift or Postgres was generated by a specific ETL run.

[alexanderdean]

On disk, run ids look like this: run=2013-08-22-11-20-46

I'm wondering if we should store the run id in Redshift as:

1. string, run=2013-08-22-11-20-46
2. string, 2013-08-22-11-20-46
3. timestamp, 2013-08-22 11:20:46

#3 Might be most flexible for querying... Maybe we call it etl_ts rather than run_id.

@yalisassoon what do you reckon?

[yalisassoon]

I think timestamp would be fastest for querying and richest (because you
could analyze e.g. how regularly queries occur etc.)
Also agree etl_ts is a better name for this in the database

On Sat, Oct 26, 2013 at 2:44 PM, Alexander Dean notifications@github.comwrote:

On disk, run ids look like this: run=2013-08-22-11-20-46

I'm wondering if we should store the run id in Redshift as:

1. string, run=2013-08-22-11-20-46
2. string, 2013-08-22-11-20-46
3. timestamp, 2013-08-22 11:20:46

#3 #3 Might be most flexible
for querying... Maybe we call it etl_ts rather than run_id.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/396#issuecomment-27146532
.

[alexanderdean]

Moving this back so that 0.9.1 can be a Ruby-app-only release...

[alexanderdean]

Missing from CHANGELOG

[alexanderdean]

Fixed in 0.9.6, closing