Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

StorageLoader: update manifest table as part of Redshift load transaction #2280

Closed
alexanderdean opened this issue Dec 31, 2015 · 0 comments
Closed

StorageLoader: update manifest table as part of Redshift load transaction #2280

alexanderdean opened this issue Dec 31, 2015 · 0 comments
Assignees
@alexanderdean
Labels
4. Storage
Milestone
R87 Chichen Itza

Comments

@alexanderdean
Copy link
Member

alexanderdean commented Dec 31, 2015 •
edited

As part of the transaction, we should write to a table, load_manifest, containing:

  • etl_tstamp
  • load_tstamp
  • event_count (assuming this can be retrieved from within transaction correctly)

To start with we will just populate this, but in the future it can be used to drive/validate the load...
@alexanderdean alexanderdean self-assigned this Dec 31, 2015
@alexanderdean alexanderdean modified the milestones: R8x [HAD] Stability and web model, Redshift load improvements #2 Nov 27, 2016
@alexanderdean alexanderdean changed the title StorageLoader: update manifest table as part of load transaction StorageLoader: update manifest table as part of Redshift load transaction Dec 6, 2016
alexanderdean added a commit that referenced this issue Jan 12, 2017
@alexanderdean
StorageLoader: update manifest table as part of Redshift load transac…
621f949 
…tion (close #2280)
alexanderdean added a commit that referenced this issue Jan 12, 2017
@alexanderdean
StorageLoader: update manifest table as part of Redshift load transac…
f03e4c0 
…tion (close #2280)
alexanderdean added a commit that referenced this issue Jan 15, 2017
@alexanderdean
StorageLoader: update manifest table as part of Redshift load transac…
961a7fd 
…tion (close #2280)
jbeemster pushed a commit that referenced this issue Jan 16, 2017
@alexanderdean @jbeemster
StorageLoader: update manifest table as part of Redshift load transac…
2f9c311 
…tion (close #2280)
alexanderdean added a commit that referenced this issue Jan 17, 2017
@alexanderdean
StorageLoader: update manifest table as part of Redshift load transac…
ab0fe8e 
…tion (close #2280)
alexanderdean added a commit that referenced this issue Jan 26, 2017
@alexanderdean
StorageLoader: update manifest table as part of Redshift load transac…
accb0c8 
…tion (close #2280)
alexanderdean added a commit that referenced this issue Feb 16, 2017
@alexanderdean
StorageLoader: update manifest table as part of Redshift load transac…
ac33130 
…tion (close #2280)
alexanderdean added a commit that referenced this issue Feb 16, 2017
@alexanderdean
StorageLoader: update manifest table as part of Redshift load transac…
bd6df27 
…tion (close #2280)
alexanderdean added a commit that referenced this issue Feb 16, 2017
@alexanderdean
StorageLoader: update manifest table as part of Redshift load transac…
b401774 
…tion (close #2280)
@chuwy chuwy mentioned this issue Aug 17, 2017
Closed
chuwy pushed a commit to snowplow/snowplow-rdb-loader that referenced this issue Sep 5, 2017
@alexanderdean @chuwy
StorageLoader: update manifest table as part of Redshift load transac…
3fa4794 
…tion (snowplow/snowplow#2280)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@alexanderdean alexanderdean

Labels
4. Storage
Projects
None yet
Milestone
R87 Chichen Itza
Development

No branches or pull requests
1 participant
@alexanderdean