-
Couldn't load subscription status.
- Fork 4
Closed
Labels
Description
Context / Goal
A core feature of the design is to be able to take data from a source DB, load the rows and hash the data and then persist them to a reconciliation DB in hashed form, that can be later compared to target.
We can start by being able to stream data
Expected Outcome
- Be able to execute an arbitrary dataset query, expressed as SQL, against a configured source DB
- Hash the column values other than a designated
MigrationKeyusing SHA256- start off with handling for columns that can be mapped easily to core Java types (boolean, integer numerics, floating points, strings)
- Be able to persist the
(migrationKey, sourceHashValue)to a 3rd "reconciliation DB" owned by the reconciliation tool
Out of Scope
- Loading data from the target DB (Ability to persist/merge hashed target data into reconcilation DB #18)
- Producing a report of the results (Persist reconcilation report results in a queryable manner #12)
- performance testing (Exploratory performance testing of load and persist #26)
Additional context / implementation notes
- Assume using an initial approach based on Spike use of r2dbc to stream data off databases #20