Changed gl_journal_entries replication key

[DownstreamDataTeam]

Description of change

Changed replication_keys and bookmark_field for gl_journal_entries stream from booking_date to creation_date.
The problem that this change solves is that, currently, we are missing journal entries records when transactions are reversed.

A test that was performed (without code changes):

1. set start_date to current date
2. reversed a transaction with a booking date in the past
3. synced the gl_journal_entries stream records.

The expected result: at least the reversed transaction records.
The actual result: didn't get the records for the reversed transaction.

After the change from booking_date to creation_date, the reversed transactions were synced.

Here are some json snippets ("start_date": "2021-10-18T00:00:00.000000Z"):
Result before code changes

{"type": "RECORD", "stream": "gl_journal_entries", "record": {"entry_id": "31986", "creation_date": "2021-10-17T21:25:32.000000Z", "entry_date": "2021-10-18T00:00:00.000000Z", "amount": "295.75", "booking_date": "2021-10-18T00:00:00.000000Z"}, "time_extracted": "2021-10-18T09:49:40.267352Z"}

{"type": "RECORD", "stream": "gl_journal_entries", "record": {"entry_id": "31987", "creation_date": "2021-10-17T21:25:32.000000Z", "entry_date": "2021-10-18T00:00:00.000000Z", "amount": "295.75", "booking_date": "2021-10-18T00:00:00.000000Z"}, "time_extracted": "2021-10-18T09:49:40.267352Z"}

{"type": "RECORD", "stream": "gl_journal_entries", "record": {"entry_id": "31990", "creation_date": "2021-10-17T21:26:03.000000Z", "entry_date": "2021-10-18T00:00:00.000000Z", "amount": "277.78", "booking_date": "2021-10-18T00:00:00.000000Z"}, "time_extracted": "2021-10-18T09:49:40.267352Z"}

{"type": "RECORD", "stream": "gl_journal_entries", "record": {"entry_id": "31991", "creation_date": "2021-10-17T21:26:03.000000Z", "entry_date": "2021-10-18T00:00:00.000000Z", "amount": "277.78", "booking_date": "2021-10-18T00:00:00.000000Z"}, "time_extracted": "2021-10-18T09:49:40.267352Z"}

Result after code changes

{"type": "RECORD", "stream": "gl_journal_entries", "record": {"entry_id": "32094", "creation_date": "2021-10-18T09:38:40.000000Z", "entry_date": "2021-10-15T00:00:00.000000Z", "amount": "38.33", "booking_date": "2021-10-15T00:00:00.000000Z"}, "time_extracted": "2021-10-18T09:48:41.300660Z"}

{"type": "RECORD", "stream": "gl_journal_entries", "record": {"entry_id": "32095", "creation_date": "2021-10-18T09:38:40.000000Z", "entry_date": "2021-10-15T00:00:00.000000Z", "amount": "1.67", "booking_date": "2021-10-15T00:00:00.000000Z"}, "time_extracted": "2021-10-18T09:48:41.300660Z"}

{"type": "RECORD", "stream": "gl_journal_entries", "record": {"entry_id": "32096", "creation_date": "2021-10-18T09:38:40.000000Z", "entry_date": "2021-10-15T00:00:00.000000Z", "amount": "40", "booking_date": "2021-10-15T00:00:00.000000Z"}, "time_extracted": "2021-10-18T09:48:41.300660Z"}

{"type": "RECORD", "stream": "gl_journal_entries", "record": {"entry_id": "32097", "creation_date": "2021-10-18T09:38:41.000000Z", "entry_date": "2021-10-13T00:00:00.000000Z", "amount": "20", "booking_date": "2021-10-13T00:00:00.000000Z"}, "time_extracted": "2021-10-18T09:48:41.300660Z"}

{"type": "RECORD", "stream": "gl_journal_entries", "record": {"entry_id": "32098", "creation_date": "2021-10-18T09:38:41.000000Z", "entry_date": "2021-10-13T00:00:00.000000Z", "amount": "18.33", "booking_date": "2021-10-13T00:00:00.000000Z"}, "time_extracted": "2021-10-18T09:48:41.300660Z"}

{"type": "RECORD", "stream": "gl_journal_entries", "record": {"entry_id": "32099", "creation_date": "2021-10-18T09:38:41.000000Z", "entry_date": "2021-10-13T00:00:00.000000Z", "amount": "1.67", "booking_date": "2021-10-13T00:00:00.000000Z"}, "time_extracted": "2021-10-18T09:48:41.300660Z"}

{"type": "RECORD", "stream": "gl_journal_entries", "record": {"entry_id": "32100", "creation_date": "2021-10-18T09:38:41.000000Z", "entry_date": "2021-10-15T00:00:00.000000Z", "amount": "40", "booking_date": "2021-10-15T00:00:00.000000Z"}, "time_extracted": "2021-10-18T09:48:41.300660Z"}

{"type": "RECORD", "stream": "gl_journal_entries", "record": {"entry_id": "32101", "creation_date": "2021-10-18T09:38:41.000000Z", "entry_date": "2021-10-15T00:00:00.000000Z", "amount": "40", "booking_date": "2021-10-15T00:00:00.000000Z"}, "time_extracted": "2021-10-18T09:48:41.300660Z"}

Rollback steps

• revert this branch

[cmerrick]

Hi @DownstreamDataTeam, thanks for your contribution!

In order for us to evaluate and accept your PR, we ask that you sign a contribution license agreement. It's all electronic and will take just minutes.

[gvillafanetapia]

This looks great we have the exact same issue!

[singer-bot]

You did it @DownstreamDataTeam!

Thank you for signing the Singer Contribution License Agreement.