Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

When we encounter a truncated CSV get a new stateful query session #54

merged 1 commit into from Mar 31, 2021


Copy link

@cosimon cosimon commented Mar 31, 2021

Description of change

When we encounter a truncated CSV get a new stateful query session.

The Zuora API can sometimes send us a non-rectangular CSV file (ie the number if items in a row is less than the number of columns). Testing has shown that when this happens in a stateful AQuA session then that session will from then on continuously return non-rectangular CSV.

This tap uses the version bookmark as part of the key for the AQuA query session. By generating a new version bookmark the tap ensures that on the next sync after encountering the non-rectangular CSV it will get a new query session.

Manual QA steps

  • Ran this on a connection that was receiving this non-rectangular exception. Running on this branch the tap encountered the exception once and fixed its state. On the subsequent run it was able to complete the sync.


Rollback steps

  • revert this branch

@cosimon cosimon merged commit b94e4f3 into master Mar 31, 2021
@cosimon cosimon deleted the fix/drop-query-session-on-truncated-csv branch March 31, 2021 18:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
None yet
None yet

Successfully merging this pull request may close these issues.

None yet

2 participants