When we encounter a truncated CSV get a new stateful query session #54
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description of change
When we encounter a truncated CSV get a new stateful query session.
The Zuora API can sometimes send us a non-rectangular CSV file (ie the number if items in a row is less than the number of columns). Testing has shown that when this happens in a stateful AQuA session then that session will from then on continuously return non-rectangular CSV.
This tap uses the
version
bookmark as part of the key for the AQuA query session. By generating a newversion
bookmark the tap ensures that on the next sync after encountering the non-rectangular CSV it will get a new query session.Manual QA steps
non-rectangular
exception. Running on this branch the tap encountered the exception once and fixed its state. On the subsequent run it was able to complete the sync.Risks
Rollback steps