Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

CDK: Incremental read should always produce state if any records were read #5832

Closed
keu opened this issue Sep 3, 2021 · 0 comments · Fixed by #5833
Closed

CDK: Incremental read should always produce state if any records were read #5832

keu opened this issue Sep 3, 2021 · 0 comments · Fixed by #5833
Assignees
@avida
Labels
CDK Connector Development Kit type/bug Something isn't working
Milestone
Connectors, Septe...

Comments

@keu
Copy link
Contributor

keu commented Sep 3, 2021 •
edited

It appears there is a bug after we introduced the _limit parameter. Whenever we specify some _limit and use incremental read the state is not produced anymore.
The reason is that the original logic relied on the fact that we will read at least a single slice completely, which is not the case anymore if slices are bigger than the value of _limit.

Screenshot 2021-09-03 at 09 31 09

Desired behavior:
Incremental read should always produce state if any records were read
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@avida avida

Labels
CDK Connector Development Kit type/bug Something isn't working
Projects
None yet
Milestone
Connectors, September 17th 2021
Development

Successfully merging a pull request may close this issue.

🐛 CDK: fix bug with limit parameter for incremental stream airbytehq/airbyte
3 participants
@avida @keu @sherifnada