Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clarify cdk parity test behavior #37407

Merged
merged 2 commits into from
Apr 19, 2024
Merged

Clarify cdk parity test behavior #37407

merged 2 commits into from
Apr 19, 2024

Conversation

erohmensing
Copy link
Contributor

@erohmensing erohmensing commented Apr 18, 2024

A small change to clarify that the check for "we get a state message from the stream" is not a temporary test behavior, it is permanent for the short term (in the long term, we'd want everything to work as concurrent does) as concurrent streams don't emit state messages, the concurrent source does.

Changes the test to check the state at the end of the test for concurrent sources, which is about the best we can do right now.

Copy link

vercel bot commented Apr 18, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Ignored Deployment
Name Status Preview Comments Updated (UTC)
airbyte-docs ⬜️ Ignored (Inspect) Visit Preview Apr 18, 2024 11:28pm

@octavia-squidington-iii octavia-squidington-iii added the CDK Connector Development Kit label Apr 18, 2024
Copy link
Contributor Author

erohmensing commented Apr 18, 2024

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @erohmensing and the rest of your teammates on Graphite Graphite

@erohmensing erohmensing marked this pull request as ready for review April 18, 2024 15:49
@erohmensing erohmensing requested a review from a team as a code owner April 18, 2024 15:49
Comment on lines +304 to +306
if constructor == _concurrent_stream:
assert hasattr(stream._cursor, "state")
assert str(stream._cursor.state) == "{'__ab_full_refresh_state_message': True}"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Weirdly enough you can even check this in the beginning before the _read because the cursor has a constant state. but we can check it here at the end

Later when we switch to incremental checkpointing as it wants, if we give it the right cursor, we should get proper state messages with the right cursor

@erohmensing erohmensing merged commit 0439cbc into master Apr 19, 2024
33 checks passed
@erohmensing erohmensing deleted the ella/fixtest branch April 19, 2024 15:47
strosek pushed a commit that referenced this pull request Apr 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CDK Connector Development Kit
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants