Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix partitioned state saving issue #37389

Merged
merged 1 commit into from
Apr 18, 2024

Conversation

maxi297
Copy link
Contributor

@maxi297 maxi297 commented Apr 18, 2024

What

Currently, we save the state as if the partition sync was always successful. If the partition failed, we should not close the partition

How

By adding a parameter that mentions if the sync was successful in the PartitionCompleteSentinel.

User Impact

The slices that are failing will retry on the next sync from now on. As for the other issues, we have two syncs that failed with system_errors since version 2.5.0. I'll identify the workspace and communicate with TCS for a reset. I'll also check the config errors but from a quick look at it, all of them did not occur during a sync so we should be good there.

Can this PR be safely reverted and rolled back?

  • YES 💚
  • NO ❌

@maxi297 maxi297 requested a review from a team as a code owner April 18, 2024 00:42
Copy link

vercel bot commented Apr 18, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Ignored Deployment
Name Status Preview Updated (UTC)
airbyte-docs ⬜️ Ignored (Inspect) Apr 18, 2024 0:42am

@octavia-squidington-iii octavia-squidington-iii added the CDK Connector Development Kit label Apr 18, 2024
@maxi297 maxi297 requested a review from girarda April 18, 2024 01:11
Copy link
Contributor

@girarda girarda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm ✅

@maxi297 maxi297 merged commit d2c8e63 into master Apr 18, 2024
33 checks passed
@maxi297 maxi297 deleted the maxi297/fix_partitioned_state_saving_issue branch April 18, 2024 02:57
strosek pushed a commit that referenced this pull request Apr 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CDK Connector Development Kit
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants