Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Upgrading leaves deadline entries without jobs #13881

Closed
lenaschoenburg opened this issue Aug 14, 2023 · 0 comments · Fixed by #13886
Closed

Upgrading leaves deadline entries without jobs #13881

lenaschoenburg opened this issue Aug 14, 2023 · 0 comments · Fixed by #13886
Labels
kind/bug Categorizes an issue or PR as a bug severity/mid Marks a bug as having a noticeable impact but with a known workaround version:8.2.12 Marks an issue as being completely or in parts released in 8.2.12 version:8.3.0-alpha5 Marks an issue as being completely or in parts released in 8.3.0-alpha5 version:8.3.0-alpha6 Marks an issue as being completely or in parts released in 8.3.0-alpha6 version:8.3.0 Marks an issue as being completely or in parts released in 8.3.0

Comments

@lenaschoenburg
Copy link
Member

lenaschoenburg commented Aug 14, 2023 •
edited

Describe the bug

Upgrading a Zeebe cluster to a new version that includes fixes for #12797 and #13041, entries in the job deadline column family might be leaked, causing repeated error logs that a corresponding job cannot be found.
The cause is a change in behavior where we now expect no duplicate deadline entries and don't clean up duplicated or orphaned entries.

To Reproduce

Use 8.2.8, force the creation of duplicated deadline entries (probably happens with a long processing queue) and then upgrade to 8.2.9 or later.

Expected behavior

A migration cleans up orphaned entries so that they are removed from the state and do not cause error logs forever.

Environment:
Zeebe <=8.2.8 upgrades to > 8.2.8
Zeebe <= 8.1.? upgrades to > 8.1.?
Zeebe <= 8.0.? upgrades to > 8.0.?
@lenaschoenburg lenaschoenburg added kind/bug Categorizes an issue or PR as a bug severity/mid Marks a bug as having a noticeable impact but with a known workaround labels Aug 14, 2023
@lenaschoenburg lenaschoenburg mentioned this issue Aug 14, 2023
Merged
@megglos megglos added the version:8.2.12 Marks an issue as being completely or in parts released in 8.2.12 label Aug 18, 2023
@abbasadel abbasadel added version:8.3.0-alpha5 Marks an issue as being completely or in parts released in 8.3.0-alpha5 version:8.3.0-alpha6 Marks an issue as being completely or in parts released in 8.3.0-alpha6 labels Sep 7, 2023
@lenaschoenburg lenaschoenburg mentioned this issue Sep 19, 2023
Closed
@megglos megglos added the version:8.3.0 Marks an issue as being completely or in parts released in 8.3.0 label Oct 5, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
kind/bug Categorizes an issue or PR as a bug severity/mid Marks a bug as having a noticeable impact but with a known workaround version:8.2.12 Marks an issue as being completely or in parts released in 8.2.12 version:8.3.0-alpha5 Marks an issue as being completely or in parts released in 8.3.0-alpha5 version:8.3.0-alpha6 Marks an issue as being completely or in parts released in 8.3.0-alpha6 version:8.3.0 Marks an issue as being completely or in parts released in 8.3.0
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix(engine): cleanup orphaned job timeouts and backoffs on migration camunda/camunda
3 participants
@megglos @abbasadel @lenaschoenburg