Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cleanup abandoned state stores on startup #7720

Closed
AlanConfluent opened this issue Jun 23, 2021 · 1 comment
Closed

Cleanup abandoned state stores on startup #7720

AlanConfluent opened this issue Jun 23, 2021 · 1 comment
Assignees
Labels
bug fix-it-week operability Issues pertaining to running and operating KSQL, notably in production query-engine Issues owned by the ksqlDB Query Engine team

Comments

@AlanConfluent
Copy link
Member

Describe the bug
When queries are terminated, we attempt to call kafkastreams.close(timeout) and if we hit the timeout, we skip cleaning up the streams state via kafkastreams.cleanup(). This was added in #7717 since calling cleanup anyway resulted in an exception. If the query has been terminated, then on startup, when we know the query is no longer running, the state should be cleaned up.

To Reproduce
Requires significantly stressing the server so that it misses the the timeout in kafkastreams.close(timeout). This has happened when running tests on CI.

Expected behavior
State must be kept so that on startup, we know which state stores we can clean up. Might want to use a similar mechanism as was used for #4009, which was a local file of things to check up on implemented here: #6714

Actual behaviour
No cleanups.

@spena spena added the query-engine Issues owned by the ksqlDB Query Engine team label Jun 25, 2021
@agavra agavra added fix-it-week operability Issues pertaining to running and operating KSQL, notably in production and removed needs-triage labels Jun 28, 2021
@nateab
Copy link
Member

nateab commented Sep 30, 2021

Fixed by #7729

@nateab nateab closed this as completed Sep 30, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug fix-it-week operability Issues pertaining to running and operating KSQL, notably in production query-engine Issues owned by the ksqlDB Query Engine team
Projects
None yet
Development

No branches or pull requests

4 participants