Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Epic] Materialize does not cleanup orphaned resources #15654

Closed
antiguru opened this issue Oct 24, 2022 · 0 comments · Fixed by #16200
Closed

[Epic] Materialize does not cleanup orphaned resources #15654

antiguru opened this issue Oct 24, 2022 · 0 comments · Fixed by #16200
Assignees
Labels
A-ADAPTER Topics related to the ADAPTER layer A-compute Area: compute C-bug Category: something is broken

Comments

@antiguru
Copy link
Member

The Materialize does not cleanup resources that it acquired but forgot about. The orchestrator exposes an API to list all services it created, but Materialize does not use this API to reconcile its expected resources with the ones currently allocated.

Change this such that on startup Materialize allocates its expected services, and afterwards reconciles the set of existing services with what it expects to have. This needs to take into account the possibility of multiple environmentd processes concurrently initializing. Hence, only resources that must have existed before initializing can be reclaimed; later resources must come from a newer environmentd. We can use an ever-increasing resource counter to determine whether a resource is newer or older than the current environmentd.

@antiguru antiguru added C-bug Category: something is broken A-compute Area: compute A-ADAPTER Topics related to the ADAPTER layer labels Oct 24, 2022
@antiguru antiguru changed the title Materialize does not cleanup orphaned resources [Epic] Materialize does not cleanup orphaned resources Nov 1, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-ADAPTER Topics related to the ADAPTER layer A-compute Area: compute C-bug Category: something is broken
Projects
None yet
2 participants