Skip to content

fix: prevent containers from sticking during shutdown - #1826

Merged
luke-lombardi merged 1 commit into
mainfrom
ll/cleanup-stopping-containers
Aug 3, 2026
Merged

fix: prevent containers from sticking during shutdown#1826
luke-lombardi merged 1 commit into
mainfrom
ll/cleanup-stopping-containers

Conversation

@luke-lombardi

@luke-lombardi luke-lombardi commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Summary by cubic

Fixes containers sticking during shutdown by reliably cancelling startup when a stop is issued and recovering from wedged workspace mounts (GeeseFS) after SIGKILL.

  • Bug Fixes

    • Replace “build” cancels with per-container cancels; cancel startup contexts on stop events and after worker reconnects.
    • Close startup race: cancelContainerIfAlreadyStopping checks local lifecycle state first, then repository with a 2s timeout.
    • Recover stuck shutdowns: abortStuckWorkspaceMount unmounts GeeseFS workspaces after SIGKILL timeout even if no runtime existed, without impacting other containers.
    • Swap cancelBuildcancelContainer in lifecycle handling; improved logging and tests for race and mount recovery.
  • Dependencies

    • Update github.com/beam-cloud/geesefs replacement to a newer commit.

Written for commit 496180a. Summary will update on new commits.

Review in cubic

@luke-lombardi
luke-lombardi requested a review from mernit August 3, 2026 22:55
@luke-lombardi
luke-lombardi merged commit 17bbb4e into main Aug 3, 2026
4 checks passed
@luke-lombardi
luke-lombardi deleted the ll/cleanup-stopping-containers branch August 3, 2026 22:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants