Skip to content

fix(gateway): batch SSH session cleanup writes - #3000

Open
drew wants to merge 1 commit into
mainfrom
codex/2999-batch-ssh-cleanup/drew
Open

fix(gateway): batch SSH session cleanup writes#3000
drew wants to merge 1 commit into
mainfrom
codex/2999-batch-ssh-cleanup/drew

Conversation

@drew

@drew drew commented Aug 28, 2026

Copy link
Copy Markdown
Collaborator

Summary

Batch SSH-session persistence cleanup into bounded set-based deletes so sandbox teardown and the session reaper do not issue one database write per session. This addresses the gateway-side contention risk reported in #2999 without changing SQLite journal settings, timeouts, or session ownership schema.

Related Issue

Part of #2999.

This is PR 1 of 2. The issue does not currently carry an accepted-state label; the direct implementation and ready-for-review request authorized this submission, while maintainers retain the acceptance and merge decision.

Changes

  • add a backend-neutral delete_many store operation with fixed 128-ID chunks for SQLite and Postgres
  • paginate SSH-session discovery before mutation and batch sandbox-owned cleanup writes
  • batch expired/revoked session reaping and replace token-bearing per-session errors with aggregate telemetry
  • add persistence, pagination, type-isolation, reaper, and file-backed SQLite concurrency coverage
  • document the bounded owned-record cleanup invariant

Testing

  • mise run pre-commit
  • mise run test (review rerun: 1,443 server tests passed)
  • focused persistence, compute cleanup, and reaper regression tests
  • exact OpenClaw fixture reproduced on main, then passed on this PR and the combined branches
  • mise run e2e:docker
  • mise run ci passes locally — all Rust, Python, TypeScript, lint, compile, and policy checks passed; three unrelated Go gateway fixture tests fail because this workstation's hard-coded /etc/openshell/gateways/default is included as an unexpected system fixture

Review

A full line-by-line review covered both persistence backends, chunk bounds, empty/duplicate/missing IDs, object-type isolation, pagination before mutation, partial-chunk idempotency, cleanup/reaper telemetry, secret-safe logging, tests, and architecture documentation. No actionable findings remained.

Checklist

  • Follows Conventional Commits
  • Commits are signed off (DCO)
  • Architecture docs updated (if applicable)

Part of #2999.

Signed-off-by: Drew Newberry <anewberry@nvidia.com>
@copy-pr-bot

copy-pr-bot Bot commented Aug 28, 2026

Copy link
Copy Markdown

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

@drew

drew commented Aug 28, 2026

Copy link
Copy Markdown
Collaborator Author

🏗️ build-from-issue-agent

Issue fixture verification

Verified commit 36e598574fcef3b122e7fa09037dc42e21e94ca2 with the exact OpenClaw fixture referenced by #2999.

  • Untouched main: fixture failed 1 of 5 tests; gateway emitted 1,151 individual SSH-session DELETE statements. The largest 568-session cleanup took about 8.49 seconds.
  • This PR: fixture passed all 5 tests; the same cleanup load used 13 bounded statements and zero individual SSH-session DELETE statements. The 568-session cleanup used 5 batches and took about 51 ms.
  • Temporary combined build with fix(cli): wait for terminal sandbox deletion #3001: all 5 tests passed with the same bounded cleanup shape.
  • No SQLite read failure occurred locally on baseline or PR, so this attests removal of the reproduced serial-write contention mechanism rather than claiming the intermittent INTERNAL symptom was observed.

Full evidence: #2999 (comment)

@drew

drew commented Aug 28, 2026

Copy link
Copy Markdown
Collaborator Author

Full code review completed.

  • Reviewed both persistence backends, bounded query construction, type isolation, pagination-before-mutation, cleanup/reaper idempotency and telemetry, log safety, tests, and architecture docs.
  • No actionable findings remained.
  • Review validation passed: mise run pre-commit, the full mise run test suite (including 1,443 server tests), focused bulk-delete/cleanup/reaper tests, Docker E2E, and the exact Control-plane read failures during sandbox SSH-session cleanup; delete acknowledgment can precede durable cleanup #2999 OpenClaw fixture.
  • The exact fixture changed the SSH cleanup from 1,151 individual writes on main to 13 bounded deletes here; the largest 568-session cleanup completed in about 51 ms.

Ready for maintainer review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant