Skip to content

fix(plugin): debounce Stop-hook curate to prevent stampede - #62

Merged
ovidb merged 1 commit into
mainfrom
fix/curate-stop-hook-stampede
Apr 17, 2026
Merged

fix(plugin): debounce Stop-hook curate to prevent stampede#62
ovidb merged 1 commit into
mainfrom
fix/curate-stop-hook-stampede

Conversation

@ovidb

@ovidb ovidb commented Apr 17, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Add per-session cooldown gate in plugin/scripts/stop-curate.sh — Claude Code's Stop hook fires per agent turn, not per session, so long sessions stampeded rememora curate + its two claude -p children. Tunable via REMEMORA_CURATE_COOLDOWN_SECS (default 300s, set to 0 to disable).
  • Rewrite plugin/scripts/session-end.sh to run a final catch-up curate and clean the session's lockfile, guaranteeing the tail of a session is captured even if the last Stop fire was inside the cooldown window.
  • Document the env var in README.md.

Why

Observed in a real session before the fix: 37 concurrent claude -p processes and ~105 total rememora-related processes piled up from a single multi-hour Claude Code session. Each Stop hook fork stays alive for the duration of a curate call (Haiku signal gate + optionally Sonnet AUDN curator), which can be minutes — new fires stack on top before earlier ones finish. Result: CPU burn, API token burn, and a voice-notification hook chain firing in lockstep.

Design notes

  • Lockfile at ${TMPDIR:-/tmp}/rememora-curate-${SESSION_ID}.last — per-session means each fresh Claude Code session starts with an empty bucket (no cross-session surprises); /tmp auto-cleans on reboot so no long-lived state accumulates.
  • touch before the fork — rapid-fire second fires see the fresh mtime and exit before spawning anything.
  • Portable statstat -f %m (BSD/macOS) with a stat -c %Y (GNU/Linux) fallback, since rememora targets both.
  • SessionEnd as safety net — if the session ends inside the cooldown window, the tail would otherwise be missed; SessionEnd now runs a final curate pass (once, no cooldown needed) and unlinks the lockfile.
  • Not changing curate's CLI — a --cooldown-secs flag on the Rust side would be dead weight; the shell gate already prevents the process from spawning when throttled.

Test plan

  • bash -n plugin/scripts/{stop-curate,session-end}.sh — syntax clean
  • Cold run → lockfile created, curate runs
  • Second run within cooldown → mtime unchanged, curate NOT invoked
  • REMEMORA_CURATE_COOLDOWN_SECS=0 → gate bypassed, curate runs again
  • SessionEnd hook → curate runs + lockfile removed
  • No straggler processes after test harness completes
  • Install patched plugin + run a 10+ turn real session → pgrep -f "claude -p" | wc -l stays ≤2 (to be verified after merge & plugin update)
  • Linux syntax sanity check (BSD vs GNU stat) — fallback path untested on CI

Closes

Fixes a latent issue reported today — no existing issue, but I can file one if desired.

Claude Code's Stop hook fires after every agent turn, not per session.
Each fire forked `rememora curate`, which spawns two `claude -p`
subprocesses (Haiku signal gate + Sonnet curator). Long sessions piled
up dozens of concurrent curate runs — observed ~37 `claude -p` and 100+
rememora processes in a single session — burning CPU and API tokens.

Fix: per-session cooldown gate at the hook script layer.

- plugin/scripts/stop-curate.sh: skip if the session's lockfile
  (/tmp/rememora-curate-$SESSION_ID.last) was touched within
  REMEMORA_CURATE_COOLDOWN_SECS (default 300). BSD/GNU stat both
  supported; lockfile is auto-cleaned from /tmp on reboot.
- plugin/scripts/session-end.sh: run a final catch-up curate and
  remove the lockfile, so the tail of a session is never lost even
  if the last Stop fire was inside the cooldown window.
- README.md: document REMEMORA_CURATE_COOLDOWN_SECS.

Verified via simulation: cold run touches lockfile + runs curate; a
second run within cooldown exits early with unchanged mtime;
cooldown=0 bypasses the gate; SessionEnd runs curate and unlinks
the lockfile.
@ovidb
ovidb merged commit 0804af8 into main Apr 17, 2026
2 checks passed
@ovidb
ovidb deleted the fix/curate-stop-hook-stampede branch April 17, 2026 21:22
ovidb added a commit that referenced this pull request Apr 17, 2026
The 300s per-session lockfile added in #62 gated frequency between curate
*starts* but not concurrency between start and finish. Real curate runtime
(dominated by the `claude -p` signal-detector) routinely exceeds 5 minutes;
when runtime > cooldown, every Stop hook in the overlap window spawns a
second curate while the first is still running.

Observed in the wild: 110 concurrent `rememora curate` procs + 55 `claude -p`
children across 29 live sessions (avg 2.1 in-flight per session, not 1).

Fix adds a primary concurrency gate via `pgrep -f` matched on the session
UUID embedded in the jsonl path — kernel truth, not a timestamp proxy.
Also moves the cooldown stamp `touch` to after curate returns, so the
frequency window measures idle-since-finished instead of start-to-start.

Verified: 5 rapid Stop fires for one session → 1 in-flight curate (was: N).

Plugin: 1.0.1 → 1.0.2.
ovidb added a commit that referenced this pull request Apr 17, 2026
…2) (#63)

The 300s per-session lockfile added in #62 gated frequency between curate
*starts* but not concurrency between start and finish. Real curate runtime
(dominated by the `claude -p` signal-detector) routinely exceeds 5 minutes;
when runtime > cooldown, every Stop hook in the overlap window spawns a
second curate while the first is still running.

Observed in the wild: 110 concurrent `rememora curate` procs + 55 `claude -p`
children across 29 live sessions (avg 2.1 in-flight per session, not 1).

Fix adds a primary concurrency gate via `pgrep -f` matched on the session
UUID embedded in the jsonl path — kernel truth, not a timestamp proxy.
Also moves the cooldown stamp `touch` to after curate returns, so the
frequency window measures idle-since-finished instead of start-to-start.

Verified: 5 rapid Stop fires for one session → 1 in-flight curate (was: N).

Plugin: 1.0.1 → 1.0.2.
ovidb added a commit that referenced this pull request Apr 17, 2026
plugin/README.md never mentioned the Stop hook or stop-curate.sh — they've
been shipping since #62 but the README still described a SessionStart +
SessionEnd plugin. Adds the missing hook to the feature list, workflow,
and structure tree.

README.md's REMEMORA_CURATE_COOLDOWN_SECS note described the cooldown as
the anti-stampede guarantee, which was accurate under v1.0.1's single-gate
design but is no longer correct. Under v1.0.2 the concurrency gate (kernel-
level pgrep) is the guarantee, and the cooldown is a secondary frequency
limiter that measures idle-since-finished. Clarified accordingly.

No version bump — docs-only follow-up to plugin v1.0.2 / #63.
ovidb added a commit that referenced this pull request Apr 17, 2026
plugin/README.md never mentioned the Stop hook or stop-curate.sh — they've
been shipping since #62 but the README still described a SessionStart +
SessionEnd plugin. Adds the missing hook to the feature list, workflow,
and structure tree.

README.md's REMEMORA_CURATE_COOLDOWN_SECS note described the cooldown as
the anti-stampede guarantee, which was accurate under v1.0.1's single-gate
design but is no longer correct. Under v1.0.2 the concurrency gate (kernel-
level pgrep) is the guarantee, and the cooldown is a secondary frequency
limiter that measures idle-since-finished. Clarified accordingly.

No version bump — docs-only follow-up to plugin v1.0.2 / #63.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant