Skip to content

fix(store): prevent FileStore persistence starvation during continuous writes #2345

Description

@DeliciousBuding

Summary

FileStore's background loop resets its 50 ms debounce timer for every write notification. A sustained stream with no 50 ms quiet period can therefore postpone persistence indefinitely, including the earliest accepted message. This is a correctness/liveness issue in the supported --store-backend file and legacy --store-file paths, not a claim about production request frequency or JSON throughput.

On source base 81e02d403a48fdb5130c233ed2da9b8919d2a3fb, a regression using the real NewFile constructor and CreateProject/CreateThread/CreateItem methods feeds 40 further writes at half the existing interval. After 20 batch intervals, memory has 41 items and the actual disk restore path has zero. No final Flush/Close is allowed to repair the file before inspection. testing/synctest drives virtual scheduling time; the snapshot and restore I/O are real.

Observed failure: continuous writes starved FileStore persistence for 20 batch intervals: memory items=41, durable items=0.

The earlier #2256 / #2333 / #2334 work measured explicit full-snapshot flush cost and retained that storage format. It did not exercise background notification timing. This issue preserves full-snapshot persistence and does not reopen those throughput candidates. Assistant transcript emission is buffered until run completion; this is not a per-token transcript-write claim.

Scope / file ownership

One writer in an isolated task worktree. Allowed files:

  • edge-server/internal/store/file_store.go: arm the existing 50 ms timer on the first pending write; do not let later writes push the batch deadline forward; re-arm for later batches and stop the timer on shutdown.
  • edge-server/internal/store/file_store_persist_loop_test.go: deterministic real-file constructor-loop tests for sustained progress, coalescing, new batches, idle behavior and pending-close persistence.
  • docs/architecture/02-edge-server.md: describe bounded batching rather than trailing-edge-only debounce.

Interfaces / invariants / exclusions

No public API, snapshot schema, full-snapshot encoder, Sync/rename sequence, channel capacity, backend defaults or runtime settings change. Writes remain asynchronous/non-blocking with respect to disk persistence. The existing 50 ms value becomes a first-write batch scheduling window rather than an indefinitely extensible quiet period; it is not an fsync latency or durability-under-I/O-failure SLA. No idle polling writes and no per-notification synchronous flush. Preserve Flush, Close, LastPersistError and post-close memory-write behavior; do not add automatic retry policy or a new storage abstraction. No deployment, restart, production load test or real process-kill claim.

Acceptance

  • Reproduce durable starvation on unchanged production source using real file I/O and the constructor-owned loop.
  • Sustained writes make repeated durable progress before the stream becomes idle, without final Flush/Close.
  • Short bursts coalesce until the first-write deadline; later writes cannot postpone it; later batches still persist.
  • An idle store does not persist repeatedly and existing error reporting/close flushing behavior remains effective.
  • Complete relevant Windows and Linux race tests, vet/staticcheck and document checks.
  • All required PR checks pass on the final head.
  • Merge, synchronize source-only owner notes and clean the completed branch/worktree.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions