Skip to content

fix(watcher): reindex dirty repos once per distinct state, not per poll#1045

Merged
DeusData merged 1 commit into
mainfrom
fix/937-watcher-dirty-signature
Jul 12, 2026
Merged

fix(watcher): reindex dirty repos once per distinct state, not per poll#1045
DeusData merged 1 commit into
mainfrom
fix/937-watcher-dirty-signature

Conversation

@DeusData

Copy link
Copy Markdown
Owner

Closes #937

Bug

check_changes() answered is the tree dirty where the watcher needs did the tree change: any dirty/untracked worktree returned true on every poll, so an idle repo with one uncommitted file re-triggered a full reindex — and its DB/artifact rewrite — every 5–60s cycle. Multiplied across several watched repos this produced the reported sustained >1 TB/day of disk writes.

A second, adjacent lost-update flaw: check_changes committed the new HEAD at check time, so a commit observed while the pipeline lock was busy (watcher.skip pipeline_busy returns success) was recorded as seen and never indexed.

Fix

Dirty-state signature. The watcher now hashes the dirty state — FNV-1a over the entries of git status --porcelain -uall -z plus each listed path's (size, mtime_ns) — and reindexes only when the signature changes:

  • same dirty state across polls → no reindex (the amplification killer)
  • editing an already-dirty file (size/mtime move), adding/removing a file, or reverting the tree to clean → new signature → exactly one reindex
  • -uall lists files inside untracked directories individually, so nested additions aren't hidden behind an unchanged ?? dir/ entry; -z gives unquoted, stable paths that also stat cleanly
  • submodule porcelain output is folded in on POSIX (paths re-rooted at the superproject), mirroring the old submodule dirty probe; git_is_dirty() is fully subsumed and removed
  • 0 is reserved for "clean"; a dirty-at-baseline tree still reindexes once, because the watcher cannot know whether that state reached the DB (e.g. server restart with a stale artifact) — at-least-once, never stale

Commit baselines only on success. check_changes no longer mutates last_head; observations are staged in pending_* fields and committed by poll_project only after a successful reindex. The cbm_index_fn contract gains a positive "skipped, retry" return used by the busy/shutdown paths in main.c — a busy-skip or failed run now retries on the next poll instead of silently losing the change.

Reproduce-first

Both RED on the old code, GREEN with the fix (tests/test_watcher.c):

  • watcher_dirty_state_reindexes_once_issue937 — dirty once → 1 reindex; 3 idle polls on the same state → still 1 (was 4); edit again → 2; revert to clean → 3; stable clean → 3.
  • watcher_failed_reindex_retries_issue937 — HEAD moves, first reindex attempt fails → retried and succeeds on the next poll (was: silently lost).
  • watcher_continued_dirty previously asserted one reindex per poll while dirty — that assertion was the Watcher can repeatedly re-index dirty repos and cause large disk write amplification #937 amplification; updated to the corrected contract.

Verification

  • Watcher suite 60/60; full local suite 6000 passed, 1 skipped, zero regressions
  • lint-ci clean
  • Poll cost is unchanged in shape (one git status + the existing POSIX submodule probe per poll, plus one stat() per dirty file)

A persistently dirty or untracked worktree re-triggered a full reindex
(and its DB/artifact rewrite) on every poll cycle - git_is_dirty()
answered 'is the tree dirty' where the watcher needs 'did the tree
CHANGE'. Idle repos with one uncommitted file produced sustained disk
write amplification (reported at 1 TB/day across several repos).

The watcher now tracks a dirty-state signature: FNV-1a over the
entries of git status --porcelain -uall -z plus each listed path's
(size, mtime_ns). Same signature -> no reindex; editing a dirty file,
adding/removing one, or reverting the tree to clean each produce a new
signature and trigger exactly one reindex. Submodule status is folded
in on POSIX, mirroring the old submodule dirty probe. 0 is reserved
for a clean tree; a dirty-at-baseline tree still reindexes once (the
watcher cannot know whether that state reached the DB, e.g. after a
server restart with a stale artifact).

Baselines (HEAD and signature) are now committed only after a
SUCCESSFUL reindex: check_changes stages observations in pending_*
fields, and busy-skips (index_fn > 0, e.g. pipeline lock held) or
failed runs leave the baselines untouched so the change is retried on
the next poll instead of being recorded as seen and silently lost -
the old code stored the new HEAD at check time, dropping any commit
that arrived while the pipeline was busy.

Closes #937

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
@DeusData DeusData added this to the 0.9.1-rc milestone Jul 12, 2026
@DeusData DeusData enabled auto-merge July 12, 2026 11:43
@DeusData DeusData merged commit 94be485 into main Jul 12, 2026
21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Watcher can repeatedly re-index dirty repos and cause large disk write amplification

1 participant