fix(diff): drop temp-index recipe from working_tree_diff_stats_with_untracked

[worktrunk-bot]

Problem

working_tree_diff_stats_with_untracked (HEAD± with untracked, used by wt list --full and wt list statusline) computes diff stats via a copy of the real index plus git add --intent-to-add . plus git diff --shortstat HEAD. That recipe intermittently drops the modification of a tracked file from the downstream diff: on a "1 modified line + 2 untracked" repro, added came back 2 instead of 3 once every few hundred iterations on the GitHub Actions Linux runner.

That's exactly the shape the failing test reported on the failed run (and the same test failed identically on the prior e4dd5917 run — same test, same left: 2, right: 3):

assertion `left == right` failed: 1 modified line + 2 untracked
  left: 2
 right: 3

I reproduced it locally with a tight bash loop — about 0.5% failure rate. The failing shape is 1 file changed, 2 insertions(+) (untracked.txt only — the tracked.txt +1/-1 modification is lost). Inspecting the temp index at failure showed the tracked.txt entry kept its pre-modification hash and mtime, and the follow-up diff treated it as unchanged.

Solution

Compute tracked stats with the existing working_tree_diff_stats() (git diff --shortstat HEAD) and count lines in git ls-files --others --exclude-standard paths directly, in Rust. No temp index, no git add --intent-to-add — the race surface disappears.

The line-count helper mirrors git diff --shortstat accounting:

• Skip non-regular files (symlinks, gitlinks, sockets).
• Skip binary content (NUL byte in the first 8 KiB — same heuristic as buffer_is_binary in xdiff-interface.c).
• For text, count newlines plus one more if the file ends without a trailing newline.

A new unit test exercises empty, with/without trailing newline, single-char, single-newline, binary-with-NUL, and missing-path cases. The existing real-index-preservation assertion still holds — structurally rather than mechanism-coupled, since no index manipulation happens at all.

Scope: wt step diff and WorkingTreeConflictsTask keep the TempIndex helper for their git diff <merge-base> and git write-tree work. The race I reproduced is specific to the add --intent-to-add . → diff --shortstat HEAD pairing that this method used.

Testing

• cargo test --lib working_tree — 13 tests pass including the previously-flaking one.
• cargo nextest run --lib — full 1229-test lib suite clean.
• 200 sequential reruns of the originally-failing test under the fix: 200/200 pass.
• 5 sequential full-suite nextest runs: clean.
• cargo clippy --all-targets -- -D warnings and cargo fmt --check clean.
• The 10 nu-related integration test failures observed locally are environment-specific (this runner lacks nu on PATH); CI installs it via hustcer/setup-nu.

---

Automated fix for failed run

[worktrunk-bot]

Self-review of an automated CI fix. The diagnosis (intent-to-add followed by git diff racing with the stat cache to drop a tracked-file modification) lines up with the failing-run shape, and dropping that pairing for direct counting eliminates the race surface. Tests cover the trailing-newline, single-char, empty, and binary-NUL accounting cases, and the integration test still pins the real-index invariant structurally.

One docstring nit inline. Not blocking.

(Self-authored — leaving the merge decision to a maintainer.)

[worktrunk-bot]

The implementation uses 8000 bytes (content.len().min(8000) on line 654), matching git's FIRST_FEW_BYTES = 8000 in xdiff-interface.c. 8 KiB is 8192 bytes — the docstring overstates the probe by ~2%.

Suggested change

	/// byte in the first 8 KiB, mirroring `buffer_is_binary` in
	/// byte in the first 8000 bytes, mirroring `buffer_is_binary` in

[max-sixty]

but this is a worse way of doing it, we'd rather push these down to git rather than handle them ourselves

why is the existing appreach with git failing? is there a race condition

[worktrunk-bot]

Yes — it's a real race, and you're right that I jumped to the wrong shape of fix. Just reproduced the old recipe on this runner (git 2.54.0, Linux): 1 failure in 500 iterations, same 1 file changed, 2 insertions(+) shape as the failing CI run.

What I see at the moment of failure, with git ls-files --stage --debug against the temp index after git add --intent-to-add .:

100644 3367afdbbf91… 0  tracked.txt
  mtime: 1779555219:992104538     ← stale (matches the pre-modification stat)
  size: 4

The actual file's mtime is 1779555219:998104567. --intent-to-add documents that it doesn't apply to entries already in the index — so git add --intent-to-add . skipped tracked.txt entirely, leaving both its blob hash and its stat info pointing at the pre-modification state. The followup git diff --shortstat HEAD then misses the tracked-file modification (same hash as HEAD; stat appears clean — same second, only nanoseconds differ — depending on which core.checkStat path git takes). Output: 1 file changed, 2 insertions(+) — exactly the failing-CI shape.

The temp-index recipe was inherently coupling two surfaces that don't compose cleanly: --intent-to-add is for new files, and the only reason it touched tracked entries at all was the implicit pathspec .. So agreed, my fix is the wrong direction.

A cleaner all-git alternative that I just tested 1000/1000 clean on the same runner:

let mut stats = self.working_tree_diff_stats()?;  // git diff --shortstat HEAD, tracked only
for untracked in self.run_command(&["ls-files", "--others", "--exclude-standard", "-z"])?
    .split('\0').filter(|s| !s.is_empty())
{
    let out = self.run_command(&[
        "diff", "--shortstat", "--no-index", "--", "/dev/null", untracked,
    ])?;
    stats.added += LineDiff::from_shortstat(&out).added;
}

All accounting stays in git (including binary detection, no-trailing-newline handling, symlinks); no temp index, no stat-cache interaction. The cost is one extra git diff per untracked file, but the hot caller is wt list statusline --claude-code where the typical untracked count is 0–2.

Want me to rework the PR to that, or would you prefer a different angle (e.g. keep the old recipe but git update-index --refresh before the diff)? Happy to retire this PR and open the replacement.