Summary
In a repo with large Git LFS files, T3 Code's background git activity (checkpoint git add -A plus frequent git status --porcelain=2 --branch polls) filled .git/lfs/tmp with roughly 150 GB of stranded temp files in about 9 hours. The trigger was a stale .git/index.lock, but T3 kept hammering the repo through the failure all day instead of backing off, and never cleaned up the partial files it left behind.
Environment
- T3 Code (Alpha), macOS 15.6 (Darwin 24.6.0)
- git 2.50.1 (Apple), git-lfs 3.7.1
- Repo with LFS-tracked files up to ~160 MB in the worktree
What happened
- A stale, empty
.git/index.lock appeared at 08:44 (likely left by an interrupted git process; T3's checkpointing runs git add -A on this repo constantly, so it is the most frequent lock holder).
- With the lock in place,
git update-index --refresh fails, so git can never cache clean-filter results in the index.
- Every T3 poll (
git status --porcelain=2 --branch, run every few seconds) therefore re-ran git lfs clean over every stat-dirty LFS file. Each pass wrote fresh multi-hundred-MB temp files into .git/lfs/tmp and then failed at the index write, stranding them.
- By evening,
.git/lfs/tmp held ~150 GB. After I wiped it, it regrew at roughly 4 GB/hour until I found and removed the stale lock.
Confirmed the parentage with ps: the headless git status and git-lfs filter-process processes were children of T3 Code (Alpha).app/Contents/Resources/app.asar/apps/server/dist/bin.mjs. The temp file sizes matched the repo's large LFS files exactly (159 MB, 144 MB x2, 89 MB x2).
Removing the orphaned lock fixed it immediately: the next git status cached its results and .git/lfs/tmp now stays at 0 bytes, including through checkpoint runs.
Suggested fixes
Any one of these would have contained the damage; together they'd prevent it:
- Recover from stale locks. Before polling or checkpointing, if
.git/index.lock exists, is old (minutes, not seconds), and no live process holds it (lsof comes back empty), remove it or at least stop polling and surface a warning. Git itself prints "another git process seems to be running" hints for this case.
- Back off on repeated failure. The status poll failed the same way for ~9 hours at a few-second cadence. Any error backoff would have cut the damage by orders of magnitude.
- Clean up
.git/lfs/tmp. Stranded partials there are cheap to detect (files older than a few minutes with no open file handle). Even git-lfs itself doesn't reap them aggressively, so a periodic sweep in repos T3 manages would help.
- Consider excluding large LFS files from checkpoint
git add -A, or at least rate-limiting checkpoints in repos where the clean filter is expensive. A side effect worth noting: refs/t3/checkpoints/* pin LFS objects, so git lfs prune retains more than users expect.
Happy to provide more detail from the session logs if useful.
Summary
In a repo with large Git LFS files, T3 Code's background git activity (checkpoint
git add -Aplus frequentgit status --porcelain=2 --branchpolls) filled.git/lfs/tmpwith roughly 150 GB of stranded temp files in about 9 hours. The trigger was a stale.git/index.lock, but T3 kept hammering the repo through the failure all day instead of backing off, and never cleaned up the partial files it left behind.Environment
What happened
.git/index.lockappeared at 08:44 (likely left by an interrupted git process; T3's checkpointing runsgit add -Aon this repo constantly, so it is the most frequent lock holder).git update-index --refreshfails, so git can never cache clean-filter results in the index.git status --porcelain=2 --branch, run every few seconds) therefore re-rangit lfs cleanover every stat-dirty LFS file. Each pass wrote fresh multi-hundred-MB temp files into.git/lfs/tmpand then failed at the index write, stranding them..git/lfs/tmpheld ~150 GB. After I wiped it, it regrew at roughly 4 GB/hour until I found and removed the stale lock.Confirmed the parentage with
ps: the headlessgit statusandgit-lfs filter-processprocesses were children ofT3 Code (Alpha).app/Contents/Resources/app.asar/apps/server/dist/bin.mjs. The temp file sizes matched the repo's large LFS files exactly (159 MB, 144 MB x2, 89 MB x2).Removing the orphaned lock fixed it immediately: the next
git statuscached its results and.git/lfs/tmpnow stays at 0 bytes, including through checkpoint runs.Suggested fixes
Any one of these would have contained the damage; together they'd prevent it:
.git/index.lockexists, is old (minutes, not seconds), and no live process holds it (lsofcomes back empty), remove it or at least stop polling and surface a warning. Git itself prints "another git process seems to be running" hints for this case..git/lfs/tmp. Stranded partials there are cheap to detect (files older than a few minutes with no open file handle). Even git-lfs itself doesn't reap them aggressively, so a periodic sweep in repos T3 manages would help.git add -A, or at least rate-limiting checkpoints in repos where the clean filter is expensive. A side effect worth noting:refs/t3/checkpoints/*pin LFS objects, sogit lfs pruneretains more than users expect.Happy to provide more detail from the session logs if useful.