fix: cache.save_cached race + _clone_repo argument-injection hardening#589
fix: cache.save_cached race + _clone_repo argument-injection hardening#589Theprofitplatform wants to merge 2 commits into
Conversation
Concurrent save_cached calls for the same hash used to share a single
`entry.with_suffix(".tmp")` path. Two writers would race on the same temp
file — one's bytes could land in the other's atomic os.replace, leaving a
truncated or mixed-write cache entry. tempfile.mkstemp gives each writer
its own filename in the same directory, so os.replace remains atomic and
the only contention is the final rename (which is benign — last writer
wins, same content given the same hash).
Adds a regression test that fans out 8 threads writing the same hash and
asserts the final entry parses, no orphan .tmp files remain, and no
writer raised.
Two changes to the git invocation in _clone_repo:
1. Reject branch names starting with "-". git treats refspecs starting
with `-` as options in some positions, so a value like
`--upload-pack=evilcmd` passed as `branch` could be reinterpreted by
git as an option for the next argument. Fail fast with a clear error.
2. Add a `--` separator before positional URL/dest arguments. Today the
git_url is regex-locked to github.com URLs so the immediate exposure
is small, but the `--` makes it impossible for any future change to
the URL-building code to leak an option through this surface. Same
guard added to the `git pull origin <branch>` path.
Adds tests/test_clone.py covering:
* branch starting with "-" causes SystemExit before any subprocess.run
* the constructed argv contains a literal "--" separator before
positional URL/dest pairs
|
Hi, Severity: action required | Category: reliability How to fix: Ensure fd closed on failure Agent prompt to fix - you can give this to your LLM of choice:
Found by Qodo. Free code review for open-source maintainers. |
|
The requested changes have been implemented in 3fdae8f. Cache writer now uses |
…bs#638 Graphify-Labs#589 Graphify-Labs#586 Graphify-Labs#593: kimi thinking, manifest, inline comments, query boost, cache race, markdownify, content hash Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Two small, independent hardening fixes — each in its own commit with a regression test.
1. cache.save_cached: per-writer unique tempfile
save_cachedusedentry.with_suffix(\".tmp\")for the staging path, which is a single shared filename per hash. Concurrent writers for the same hash (parallel extractors, retries) raced on that file — one writer's partial bytes could land in another's atomic `os.replace`, leaving a truncated/mixed cache entry on disk.Switched to `tempfile.mkstemp(dir=target_dir, prefix=f"{h}.", suffix=".tmp")` so every writer gets its own staging file. The final `os.replace` is still atomic; only the final rename contends, and last-writer-wins is fine because two writers with the same hash are writing the same content.
Test: `test_save_cached_concurrent_writers_no_collision` fans out 8 threads writing the same hash, asserts the final entry is parseable, no orphan `.tmp` files remain, and no writer raised.
2. _clone_repo: argument-injection hardening
Two changes to the git invocation:
Test: `tests/test_clone.py`
Out of scope (filing separately)
These came up in the same review pass but need design discussion, not a 5-fix PR:
I'll open these as separate issues so they can be discussed independently.
Test run
```
$ pytest tests/test_cache.py tests/test_clone.py
12 passed (1 pre-existing failure in test_clear_cache that exists on v5 HEAD without my changes, due to cache_dir now returning .../cache/{kind}/)
```
Co-Authored-By: Claude Opus 4.7 (1M context) noreply@anthropic.com