fix(hashing): op.Directory hashing now uses FileHasher cache (ITL-511) by kurodo3[bot] · Pull Request #215 · nauticalab/orcapod-python

kurodo3 · 2026-07-08T01:35:06Z

Summary

BasicDirectoryHasher._hash_dir() was calling hash_utils.hash_file() directly, bypassing the injected FileContentHasherProtocol entirely. This meant op.Directory hashing always recomputed every file hash from scratch, making the file hash cache dead weight for directory-heavy workloads.
enable_file_hash_caching() only patched FileHandler; DirectoryHandler was untouched.

Changes

BasicDirectoryHasher: file_hasher: FileContentHasherProtocol is now a required first argument; _hash_dir() delegates to it for per-file leaves
v0.1.json: wires {"_ref": "file_hasher"} into BasicDirectoryHasher config so the default context injects the same FileHasher instance
enable_file_hash_caching(): also patches DirectoryHandler using the same CachedFileHasher instance as FileHandler — a file cached via op.File is a cache hit during directory traversal
New tests in test_directory_hash_caching.py covering cache write/hit/invalidation and the enable_file_hash_caching() wiring
Updated test_directory_handler.py: all BasicDirectoryHasher() calls now pass file_hasher=FileHasher()
Updated test_hash_cachers.py: fixture restores DirectoryHandler; 2 new tests for DirectoryHandler patching
New doc docs/concepts/file-hash-caching.md: explains the caching architecture, activation, directory behaviour, storage, and trade-offs

Test plan

uv run pytest tests/test_hashing/ -v — all hashing tests pass
uv run pytest tests/ -q — full suite passes (4160 passed, 56 skipped)

Fixes ITL-511

…file cache (ITL-511) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Document the file_hasher injection requirement for BasicDirectoryHasher in the v0.1 context changelog.

…ryHasher in tests Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Wire the same CachedFileHasher into DirectoryHandler so that op.Directory traversal consults the per-file cache. A file cached via op.File hashing is a cache hit during directory traversal.

codecov · 2026-07-08T01:36:49Z

Codecov Report

❌ Patch coverage is 93.75000% with 1 line in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
src/orcapod/contexts/__init__.py	88.88%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!

…uses-filehasher-cache-document

Copilot

Pull request overview

This PR fixes op.Directory hashing so it reuses the injected FileContentHasherProtocol (and therefore the file-hash cache) instead of re-hashing file leaves from scratch, and extends enable_file_hash_caching() to patch both FileHandler and DirectoryHandler with the same cached hasher instance.

Changes:

BasicDirectoryHasher now requires a file_hasher and delegates file-leaf hashing to it during Merkle traversal.
Default context wiring updated (v0.1.json + builtin handler fallback) so directories receive the same FileHasher instance as files.
enable_file_hash_caching() now patches DirectoryHandler too, sharing one CachedFileHasher instance across file + directory hashing; adds tests + documentation.

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
`src/orcapod/hashing/directory_hashers.py`	Use injected `file_hasher` for file leaves; make `file_hasher` required.
`src/orcapod/hashing/semantic_hashing/builtin_handlers.py`	Fallback `BasicDirectoryHasher` construction now passes `file_hasher`.
`src/orcapod/contexts/__init__.py`	`enable_file_hash_caching()` now also patches `DirectoryHandler` with the same cached hasher instance.
`src/orcapod/contexts/data/v0.1.json`	Inject `file_hasher` reference into `BasicDirectoryHasher` config; add changelog entry.
`tests/test_hashing/test_directory_handler.py`	Update all `BasicDirectoryHasher()` call sites to pass `file_hasher=FileHasher()`.
`tests/test_hashing/test_directory_hash_caching.py`	New integration tests for directory hashing cache behavior + wiring.
`tests/test_hashing/test_hash_cachers.py`	Extend cache-enablement tests to include `DirectoryHandler` patching and shared instance assertions.
`docs/concepts/file-hash-caching.md`	New user documentation describing caching and activation.
`superpowers/specs/2026-07-08-itl-511-directory-hash-caching-design.md`	Design/spec for the change set.
`superpowers/plans/2026-07-08-itl-511-directory-hash-caching.md`	Implementation plan/checklist for the work.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

kurodo3 · 2026-07-08T01:52:45Z

+To inspect cached entries using the SQLite command-line tool:
+
+```bash
+sqlite3 ~/.orcapod/file_hash_cache.db "SELECT path, method FROM file_hashes LIMIT 10;"


Fixed — corrected table name from file_hashes to file_hash_cache and updated the SELECT columns to valid schema columns (path, size, cached_at) — the actual schema has no method column.

kurodo3 · 2026-07-08T01:52:47Z

+Orcapod can cache the SHA-256 digest of files so that re-hashing the same
+unchanged file — across pipeline runs or between `op.File` and `op.Directory`
+accesses — skips the disk read entirely. This page explains how the cache
+works, how to turn it on, and when it helps.


Fixed — changed 'SHA-256 digest' to 'content hash (SHA-256 by default, configurable via the algorithm argument)' so it's accurate for any configured algorithm.

eywalker · 2026-07-08T01:49:06Z

+called the file hasher directly, bypassing the per-file cache even when
+caching was enabled.
+
+After ITL-511, the `DirectoryHandler`'s `BasicDirectoryHasher` uses the


Do NOT mention Linear issue numbers anywhere in the documentation -- these issue numbers are NOT helpful.

Fixed — removed the ITL-511 reference from the directory hashing section. The paragraph now reads as timeless documentation without any internal issue numbers.

Follow-up fix: also removed the issue number from the v0.1.json changelog entry we added in this PR. The entry now reads as plain prose without any Linear reference.

…hema - enable_file_hash_caching(): add explicit RuntimeError when no DirectoryHandler is registered (mirrors existing FileHandler guard) - docs: fix SQLite table name in inspect/clear examples (file_hashes -> file_hash_cache); fix SELECT columns (method -> size, cached_at); describe FileHasher as returning a content hash (SHA-256 by default) rather than implying SHA-256 is the only algorithm

…caching guide

kurodo3 · 2026-07-08T01:52:58Z

Review round summary

Four issues fixed in a single commit (fix(contexts): guard DirectoryHandler None; fix doc table name and schema), plus a follow-up commit to remove the internal issue number from the docs:

Comment	Fix
Missing `None` guard for `DirectoryHandler` in `enable_file_hash_caching()`	Added explicit `RuntimeError` check after `registry.get_handler_for_type(Directory)`, matching the existing `FileHandler` guard
SQL inspect example used wrong table name `file_hashes`	Corrected to `file_hash_cache`; also fixed column names to match the actual schema (`path, size, cached_at` — there is no `method` column)
SQL delete example used wrong table name `file_hashes`	Corrected to `file_hash_cache`
`FileHasher` described as always returning a "SHA-256 digest"	Changed to "content hash (SHA-256 by default, configurable via the `algorithm` argument)"
Internal issue number (ITL-511) referenced in user-facing docs	Removed; the directory hashing section is now written as timeless documentation

All 507 hashing tests still pass after the changes.

kurodo3 · 2026-07-08T01:58:58Z

Review round summary

Removed the remaining issue number reference from src/orcapod/contexts/data/v0.1.json.

The changelog entry added by this PR previously ended with (ITL-511). That suffix has been removed so the entry reads as plain prose, consistent with eywalker's feedback to keep issue numbers out of documentation.

All 507 hashing tests still pass.

kurodo3 Bot and others added 7 commits July 8, 2026 00:14

docs(specs): add ITL-511 directory hash caching design spec

aef781c

docs(plans): add ITL-511 directory hash caching implementation plan

5034d08

feat(hashing): require file_hasher in BasicDirectoryHasher; wire per-…

328e27f

…file cache (ITL-511) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

chore(context): add ITL-511 changelog entry to v0.1.json

1c16086

Document the file_hasher injection requirement for BasicDirectoryHasher in the v0.1 context changelog.

test(directory_hasher): pass file_hasher=FileHasher() to BasicDirecto…

4a4d0e5

…ryHasher in tests Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

feat(contexts): enable_file_hash_caching() also patches DirectoryHandler

7264b59

Wire the same CachedFileHasher into DirectoryHandler so that op.Directory traversal consults the per-file cache. A file cached via op.File hashing is a cache hit during directory traversal.

docs(hashing): add file-hash-caching activation guide

2ae651d

eywalker requested a review from Copilot July 8, 2026 01:47

Merge branch 'main' into eywalker/itl-511-verify-opdirectory-hashing-…

bdf468a

…uses-filehasher-cache-document

Copilot started reviewing on behalf of eywalker July 8, 2026 01:48 View session

Copilot AI reviewed Jul 8, 2026

View reviewed changes

eywalker requested changes Jul 8, 2026

View reviewed changes

kurodo3 Bot added 2 commits July 8, 2026 01:51

docs(hashing): remove internal issue number reference from file-hash-…

f9abf12

…caching guide

chore(context): remove issue number from v0.1.json changelog entry

0812342

eywalker approved these changes Jul 8, 2026

View reviewed changes

eywalker merged commit 9c6e124 into main Jul 8, 2026
11 checks passed

eywalker deleted the eywalker/itl-511-verify-opdirectory-hashing-uses-filehasher-cache-document branch July 8, 2026 02:07

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

fix(hashing): op.Directory hashing now uses FileHasher cache (ITL-511)#215

fix(hashing): op.Directory hashing now uses FileHasher cache (ITL-511)#215
eywalker merged 11 commits into
mainfrom
eywalker/itl-511-verify-opdirectory-hashing-uses-filehasher-cache-document

kurodo3 Bot commented Jul 8, 2026

Uh oh!

codecov Bot commented Jul 8, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

kurodo3 Bot Jul 8, 2026

Uh oh!

Uh oh!

kurodo3 Bot Jul 8, 2026

Uh oh!

eywalker Jul 8, 2026

Uh oh!

kurodo3 Bot Jul 8, 2026

Uh oh!

kurodo3 Bot Jul 8, 2026

Uh oh!

kurodo3 Bot commented Jul 8, 2026

Uh oh!

kurodo3 Bot commented Jul 8, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

kurodo3 Bot commented Jul 8, 2026

Summary

Changes

Test plan

Uh oh!

codecov Bot commented Jul 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

kurodo3 Bot Jul 8, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

kurodo3 Bot Jul 8, 2026

Choose a reason for hiding this comment

Uh oh!

eywalker Jul 8, 2026

Choose a reason for hiding this comment

Uh oh!

kurodo3 Bot Jul 8, 2026

Choose a reason for hiding this comment

Uh oh!

kurodo3 Bot Jul 8, 2026

Choose a reason for hiding this comment

Uh oh!

kurodo3 Bot commented Jul 8, 2026

Review round summary

Uh oh!

kurodo3 Bot commented Jul 8, 2026

Review round summary

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

codecov Bot commented Jul 8, 2026 •

edited

Loading