Skip to content

perf: content-hash based cache keys for CI compatibility#5

Merged
cmilesdev merged 1 commit intogoforj:cmilesdev/custom-loaderfrom
zzzz465:fix/artifact-cache-content-hash
Mar 29, 2026
Merged

perf: content-hash based cache keys for CI compatibility#5
cmilesdev merged 1 commit intogoforj:cmilesdev/custom-loaderfrom
zzzz465:fix/artifact-cache-content-hash

Conversation

@zzzz465
Copy link
Copy Markdown

@zzzz465 zzzz465 commented Mar 27, 2026

Replace mtime-based cache invalidation with content hashing:

  • artifact_cache: use SHA-256 of file content instead of ModTime
  • discovery_cache: use content hash for file matching, add WIRE_DISCOVERY_CACHE_DIR env var
  • Bump cache versions (artifact v4, discovery v4)

This enables wire cache to work correctly in CI environments where file mtimes are not preserved across runs (e.g., S3 cache restore, git checkout).

Replace mtime-based cache invalidation with content hashing:
- artifact_cache: use SHA-256 of file content instead of ModTime
- discovery_cache: use content hash for file matching, add WIRE_DISCOVERY_CACHE_DIR env var
- Bump cache versions (artifact v4, discovery v4)

This enables wire cache to work correctly in CI environments where
file mtimes are not preserved across runs (e.g., S3 cache restore,
git checkout).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@cmilesdev
Copy link
Copy Markdown
Member

Hey! Thanks for contributing.

Main concern for this would be overhead introduced from doing content based hashing. Ran the benchmarks locally and didn't seem to make a substantial difference to cause any concern.

There's a few cache knobs that I think we should unify base env vars on but I'm totally fine with that change as well.

I'm glad the new loader changes has made quite a bit of difference for you. I imagine this was on some pubg services?

scripts/import-benchmarks.sh scenarios
+---------------+-------+--------+----------+----------------------+----------+----------+---------+
|       profile | local | stdlib | external | change type          | stock    | current  | speedup |
+---------------+-------+--------+----------+----------------------+----------+----------+---------+
|         local | 41    | 191    | 1        | cold run             | 379.0ms  | 3088.8ms | 0.12x   |
|         local | 41    | 191    | 1        | unchanged rerun      | 341.8ms  | 9.1ms    | 37.65x  |
|         local | 41    | 191    | 1        | body-only local edit | 325.5ms  | 36.8ms   | 8.85x   |
|         local | 41    | 191    | 1        | shape change         | 324.8ms  | 149.3ms  | 2.18x   |
|         local | 41    | 191    | 1        | import change        | 359.7ms  | 149.0ms  | 2.41x   |
|         local | 41    | 191    | 1        | known import toggle  | 326.4ms  | 149.6ms  | 2.18x   |
|    local-high | 1016  | 191    | 1        | cold run             | 741.9ms  | 5134.3ms | 0.14x   |
|    local-high | 1016  | 191    | 1        | unchanged rerun      | 603.9ms  | 81.6ms   | 7.40x   |
|    local-high | 1016  | 191    | 1        | body-only local edit | 599.5ms  | 173.5ms  | 3.45x   |
|    local-high | 1016  | 191    | 1        | shape change         | 587.3ms  | 690.3ms  | 0.85x   |
|    local-high | 1016  | 191    | 1        | import change        | 591.0ms  | 696.3ms  | 0.85x   |
|    local-high | 1016  | 191    | 1        | known import toggle  | 610.3ms  | 677.8ms  | 0.90x   |
|  external-low | 42    | 243    | 342      | cold run             | 1588.3ms | 8043.6ms | 0.20x   |
|  external-low | 42    | 243    | 342      | unchanged rerun      | 1216.7ms | 17.6ms   | 69.05x  |
|  external-low | 42    | 243    | 342      | body-only local edit | 1172.3ms | 115.0ms  | 10.19x  |
|  external-low | 42    | 243    | 342      | shape change         | 1233.5ms | 421.0ms  | 2.93x   |
|  external-low | 42    | 243    | 342      | import change        | 1253.9ms | 426.1ms  | 2.94x   |
|  external-low | 42    | 243    | 342      | known import toggle  | 1200.1ms | 412.3ms  | 2.91x   |
| external-high | 117   | 243    | 342      | cold run             | 1565.6ms | 8419.3ms | 0.19x   |
| external-high | 117   | 243    | 342      | unchanged rerun      | 1302.9ms | 22.6ms   | 57.74x  |
| external-high | 117   | 243    | 342      | body-only local edit | 1173.0ms | 125.5ms  | 9.34x   |
| external-high | 117   | 243    | 342      | shape change         | 1247.7ms | 460.6ms  | 2.71x   |
| external-high | 117   | 243    | 342      | import change        | 1100.0ms | 464.9ms  | 2.37x   |
| external-high | 117   | 243    | 342      | known import toggle  | 1172.0ms | 465.0ms  | 2.52x   |
+---------------+-------+--------+----------+----------------------+----------+----------+---------+

@cmilesdev cmilesdev merged commit b0177a6 into goforj:cmilesdev/custom-loader Mar 29, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants