Expose runtime cache CLI commands#50
Merged
Merged
Conversation
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
Introduces a separate `kv/` namespace for blobs stored under a caller-chosen digest, alongside the existing content-addressed `cas/` tree. `get_blob` stays CAS-only so an ActionResult restore cannot be poisoned by a pre-staged keyed entry whose key collides with an output digest; keyed access goes through `get_keyed_blob`/`has_keyed_blob`. Splits `Stats` into separate `blob`/`keyed` counts and runs the three `count_files` walks under `tokio::try_join!`. Tuist `has_blob` now HEADs the remote on local miss so `exists` mirrors `get_blob`'s reach for content-addressed digests. Extracts a shared `stream_to_tmp` + `rename_into_place` helper used by both `put_stream` and `put_keyed_stream`. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds `cache blob exists <digest>` with shell-friendly exit codes (0 hit, 1 miss; always 0 in `--format json`/`toon`). Adds `--key` to `blob put`, `blob get`, and `blob exists` so scripts opt into the keyed namespace explicitly; the two namespaces never overlap. `cache hash` now accepts multiple paths and combines their digests in order; duplicate `-` is rejected because a second stdin read would silently hash to empty. `cache stats` prints a separate `keyed:` row across human, JSON, and TOON output. Includes shellspec coverage for the new exit-code semantics, the two-namespace split, the multi-path hash, the duplicate-`-` guard, and the new stats row. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds a guide page covering the full `fabrik cache` surface (hash, blob put/get/exists with the keyed namespace, action get/put/forget, stats), with the two motivating examples (skipping a Vitest run when inputs are unchanged, restoring node_modules without reinstalling). Adds a blog post announcing the runtime cache CLI and the keyed-blob primitive, framing it as a script-level surface for skip-if-unchanged and restore-instead-of-regenerate workflows. Includes the social image. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The runtime cache CLI page existed but was not reachable from the sidebar. Introduces a `Primitives` section as a sibling of Native Rules and Scripts so future cache and compute primitives have a place to live alongside the runtime cache CLI. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`Runtime` was overloaded and added no signal beyond what the `Primitives` section already conveys. Shortens the page title, file slug (`/guide/cache-cli`), and sidebar entry to just `Cache CLI`, updates the H1 and intro to match. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replaces the single Commands table and the "Two blob namespaces" paragraph with per-surface sections, each carrying its own commands table: - Hashing (the helper used by all three caches) - Content-addressed cache - Keyed cache - Action cache (terminology mirrors Bazel/REAPI) - Inspecting (cache stats) Moves the Vitest and node_modules walkthroughs into an Examples section at the end. Opens with storytelling about why the layer exists (long-tail scripts, agent parallelism, redundant CI work) rather than positioning the page as "the layer underneath annotated scripts." Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The `--format json`/`toon` flags are documented globally; restating them here added noise without information. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…stderr optional Aligns the cache substrate with the Bazel/REAPI two-cache model: a content-addressed blob store plus an action cache, with no third keyed namespace. Removes the `kv/` directory, every `*_keyed_*` method on `Cas` and `CacheProvider`, and the `keyed_count`/`keyed_bytes` fields from `Stats`. The keyed namespace existed to let scripts memoise an artifact under a digest of its inputs; the same use case is served by the action cache with the artifact as a declared output. `ActionResult.stdout` and `stderr` become `Option<Digest>` so a caller that did not capture output (or had nothing worth recording) leaves them unset instead of materialising an empty blob. Tuist `prefetch_action_blobs` now skips absent fields, downstream readers in `fabrik-cli` handle the option explicitly, and `fabrik-core`'s four construction sites wrap captured digests in `Some`. `spec/cache_stats_spec.sh` drops the `keyed:` row expectation.
`cache action get` and `cache action put` now accept a repeatable `--input <spec>` flag in place of (or alongside) a positional digest; clap's `ArgGroup` enforces exactly one of the two. The input grammar covers the things that practically determine a script's outcome: <path> file or directory (directories walk sorted) path:<path> explicit path form for names containing `:` value:<str> literal string env:<NAME> environment variable, hashed as `NAME\0value` - stdin (allowed at most once) Directories are walked deterministically via walkdir; the digest is `relpath\0content\0` per file under a domain-tagged prefix, so two trees with the same files in the same paths hash identically regardless of filesystem iteration order. `cache action get --if-success` exits 0 only when the cache has a hit AND the recorded exit code is 0; on miss or on a cached failure it exits non-zero so the wrapping script re-runs. `cache action put` now defaults `--exit-code` to 0 since the common case is recording a success; failures pass an explicit value. `cache hash` is removed: declaring inputs with `--input` covers every script-level use case the command was there for, and the helpers it relied on are shared with the new path. `--key` on `cache blob` disappears too, leaving the blob cache as the pure content-addressed store the docs already describe. Specs cover the new grammar, the `--if-success` exit codes, the default `--exit-code`, deterministic directory walks, and the env-var namespacing that prevents distinct variables sharing a value from collapsing into the same key.
Drops the dedicated Hashing section (now covered by `cache action --input`), reframes the page around two caches (blob and action), and documents the input grammar once at the top so every command that takes inputs can point at it. Examples updated to the new shape: the Vitest skip-on-success example collapses to declare-inputs / `--if-success` / `put`; the npm install example uses `cache action put --output node_modules.tar=<digest>` with a `jq` extraction on the read side. A third example mirrors mise-action's caching pattern: cache `~/.local/share/mise/` keyed on platform plus mise config and lockfile, restoring the whole tool tree between branches and across machines. The blog post gets the same examples, a short "Declaring inputs" subsection explaining the grammar, and an updated command summary that drops the `cache hash` lines.
…error paths Stops reusing a blob digest as the action key in the positional-digest roundtrip spec; the two namespaces are different concepts even though both serialise as 64-hex, and the test now mints the action key from its own `cache blob put` so callers reading the spec are not nudged toward conflating them. Adds three error-path specs: `cache blob get` on a missing digest exits non-zero and surfaces "blob not found"; `cache blob put` on a missing file exits non-zero with "opening blob input"; a malformed positional digest is rejected by clap with "BLAKE3" in the message. Locks the failure surface so a regression that swallows or rephrases these errors gets caught.
Project convention is no em dashes in user-facing prose. The introductory paragraph on the mise example used one to chain a follow-on clause; a comma plus an "and" reads the same and matches the rest of the post.
The blog renderer does not style Markdown tables (no row borders, header weight, or alignment), so the grammar table came out as ungrouped cells with uneven row heights when the env: row wrapped. Switched to a bulleted definition list which reads cleanly under any renderer; the docs page keeps the table since VitePress handles it.
Reflows four spots that drifted from rustfmt during the cache CLI work: `has_blob` in fabrik-cas (chained call wrap), `open_blob_input` and the `WalkDir` `let entry =` line in commands/cache.rs (signature and binding on one line), and the `CacheActionCmd::Get` match arm in dispatch.rs (inline body). No behaviour change.
`hash_directory` walked the tree on `spawn_blocking`, returned to async, and then read each file with `tokio::fs::read` in a sequential await loop. `tokio::fs::read` itself dispatches to the blocking pool per call, so a tree of N files cost N+1 blocking dispatches and N async hops with no parallelism to show for it. Move the read + hash loop into the same blocking task as the walk: one dispatch total, `std::fs::read` since we are already on a blocking thread, and the kernel pages bytes straight through the hasher without bouncing in and out of async. Existing `directory_hash_*` unit tests cover correctness across iterations, content change, and layout change.
| } | ||
| let specs: Vec<InputSpec> = inputs.iter().map(|s| InputSpec::parse(s)).collect(); | ||
| validate_stdin_uniqueness(&specs)?; | ||
| if specs.len() == 1 { |
Contributor
There was a problem hiding this comment.
The resolve_action_digest function has a fast-path for single inputs that returns the hash directly rather than combining. This is correct but subtle - a single value:foo hashes differently than combining one digest. Consider a brief comment noting that this is intentional (avoiding domain separator overhead for common case) or use combine_digests uniformly for clarity.
🤖 Instructions for AI agents
You are an AI agent asked to address a code review finding. Treat this block as your prompt.
Finding: Consider clarifying early-return optimization for single input
Details:
The resolve_action_digest function has a fast-path for single inputs that returns the hash directly rather than combining. This is correct but subtle - a single value:foo hashes differently than combining one digest. Consider a brief comment noting that this is intentional (avoiding domain separator overhead for common case) or use combine_digests uniformly for clarity.
Location: crates/fabrik-cli/src/commands/cache.rs:289
How to fix:
- Open
crates/fabrik-cli/src/commands/cache.rs:289and read the surrounding code so you understand the context before changing anything. - Fix the underlying issue described in Details above — do not silence the symptom (e.g. by suppressing a warning, catching and discarding an error, or deleting the test that surfaces it).
- Run the project's existing test and lint commands and confirm they pass before reporting the task as complete.
- Keep the change minimal and focused on this finding; surface any unrelated concerns separately rather than bundling them in.
- Once the fix is committed, if the
ghCLI is available, mark this review thread as resolved so the human reviewer knows it's been addressed — use the GitHub GraphQLresolveReviewThreadmutation viagh api graphql(look up the thread ID for this comment first).
— Blick · default review
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
fabrik cache blob put/getfor direct content-addressed blob reads and writes from files, stdin, stdout, or output paths.fabrik cache action get/put/forgetfor inspecting, seeding, and deleting cached action results by digest.Testing
mise exec -- cargo fmt --all -- --checkmise exec -- cargo test -p fabrik-climise exec -- cargo clippy -p fabrik-cli --all-targets -- -D warningsmise exec -- cargo build --releasemise exec -- shellspec spec/cache_runtime_spec.sh