Skip to content

fix(aube): key the no-integrity warm binding by registry URL in the global store#232

Merged
colinhacks merged 1 commit into
mainfrom
refetch-fix
Jun 28, 2026
Merged

fix(aube): key the no-integrity warm binding by registry URL in the global store#232
colinhacks merged 1 commit into
mainfrom
refetch-fix

Conversation

@colinhacks

Copy link
Copy Markdown
Contributor

A no-integrity package (npm v1/legacy lock, or an integrity-stripping proxy) records a content-address binding so a frozen warm install resolves its store index without re-fetching. That binding lived inside node_modules, so rm -rf node_modules && install — the dominant CI warm pattern, which restores the store cache but not node_modules — wiped it and forced a network re-fetch of every no-integrity package the store already held. An offline relink failed outright on the first such package.

This moves the binding to the global store at <store>/v1/no-integrity/<blake3(url)>.json, keyed by the registry tarball URL the coordinate resolves to (derived from the project's configured registry, not the lockfile's baked resolved).

  • Survives a node_modules wipe — it lives next to the CAS bytes it indexes, so the warm/offline relink content-addresses straight from the store.
  • Shared across projects resolving the same coordinate from the same registry, so a second project links it with no re-fetch.
  • Cross-registry substitution stays closed: a different registry yields a different URL key, so one project can never read another's binding for the same name@version from a different registry.

Readers (warm classifier, the linker's load_index fallback, ignored-builds, the streaming-resolve path) project the global URL-keyed store down to the existing name@version lookup, so the linker keeps its current key and needs no registry client.

Verification

On a real npm-v1 fixture (113 no-integrity entries), store isolated, --offline as the request counter:

  • COLD (online) → rc=0, writes 104 bindings.
  • WARM (nm-wiped, offline) → rc=0, 739 packages in 1.8s (previously rc=1, failing on abbrev@1.1.0).
  • Second same-registry project, never installed, offline → rc=0, reuses the global bindings with no network.
  • Control (wipe only the binding dir, store bytes intact) → offline fails on aproba@1.1.1, confirming the binding is the load-bearing artifact.

Regression tests cover cross-registry isolation, same-registry sharing, nm-wipe survival, and stale-in-node_modules migration; the install suite and clippy/fmt are green.

Refs #212 #220.

…lobal store

A package whose lockfile carries no integrity — npm v1/legacy locks, or an
integrity-stripping proxy — records a content-address binding so a frozen warm
install resolves its store index without re-fetching. That binding lived under
node_modules/.<embedder>-state/, so `rm -rf node_modules && install` (the
dominant CI warm pattern: the store cache is restored, node_modules is not)
wiped it and forced a network re-fetch of every no-integrity package the store
already held. An offline relink failed outright on the first such package.

Move the binding to the global store at
<store>/v1/no-integrity/<blake3(url)>.json, keyed by the registry tarball URL
the coordinate resolves to. The URL is derived from the project's configured
registry (make_client(cwd).tarball_url), not the lockfile's baked `resolved`
field, so the user's registry config defines the trust domain. This:

- survives a node_modules wipe (the binding lives next to the CAS bytes it
  indexes), so the warm/offline relink content-addresses straight from the
  store;
- is shared across projects resolving the same coordinate from the same
  registry, so a second project links it with no re-fetch;
- keeps cross-registry substitution closed: a different registry yields a
  different URL key, so one project can never read another's binding for the
  same name@version from a different registry.

Readers (the warm classifier, the linker's load_index fallback, ignored-builds,
and the streaming-resolve path) project the global URL-keyed store down to the
existing name@version lookup, so the linker keeps its current key and needs no
registry client. One file per URL (atomic, idempotent) so concurrent installs
binding different URLs never contend.

Refs #212 #220.

Claude-Session: https://claude.ai/code/session_01DnwgDLy5Tay9vsL544STPe
Copilot AI review requested due to automatic review settings June 28, 2026 17:06

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot wasn't able to review any files in this pull request.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@vercel

vercel Bot commented Jun 28, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
nub Ready Ready Preview, Comment Jun 28, 2026 5:08pm

Request Review

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes — relocates the no-integrity content-address binding from a per-project file inside node_modules to the global store, keyed by the registry tarball URL the coordinate resolves to, so it survives a node_modules wipe and stays cross-registry isolated.

  • Move the binding into the storestate.rs drops the node_modules/.aube-state/no-integrity-index.json index and writes one self-describing {url, sha512} file per resolved tarball URL at <store>/v1/no-integrity/<blake3(url)>.json, so a rm -rf node_modules warm install no longer wipes it.
  • Key by configured-registry URL, not lockfile resolved — write/read derive the URL via client.tarball_url(registry_name, version) from this project's .npmrc registry; a different registry hashes to a different key, closing the #212/#220 cross-registry substitution surface.
  • Reader projectionread_no_integrity_index_for projects the URL-keyed bindings down to the existing name@version map the warm classifier, linker load_index fallback, and ignored-builds consume, so those readers keep their coordinate lookup and need no registry client; the streaming-resolve path reads each binding on demand by URL.
  • Collision + idempotency guardsread_no_integrity_binding re-checks binding.url == url to reject a blake3 key collision; write_no_integrity_bindings skips an unchanged value and writes one atomic file per URL so concurrent installs binding different URLs don't contend.
  • Tests — four hermetic cases cover node_modules-wipe survival, same-registry sharing, cross-registry isolation, and ignoring a stale in-node_modules binding.

ℹ️ Pre-relocation in-node_modules bindings are left stale, not cleaned

The change ignores any leftover node_modules/.aube-state/no-integrity-index.json from before this PR (a test confirms it's never read), but it doesn't delete those files. This is harmless — the file is ignored, and the dominant warm pattern wipes node_modules anyway — so it's a deliberate leave-it-stale migration rather than a bug. Flagging only so the no-cleanup choice is on the record.

Technical details
# Pre-relocation in-node_modules bindings are left stale

## Affected sites
- `vendor/aube/crates/aube/src/state.rs` — removes `NO_INTEGRITY_INDEX_FILE_NAME` and its readers/writers; no migration deletes existing `node_modules/.aube-state/no-integrity-index.json` files.

## Required outcome
- None required. Confirm the no-cleanup behavior is intentional. The file is never read post-PR (covered by `stale_in_node_modules_binding_is_ignored`), and `node_modules` is wiped in the warm/CI flow this PR targets, so the stale file self-resolves.

## Open questions for the human (optional)
- Worth a one-line note in the PR body that the old per-project file is intentionally orphaned rather than migrated/cleaned?

ℹ️ Streaming read filters only integrity.is_none(), not local_source.is_none()

The on-demand streaming read in install/mod.rs gates the binding lookup on pkg.integrity.is_none() && fetch_no_integrity_present, whereas the batch read (read_no_integrity_index_for) and the write path also filter local_source.is_none(). The asymmetry is benign: a local-source package's URL key is derived from the registry, so it won't match any real binding and simply reads nothing. No fix needed unless symmetry is preferred for clarity.

Pullfrog  | View workflow run | Using Claude Opus𝕏

@colinhacks colinhacks merged commit 6a8529d into main Jun 28, 2026
52 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants