Skip to content

refactor: adopt freenet-migrate for the legacy-contract fallback - #66

Merged
sanity merged 3 commits into
mainfrom
adopt-freenet-migrate
Aug 11, 2026
Merged

refactor: adopt freenet-migrate for the legacy-contract fallback#66
sanity merged 3 commits into
mainfrom
adopt-freenet-migrate

Conversation

@sanity

@sanity sanity commented Aug 11, 2026

Copy link
Copy Markdown
Contributor

Problem

freenet-git hand-rolls its legacy-contract migration machinery: a bespoke TOML parser in build.rs, a bespoke key derivation (contract_id_from_wasm_hash), and a bespoke probe walk in wsclient::get_state_with_legacy_fallback. Four Freenet apps independently wrote this same machinery; freenet-migrate now packages the shared, field-proven version (River's shipped UI probe, adopted by Delta and Atlas). The registry here is still empty — deliberately, per the pre-1.0 churn note in legacy_contracts.tomlwhich is exactly why now is the right moment to switch: the first real re-key (already queued as #63) should run on the shared path, not on bespoke code that has never once executed against real data. #65 (merged) gave the bespoke path a behavioural test suite for the same reason; those tests are the spec this PR is held to.

No user data is affected by this change: the registry has zero entries, so the fallback has never fired in any shipped binary, and the probe behaviour for an empty registry is unchanged (one GET at the current key).

What is adopted

  • Registrylegacy_contracts.toml moves to freenet-migrate's [[contract]] schema (explicit generation, code_hash in hex or base58, note), parsed and validated at build time by freenet-migrate-build. A typo'd hash, duplicate generation, or malformed row is now a build failure; the retired hand-rolled parser silently skipped unrecognized keys and misspelled [[entry]] blocks. The file's explanatory comments (the operator-facing docs) are kept and updated.
  • Key derivation — unchanged. freenet-git keeps its own contract_id_from_wasm_hash, because the crate's equivalent lives in the runtime half (see the constraint below).
  • Probe sequencing — the walk stays hand-rolled, for the same reason. What DID change is its order, which was a latent bug: see "The bug this actually fixes".
  • Re-key guard (new)tests/migration_guard.rs wires freenet-migrate-build's check_migration_guard to the committed contracts/repo-contract.wasm: swapping the WASM without registering the outgoing hash now fails the suite with step-by-step instructions. This closes part of the CI gap Rebuild repo-contract.wasm to ship the get_state_delta absent-vs-zero fix (requires re-key + migration) #63 calls out (CI still doesn't rebuild the WASM from source; the guard covers artifact swaps, which is the re-key vector).

The constraint that shaped this PR: only the build half is adoptable here

An earlier revision of this PR adopted the runtime half of freenet-migrate as well. It
cannot be adopted in this repo, and the reason is structural rather than a bug to work around.

freenet-stdlib/rust/src/global.rs:4-5 exports __frnt_set_id as #[no_mangle] extern "C" with
no cfg(feature) guard, so every build of the crate emits that symbol. Linking two versions
into one binary is therefore always:

rust-lld: error: duplicate symbol: __frnt_set_id
  >>> libfreenet_stdlib-6ddae59e35d551a0.rlib   (0.6, freenet-git's contracts)
  >>> libfreenet_stdlib-d9fdb837c4a2619e.rlib   (0.8, freenet-migrate)

freenet-git's contracts are pinned to stdlib 0.6, and bumping them re-keys every repo — which
is itself the migration event this machinery exists to survive. freenet-migrate 0.5.x requires
0.8. So the runtime half is unavailable until the contracts move, and that move is its own
deliberate migration, not a dependency bump.

freenet-migrate-build is unaffected and is what this PR adopts: its dependencies are serde,
toml, blake3 and bs58 — no stdlib at all — so it can never reintroduce the collision.

The local test suite could not have caught this

The first revision reported 243/243 green locally and failed at link in CI. This repo links
with mold; CI uses rust-lld. The local suite was structurally incapable of producing the
failure that gates merge. Reproduce CI's behaviour with:

CARGO_TARGET_DIR=target-lld RUSTFLAGS="-C link-arg=-fuse-ld=lld" cargo test --workspace

Verified on this head: 243 passed, 0 failed, single freenet-stdlib v0.6.1 in the tree.

The bug this actually fixes

The old walk took candidates in registry slice order (oldest first) and stopped at the first
hit. With two surviving generations that means the stale one wins — and is then re-PUT forward
onto the current key, promoting old state over new. It has never harmed anyone only because the
registry is empty.

The walk is now newest-generation-first, ordered by the explicit generation field rather than
by position in the file, so re-ordering the TOML cannot change behaviour. Both properties are
pinned: newest-first ordering, and that a stale older survivor loses.

Behavioural spec of the old code, and every difference accepted

Spec derived from the code + #65's tests before touching anything:

  1. Probe current key first; on miss walk legacy hashes in registry slice order; first non-empty state wins.
  2. NotFound and empty responses are authoritative per-key absence; timeouts/backpressure are transient; dead sockets are hard errors.
  3. A failed pass with ≥1 transient outcome and no hard error retries the whole sequence, up to 4 attempts, 2s/4s/8s backoff; all-authoritative fails fast ("no state found"); timeouts are never reported as absence ("state on the network is unknown").
  4. A legacy hit is re-PUT to the current key; a rejected re-PUT degrades to a warning and the recovered state is still served.
  5. GetSource::Legacy { index } indexes the registry slice for the log line.

Differences deliberately accepted (each is a behaviour change only for a populated registry, i.e. live for no one today):

# Old New Why
1 Legacy keys probed in registry slice order (file convention = oldest first) Newest generation first (generation descending) With ≥2 generations still holding state, oldest-first + first-hit-wins adopts the stale generation and re-PUTs it forward — the "rolled back to April" incident class the driver's ordering exists to prevent. This was a latent bug in the old walk.
2 A transient failure on one probe let the pass continue to the remaining keys (and only then retried) Any non-authoritative outcome (timeout, backpressure, transport) aborts the pass; the existing whole-sequence retry then re-runs it Continuing past an unknown newer generation to adopt an older one concludes absence from silence — the freenet-migrate#19 data-loss class. Abort-and-retry keeps the three-way present / absent / unknown distinction strict: unknown → retry, never conclude. Retry classification, backoff, attempt cap, hard-error fast-abort, and the dominant-outcome error messages are all preserved verbatim.

Everything else is preserved: the retry classifier (is_transient_host_error, ProbeOutcome, outcomes_worth_retrying), the 4-attempt budget, the backoff schedule, the dominant-outcome error messages, the empty-vs-NotFound distinction, the forward re-PUT with warning-on-reject, and the slice-index contract of GetSource::Legacy.

Explicit override of a crate default: the driver's recommended semantics treat a timeout as a per-candidate miss ("the predecessor does not have it"). That is the data-loss default tracked as freenet-migrate#19, and both prior adopters overrode it. Here the pump delivers only authoritative absence as a miss; unknown outcomes abort the pass for retry (difference 2 above makes this stricter than the old code, not just equal).

The stdlib seam — why only the build half

This workspace pins freenet-stdlib 0.6, including the contract crates, and freenet-migrate's
runtime half is built against 0.8. An earlier revision of this PR carried both. It cannot
work
: freenet-stdlib/rust/src/global.rs:4-5 exports __frnt_set_id as #[no_mangle] extern "C"
with no cfg(feature) guard, so two versions in one binary is always
rust-lld: error: duplicate symbol: __frnt_set_id. There is no feature flag that avoids it.

Bumping the contracts to 0.8 would re-key every repo — its own migration event, not a dependency
bump — so the runtime half is unavailable here until that happens. This PR therefore adopts only
freenet-migrate-build, whose dependencies are serde, toml, blake3 and bs58 (no stdlib at all),
and keeps the walk and the key derivation local.

No contract WASM changed

crates/freenet-git/contracts/repo-contract.wasm and pack-contract.wasm are byte-identical to main (b3sum: a397cc99…9c13, 4140e677…0a87); no file under crates/repo-contract/, crates/pack-contract/, or crates/types/ is touched. The new dependencies land only in the CLI crate, so no contract re-key is implied by this PR. (Workspace rust-version moves 1.86 → 1.87 for freenet-migrate's MSRV; the build toolchain is pinned at 1.94.1 and the committed WASM artifacts are, per the above, unchanged.)

Testing

Full workspace suite: 237 passed / 0 failed before (at b3b1451), 243 passed / 0 failed after. cargo fmt --check and cargo clippy --workspace --all-targets clean.

The #65 tests are preserved as the behavioural spec, with three adjustments, none deleted quietly:

  • walks_predecessors_in_registry_order_and_reports_the_indexwalks_predecessors_newest_generation_first: the ordering it pinned (slice order) is the latent-rollback behaviour difference 1 removes. The replacement pins the new ordering and that a stale older survivor loses to the newest generation, plus that the reported index stays a slice index across the reordering.
  • legacy_registry_codegen.rs is rewritten against freenet-migrate-build (the hand-rolled parser it tested is deleted). Same properties, same oracle independence: TOML → generated text → probeable key, compared against the 0.6 stdlib's own derivation; loud failure on missing hash / truncated hash; NEW: duplicate generations rejected (the old parser accepted them silently). The empty-registry canary now asserts on the compiled CONTRACT_LINEAGE const, and still fires (with update-don't-delete instructions) the day the first real entry lands.
  • The source-scrape pin production_fetch_repo_state_passes_the_generated_registry is retargeted at CONTRACT_LINEAGE, guards intact.

New tests: an_unreachable_newer_generation_is_never_skipped_for_an_older_one (the #19 override, end to end), persistent_transient_failure_exhausts_the_retry_budget (bounded retries; timeouts reported as timeout, never absence; deliberately ~17s of real time — the budget under test is wall-clock), and the two migration_guard ratchet tests.

Mutation evidence

Every mutation was applied to production code (or the guard's pin), the suite run with --no-fail-fast over the freenet-git targets (165 tests at this head; the table below was first run on the pre-rework revision and re-verified on b71d489 — see the review), and reverted; predictions written down first.

# Mutation (production code) Predicted Actual
M1 fetch_repo_state wrapper passes &[] instead of CONTRACT_LINEAGE (registry effectively deleted from production wiring) exactly 1 red: the source pin; all else green 1 red / 156 green: production_fetch_repo_state_passes_the_generated_registry
M2 probe order reversed (generations inverted before the driver, oldest-first) 4 red 4 red / 153 green: walks_predecessors_newest_generation_first, an_unreachable_newer_generation_is_never_skipped_for_an_older_one, empty_state_at_a_predecessor_is_not_mistaken_for_recovery, migration_tests::a_second_registry_entry_reports_the_right_index
M3a pump treats every legacy probe error as authoritative absence (if authoritativeif true) — the #19 class at the pump 1 red (outer retry masks it elsewhere) 1 red / 156 green: an_unreachable_newer_generation_is_never_skipped_for_an_older_one
M3b classifier calls a timeout authoritative (Timeout => TransientTimeout => Authoritative) — the #19 class at the classifier 5 red 5 red / 152 green: 2 outcomes_worth_retrying_* unit pins + transient_predecessor_timeout_is_retried_and_then_recovers, an_unreachable_newer…, persistent_transient_failure_exhausts_the_retry_budget
M4 retry cap removed (attempt >= PROBE_MAX_ATTEMPTS || deleted → retry forever) 1 red 1 red / 156 green: persistent_transient_failure_exhausts_the_retry_budget (fails fast via its own 60s bound instead of hanging)
M5 legacy generations never probed (&lineage[..0] handed to the driver — registry entries effectively deleted at the walk) 9 red 9 red / 148 green: all 5 recovery/order tests + absence + retry + all 3 migration_tests
M7 committed WASM swapped without registering the outgoing hash (pin perturbed — byte-equivalent to an artifact swap with an empty registry) exactly 2 red 2 red / 155 green: both migration_guard ratchet tests

Every mutation was verified applied (non-empty diff) before the run and reverted to a clean tree after; predictions were written into the harness before any run. M3a is the layered-defence case: the outer retry masks the pump-level bug everywhere except the adopt-older-past-unknown path, which is exactly why an_unreachable_newer_generation_is_never_skipped_for_an_older_one exists as a separate end-to-end pin.

Review notes

Closes nothing; groundwork for #63. Refs freenet/freenet-core#2776, freenet/freenet-migrate#19.

[AI-assisted - Claude]

sanity and others added 2 commits August 11, 2026 15:33
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BEgtjegwuJPWSaAnVJ3z4e
freenet-stdlib exports __frnt_set_id as an unconditional #[no_mangle]
extern "C" symbol, so linking the crate's stdlib 0.8 alongside this
workspace's 0.6 is a duplicate-symbol error under rust-lld (CI); the
local mold linker tolerated it, which is why the previous commit was
green locally. The registry/codegen/guard half (freenet-migrate-build,
no stdlib dependency) stays; the probe walk is local again, keeping the
newest-generation-first ordering and the #19 abort-on-unknown semantics.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BEgtjegwuJPWSaAnVJ3z4e
…pty lineage

freenet-migrate-build 0.2.0 derives Registry with #[serde(default)] on both
fields and no deny_unknown_fields, so a row under the wrong table name
deserializes to an EMPTY registry and the build succeeds
(freenet/freenet-migrate#20, filed today). The likeliest wrong name is
[[entry]] -- the format this very file used before adopting the crate.

That is the failure the whole mechanism exists to prevent, arriving through
the parser. An empty lineage means the fallback probes no predecessor at
all, so a repo published under an older contract WASM is intact on the
network and permanently unreachable, with no error at any layer.

This repo is unusually exposed. Its registry is legitimately EMPTY during
the pre-1.0 phase, so an "is the lineage empty?" canary cannot tell a
correct empty file from one full of unparsed rows -- registry.contract
.is_empty() is true either way. Checking table names is the only local
signal that separates them.

The guard runs BEFORE codegen, because the parser cannot report what it
silently drops. Verified red-capable rather than assumed: clean build
green; append an [[entry]] block and the build fails naming the line and
the fix; restore and it is green again. Full suite under lld (what CI
uses, not mold) 243 passed / 0 failed.

The PR body previously claimed a malformed entry is always a build
failure. That was true of hashes and duplicate generations and false of
table names; this makes it true as stated.

Found by Fable 5 review of #66.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BEgtjegwuJPWSaAnVJ3z4e
@sanity

sanity commented Aug 11, 2026

Copy link
Copy Markdown
Contributor Author

Fable 5 review — no blocking findings. Both SHOULD-FIXes addressed.

Adversarial review at head b71d489. The reviewer reproduced the CI-parity link run and applied three mutations in a git archive scratch copy, leaving this worktree untouched.

SHOULD-FIX 1 — unknown tables were silently ignored. FIXED in 06edbd6, and filed upstream.

Demonstrated empirically rather than argued: appending an [[entry]] block — this file's own pre-adoption formatbuilt successfully with exit 0 and emitted an empty CONTRACT_LINEAGE. freenet-migrate-build 0.2.0 derives Registry with #[serde(default)] on both fields and has no deny_unknown_fields anywhere in the crate (I confirmed this in the vendored source).

That directly contradicted this PR's own claim that "a malformed entry is a build failure, never a silently-skipped entry" — true of hashes and duplicate generations, false of table names, which is the retired hand parser's exact behaviour arriving by a different route.

This repo is unusually exposed, and the reason is worth stating: its registry is legitimately EMPTY during the pre-1.0 phase, so an "is the lineage empty?" canary cannot distinguish a correct empty file from one full of unparsed rows — registry.contract.is_empty() is true in both. Table names are the only local signal that separates them.

Fixed by guarding before codegen (the parser cannot report what it silently drops). Verified red-capable, not assumed: clean build green → append [[entry]] → build fails naming the line, the mechanism, and the fix → restore → green. The failure message says what is actually at stake rather than "unknown table".

Filed upstream as freenet/freenet-migrate#20, with the cross-repo analysis: Atlas is protected, though not by the crate — its literal expected-hash pin fails on an empty parse. So the working mitigation today is an adopter-side expected-content pin, which is exactly what freenet-git cannot have while its registry is legitimately empty.

SHOULD-FIX 2 — the body described the abandoned revision. FIXED.

The body still contained a "stdlib seam" section claiming the CLI carries both stdlib versions, a CarryForward non-adoption paragraph, and mutation M6 targeting to_client_id, which does not exist at this head. Anyone reading the merged record would have believed the binary links two stdlibs.

Rewritten: the stdlib section now explains why only the build half is adoptable (__frnt_set_id unconditionally #[no_mangle], contracts pinned to 0.6, and moving them re-keys every repo — its own migration event). M6 and the CarryForward paragraph are removed, and the suite size corrected from 157 to 165 with a note that the table was first run pre-rework and re-verified on b71d489.

NIT — accepted, recorded

is_transient_host_error / from_get_state_err classify by substring on error prose, so a host-forwarded error containing "not found on the network" would read as authoritative absence. The reviewer grepped freenet-core and freenet-stdlib: the phrase appears nowhere, so only git-remote-freenet's own bail site can produce it, and it is const-shared with that site. Pre-existing from #65, not this PR's change. Noted as the one residual path where wording drift could turn an error into absence.

Independently verified — including two mutations re-run at this head

  • Runtime dependency genuinely gone: exactly one freenet-stdlib node (v0.6.1); freenet-migrate-build's tree is blake3/bs58/serde/toml with no stdlib. The two surviving freenet_migrate:: references are doc comments.
  • Probe order is real, not incidental. The reviewer wrote its own extra mutant — sort by Reverse(slice index) instead of Reverse(generation), i.e. "file order pretending to be generation order" — and it was caught. So newest-first genuinely derives from the generation field, and the suspected test gap does not exist.
  • M2 (probe order inverted): 4 red, matching the author's report exactly. M3a (legacy-loop abort deleted — the error: no state found at current contract key or any of 0 legacy keys #19 class): exactly 1 red, confirming that test is the lone pin on the layered-defence path, as claimed.
  • error: no state found at current contract key or any of 0 legacy keys #19 semantics, every path traced (wsclient.rs:534-622): only NotFound for the exact id or an empty GetResponse advances the walk. Timeout/backpressure → pass aborts → whole-sequence retry. Transport/decode → fail fast. A non-authoritative current-key failure aborts before any legacy probe, so a stale generation cannot be served while the current key is merely slow. Persistent silence surfaces as "state on the network is unknown", never "no state found". No path converts silence into absence.
  • Registry validation, by breaking the TOML: truncated hash → InvalidCodeHash; duplicate generation → DuplicateGeneration; missing file → Io — the last stricter than the old build.rs, which emitted an empty table via unwrap_or_default().
  • test: exercise the legacy-contract migration fallback before it is needed #65's three test adjustments are strengthenings or justified retargets. The slice-order pin was retargeted to the new ordering with a decoy that fails under the old behaviour.
  • No re-key: both WASM b3sums identical at base, head and working tree; the pinned guard hash independently re-derived as base58(b3sum(repo-contract.wasm)).
  • CI-parity run reproduced: CARGO_TARGET_DIR=target-lld RUSTFLAGS="-C link-arg=-fuse-ld=lld" cargo test --workspace243 passed / 0 failed. Re-confirmed after the guard commit.

Bottom line

The probe-order fix is the substantive win here: the old walk took candidates oldest-first and stopped at the first hit, so with two surviving generations the stale one wins and is re-PUT forward over newer state. It has harmed nobody only because the registry is empty — which is precisely why adopting now, while the stakes are zero, was the right call.

[AI-assisted - Claude]

@sanity
sanity merged commit dfd6e03 into main Aug 11, 2026
5 checks passed
@sanity
sanity deleted the adopt-freenet-migrate branch August 11, 2026 21:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants