Fix/ed25519 signed urls - #45
Merged
Merged
Conversation
… and DomainVerificationConfirmation The file header said signed delivery URLs use HMAC-SHA256 over a secret shared between the Exchange and the CDN. No implementation ever did that, and the SDK in this repository has never contained an HMAC: signing is a detached Ed25519 signature over "GET\n<canonical URL>", base64url with no padding, and the delivery endpoint verifies it with the Exchange's published public key. Nothing secret ever reaches the edge, which is the whole point of the scheme and the opposite of what the comment promised. The identity-binding block is unchanged in substance -- the agent thumbprint is embedded in the URL and covered by the signature -- but it now says which signature covers it, and the bearer-only fallback no longer names a primitive that does not exist. DomainVerificationConfirmation carried the same claim twice. signing_key said key format is "PEM for CloudFront, hex for HMAC"; hex is an HMAC-secret encoding, so the sentence had no reading that matched the implementation. Its format now follows cdn_type: a PEM RSA public key, or the base64url raw Ed25519 public key. cdn_type's documented value set becomes "edge-ed25519" | "cloudfront". The retired values named vendors rather than schemes, which is why "fastly" was actively wrong -- a Fastly Compute deployment runs the Ed25519 verifier, so its scheme is edge-ed25519. The name edge-ed25519 is not new: ADR-012 D3 assigned it for exactly this path, so adopting it collapses two spellings rather than adding a third. The value list also moves into the field's LEADING comment: the website renders leadingComments || trailingComments, and this field already had a leading comment, so the trailing list was invisible to every reader of the reference page. No wire change. The value set lives only in a comment on an optional string, never in an enum, and cdn_type has no consumer in the SDK, the conformance corpus, or the reference implementation. gen/ is regenerated with the pinned buf; the conformance corpus is unchanged, as comments carry no constraints.
This page specified a scheme that does not exist, in enough detail that an implementer could build it and be confident. It described HMAC-SHA256 over a secret shared with the CDN, a canonical string of four fields joined by "\n", a hex digest, a timing-safe comparison, and a max-TTL bound. None of that is in the code. Four of the five things a verifier needs were wrong at once: the algorithm, the signed input, the signature encoding, and the parameter names. What ships: a detached Ed25519 signature over "GET\n<canonical URL>", base64url with no padding, verified with the Exchange's published public key. The canonical URL is the whole URL with sig removed and the query sorted, not four selected fields, so scheme, host, path and every publisher query parameter are covered too. The expiry parameter is exp, not expires. There is a kid parameter, which the page never mentioned. There is no txn_id parameter at all -- the join key for reconciliation is the signed-URL hash, recorded by both the Exchange and the edge. The "two verification modes" framing is the source of the error and is gone. There is one signing scheme with two deployment postures: a code-capable edge that verifies Ed25519 and can enforce agent binding, and CloudFront verifying RSA natively but unable to run the binding check, so bearer-only. Deletions. Timing-Safe Comparison goes because there is no secret, no expected value and no comparison -- crypto.subtle.verify is a verification, not an equality test, and the section's premise does not exist. The max-TTL check goes because the verifier does not implement one; expiry is the only time check. The Akamai target goes: EdgeAuth is a symmetric token scheme built on a shared secret, which is the model this correction exists to remove, so documenting it as supported would reproduce the defect. Additions, all previously undocumented and all load-bearing for anyone writing a verifier: verification fails closed on a key-resolution error; the signature is checked before the expiry, because both come from the same untrusted URL; the full refusal vocabulary for the URL check and the proof-of-possession check; the proof-of-possession wire contract, including that its signature parameters are ordered keyid, alg, created, expires and that its Signature header uses standard base64 while the URL's sig uses base64url; that binding enforcement defaults on and only the literal string false disables it; that directory keys carry no identifier, so kid is a locally recomputed thumbprint rather than anything a publisher asserts; and that a custodially registered agent does not fetch its own URL, because its key never leaves the registry. Single-use enforcement is marked as not implemented rather than described as available, and its KV key is rebased on sig now that txn_id does not exist.
Three pages still told an operator to deploy a secret to the edge. The edge verifies with a public key and holds nothing secret, so each of these was not just a wrong algorithm name but a wrong instruction. overview.mdx framed the system as two signing subsystems needing "different cryptographic primitives", one symmetric and one asymmetric. Both are Ed25519. The real distinction is which key signs and who verifies, so the contrast is rewritten rather than relabelled. The request-flow pseudocode also claimed an expiry check against a max-TTL bound that no verifier implements, and described agent binding as conditional when it is on by default. cdn-adapters.mdx carried the false claim in executable shape: EdgeConfig declared signingMode and an optional hmacSecret. Those fields are replaced by the configuration the edge actually reads -- the Exchange key directory URL and optional pinned public keys. The Cloudflare note said the Worker "holds the signing secret or public key"; it holds only the public key, and that difference is the security property, not a detail. The Akamai EdgeWorkers section is removed and replaced by one paragraph saying Akamai is unsupported and why. EdgeAuth verifies with a secret shared with the CDN, which is exactly the model being retired here; documenting it as a supported target would reproduce the defect this change exists to fix. The platform matrix loses the Akamai column with it, and its verification row is retitled and re-derived: an Ed25519 verify is roughly 50-100 microseconds, so a CloudFront Function fails on the 1 ms budget and the absence of Ed25519 rather than on cost, Cloudflare passes natively, and Fastly passes with the injected verifier the SDK accepts for exactly that reason. deployment.mdx told an operator to run `wrangler secret put HMAC_SECRET`. That line is deleted rather than renamed -- there is no secret to put. The config table's signing-mode and quarterly-secret-rotation rows go with it, replaced by the directory URL and optional pinned keys. The secrets-management section now states the property plainly: nothing confidential is deployed to the edge, so the signing configuration may live in plain config or KV. composition.mdx needed no change. It already names the per-tenant selector correctly and is the model the other pages are brought in line with.
These three pages carried the strongest statement of the false model anywhere on the site: not just that signed URLs use HMAC, but a stated rationale for why they should. overview.mdx argued that "different trust models require different cryptographic primitives", and storage-model.mdx said symmetric signing was appropriate "because the Exchange and the CDN edge function share a secret". They never did. Both signing subsystems are Ed25519, and what actually separates them is which key signs and who verifies -- offers are verified by agents and Brokers, delivery URLs by the delivery endpoint. The rationale is rewritten, not relabelled, because a corrected algorithm name under a false argument would have left the page still teaching the wrong idea. The Go samples described a struct that does not exist. CDNType with a CDNGenericHMAC member was a third spelling of a concept the tenant table already names signing_scheme, and PrivateKey was documented as "PEM-encoded private key or HMAC secret". Both samples now mirror the shape the Exchange really dispatches on, with the two values the enum really has, so a reader can check them against the code. The canonicalization section described a four-field string joined by newlines. The signed message is the literal GET, one newline, then the whole canonical URL with sig removed and the query sorted -- which covers scheme, host, path and every publisher query parameter, not four selected fields. Rather than restate the rules in a second place where they can drift again, the section now states the shape briefly and points at the normative page, and adds the fact this page is the right home for: reconciliation joins on signed_url_hash, recorded by both the Exchange and the delivery endpoint, because the txn_id parameter it referred to does not exist.
Twelve pages carried the same false claim in three shapes: prose assertions, sample URL literals, and two security countermeasures that described defences nobody built. The URL literals were the widest problem. Every walkthrough showed ?expires=...&agent_id=...&txn_id=...&sig=hmac-sha256-..., and all four parts were wrong: the expiry parameter is exp, there is no txn_id parameter at all, the signature is Ed25519 rather than an HMAC digest, and there is a kid parameter the examples never showed. They now show the real parameter set in the order the canonical encoding produces, which is lexicographic. The EU-regulation example is the useful one to read: its publisher parameter uri sorts last and is covered by the signature like everything else, which the old four-field canonicalization could not express. Wherever a page said txn_id enables three-sided reconciliation, the replacement is not a rename. Reconciliation joins on the signed-URL hash -- SHA-256 of the URL verbatim -- which the Exchange records on the transaction and the delivery endpoint records on its delivery event. Neither side chooses the value, which is the property that makes the join trustworthy. Two threat-model countermeasures needed more than a substitution. T7 said the provider holds the URL-signing private key and the Exchange calls out to a provider signing service; no such service exists, and the Exchange signs with a per-tenant key it holds itself. The countermeasure now states what actually defends against the threat: per-tenant key isolation, a public verification key, and no signing capability distributed anywhere. T5 promised a transaction_id URL parameter for reconciliation and is corrected to the hash join. Neither line contained the string HMAC, so a grep for the defect would have missed both. scenario-walkthrough.mdx had a third defect the sweep exposed. Its fetch trace verified agent binding as SHA256(requester.id + ":" + requester.domain), which binds nothing -- an identifier is not a secret and an attacker can recompute it. The check is a thumbprint comparison against the presented public key, followed by an RFC 9421 proof that the fetcher holds the private half. The page's own caution already said hashing an identifier is not a binding; the trace contradicted it. publisher-onboarding.mdx told an operator to generate HMAC secrets for Akamai and Fastly, and its key-type table classified three of four rows as symmetric. Fastly runs the Ed25519 verifier, so that row was not merely outdated wording. The table is now two rows, both asymmetric, with the cdn_type value each one advertises -- and it says what actually differs between them, which is whether the delivery endpoint can run code and therefore enforce agent binding.
The wording was corrected everywhere at once in the preceding commits; this stops it coming back, the same way the retired JWS phrasings are held out. Two patterns are deliberately narrower than the defect they guard. The bare token HMAC stays legal. Report tokens are genuinely HMAC-hashed, and a future document may legitimately cite HMAC as a rejected alternative -- banning the token would force a later author to write around a true statement. The bare token txn_id also stays legal, and this one was found by running the check rather than by reasoning about it: three Go samples in request-flows.mdx use txn_id as a structured log field name, which is correct and unrelated to URL parameters. The patterns are anchored to the URL forms instead. Each pattern was verified by breaking the tree and watching the gate fail, not by observing that it passes. That caught a real defect in this commit: the canonicalization pattern was written with one character too many and matched nothing, so it would have shipped as a guard that could never fire.
Exhaustive entry in the protocol changelog, curated mirror on the website, both under Unreleased and following the format of the offer-signature correction that shipped earlier in this release -- the same defect class, documented the same way. The v1.0 history is left as written. Its retrieval-URL binding entry is scheme-neutral and remains accurate.
# Conflicts: # gen/descriptor.binpb
The minimum-viable decision table named the URL parameters as Signature and Expires. The implemented parameters are sig and exp; the capitalised pair is CloudFront's own naming, which does not apply to the Ed25519 path the table describes. The same section claimed the feature set works on Fastly VCL and Akamai Property Manager rules. Neither can verify a signature, so neither can admit a paid request -- they can serve ramp.json and refuse known crawlers, and that is all. The supported-CDN table listed Akamai with the full feature set; it is not a supported target, for the reason the CDN adapters page states. The CDN adapters page still named Akamai EdgeWorkers in its frontmatter description after the section itself was replaced.
The signing_key comment said the field carries a public key while the onboarding page pushed ramp-cloudfront-private.pem through it and every other page put the private key inside the Exchange. Both readings cannot be right: either private material crossed the wire labelled public, or the field was dead. The comment now states the custody model the KeyStore implements -- the Exchange signs with a private key it holds and never publishes, the delivery endpoint verifies with the public half. Where the Exchange has to sign with a key the provider generated, as a CloudFront trusted key group requires, the private half is provisioned out of band and never travels in this field. The onboarding page pushes the public half and says where the private half goes. cdn_type said it "mirrors the Exchange-side tenant signing scheme" without saying which value mirrors which. Both mappings are now written down. "The agent is the fetcher" was true for only one of the two deployments. An agent embedding the SDK holds its own key and fetches for itself; a custodial agent never fetches at all, because its key lives in its identity service, which fetches on its behalf. Both present the same key to the edge, which is what makes the binding check work in either case, and neither was represented. "Fully offline / no JWKS fetch required" is true of the fetcher's key, which arrives in the request. The edge still resolves kid to the Exchange's public key from a cached directory, which the same page documents, so the claim now names the key it applies to. The Akamai entry in the walkthrough's adapter link list goes with the target.
The scenario walkthrough mints the agent identity hash at step 7 as SHA256(requester.id + ":" + requester.domain), puts it in the URL's agent_id, and then verifies it at the edge as thumbprint(presented public key). The example cannot pass: a hash of two public identifiers is not the thumbprint of a key, and anyone can recompute it, so it binds nothing. Step 7 now computes the RFC 7638 thumbprint of the key whose RFC 9421 signature the Exchange just verified, and says why the identifier hash was not a binding. The placeholder values told the same story backwards. agent_identity_hash is a thumbprint, which is base64url, and the examples printed hex; sig is base64url with no padding, and three walkthroughs printed hex there too. A reader copying the shape of these values would build the wrong parser. The EU walkthrough's publisher parameter is uri=CELEX:32024R1689. The signer percent-encodes the query, so the byte the signature covers is CELEX%3A..., and the unescaped colon in the example is a signature that would not verify.
'?txn_id=' is not a valid ERE. BSD grep -- the one on the macOS runner -- rejects a leading '?' with "repetition-operator operand invalid" and exits 2, which the loop's 2>/dev/null hid and its || true swallowed. The pattern was dead there and the gate reported clean. GNU grep accepts the same pattern, so a Linux-only check never noticed. The pattern is now [?]txn_id=, and every pattern in both lists is compiled against empty input before use: exit 0 or 1 means it compiled, 2 or more fails the run with the pattern named. Verified by planting an uncompilable pattern and watching the script exit 1, and by planting each of the six retired phrasings in a page and watching the gate catch it. The shared-secret pattern only allowed three characters between Exchange and CDN, so "Exchange <-> CDN shared secret" passed; it now allows eight. Four retired phrasings the sweep deleted but the list never held out are added: "hex for HMAC", "HMAC verification", signingMode, and "hmac" as a cdn_type value. The bare token HMAC stays legal on purpose, because a page may name HMAC as the alternative this scheme rejects, and the changelog does exactly that. The justification comment cited report tokens, which live in another repository and cannot be checked from here; it now cites the changelog.
The page claimed the signature is checked before the expiry, as a deliberate property, and the changelog repeated it. Only the Go verifier does that. The TypeScript and Python verifiers read exp first and return "expired" without touching the signature, and the edge runs the TypeScript face. Both orders refuse the same URLs; what differs is the reason a caller sees. The page now says so and marks the refusal reason a diagnostic rather than a contract. Aligning the three verifiers is a change to shipped code and belongs in its own ticket. The key directory is not tenant-scoped: an Exchange publishes one directory for all its tenants and the edge accepts any kid that resolves in it, so a URL signed with another tenant's key verifies on your edge. That matters when one Exchange serves publishers who do not trust each other, and pinning the expected keys is the control. Neither the property nor the control was written down.
The sweep matched retired phrasings rather than reading each touched page from top to bottom, so changed lines ended up contradicting unchanged ones on the same page. Three places still provisioned a secret to a delivery endpoint whose headline claim is that it holds none: a degradation row reading "signing secret missing", and two configuration descriptions listing signing secrets as an environment variable. The edge's failure there is that it cannot resolve a verification key, which answers 503 verify_unavailable. Two config surfaces described the same worker and disagreed. One declared maxUrlTtlSeconds, which no verifier reads -- the only expiry bound is the exp the Exchange signed -- and agentBindingEnabled, while the other had already become enforce_binding, on by default. The overview said binding and single-use were each enforced "if enabled" without saying which way either defaults; binding is on, single-use is off and best-effort. Akamai survived as a delivery target in a deploy sequence, a packaging option, a deploy-command table and an architecture diagram, on pages that also say it is not supported. Its EdgeAuth scheme verifies with a secret shared with the CDN, which is the model this correction removes.
…easure T6 said the protocol mandates a five-minute maximum URL TTL that the provider's edge function enforces independently. No verifier enforces a second bound: the only bound is the exp the Exchange signed, so a 24-hour URL is caught by reconciliation, not refused at the edge. A provider wanting a hard ceiling has to add that check itself, and now the page says so. T7's attacker is the Exchange, and the countermeasure described the Exchange's own key custody -- a control the attacker operates cannot stop the attacker. What the design actually gives you is a bounded blast radius: a leaked verification key forges nothing, a compromised edge cannot mint URLs, and a leaked tenant key forges for one tenant only. Detection belongs to T5, where the provider's delivery log and the Exchange's transaction log join on signed_url_hash. The claim that the Exchange "signs there", inside the secrets manager, also contradicted the Exchange overview: keys load at startup and signing is a pure function call.
The entry described the primitive correction but not the two facts the review of it surfaced: which half of the key pair signing_key carries and where the other half comes from, and that a custodial agent does not fetch its own content -- its identity service does, holding the same key the delivery endpoint checks the binding against.
…p an unopenable pointer The new entry used " -- " where every other entry in both files uses an em dash. It also credited an architecture record by number for the edge-ed25519 name; that record lives in the reference implementation, not here, so a reader of this repository cannot open it. The sentence now says the name was already in use without pointing at something out of reach.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The defect
The protocol docs and proto comments described signed delivery URLs as HMAC-SHA256 over a secret shared between the Exchange and the CDN. No implementation ever did that.
git grep -ci hmac -- sdk/finds nothing: signing has always been a detached Ed25519 signature that a delivery endpoint verifies with a public key.The repository contradicted itself in public —
sdk/shipsVerifyURLEd25519,website/described HMAC, at the same commit.It was wider than the algorithm name
An implementer building an edge verifier from these pages got four things wrong at once:
baseURL\nexpires\nagent_id\ntxn_id"GET\n"+ whole canonical URL,sigstripped, query sortedexpiresexptxn_idkid; there is notxn_idanywhere in the signerBecause the canonical message is the whole URL rather than four selected fields, scheme, host, path and every publisher query parameter are covered too — which the old format could not express.
An operator following the deployment pages provisioned a shared secret nothing reads, including a literal
wrangler secret put HMAC_SECRET.Commits
d7980c7DomainVerificationConfirmation, with all four regenerated artifacts, atomicallyd7079e1signed-url-verification.mdx290b41fba9b77960397f7bd8f24a631b8d6Proto goes first on purpose:
remark-protorenders the reference page'sDomainVerificationConfirmationsection from the committed descriptor, so a docs commit landing first would ship a reference page still reading "hex for HMAC" beside pages saying Ed25519 — anddocs-cinever compares descriptor to proto, so it would pass silently.cdn_typeThe documented value set becomes
"edge-ed25519"|"cloudfront". The retired values named vendors rather than schemes, which made"fastly"actively wrong — a Fastly Compute deployment runs the Ed25519 verifier.edge-ed25519is not a new name: ADR-012 D3 already assigned it for this path, so adopting it collapses two spellings rather than adding a third.The value list also moves into the field's leading comment. The reference page renders
leadingComments || trailingCommentsand this field already had a leading comment, so today's trailing value list is invisible to every reader of the rendered page.No wire change. The value set lives only in a comment on an
optional string, never in an enum, andcdn_typehas no consumer in the SDK, the conformance corpus, or the reference implementation.Three defects found while sweeping, fixed here
security/threat-model.mdxT7 said the provider holds the URL-signing private key and the Exchange calls a provider signing service. No such service exists; the Exchange signs with a per-tenant key it holds itself.security/threat-model.mdxT5 promised atransaction_idURL parameter for reconciliation. The join key issigned_url_hash, recorded independently by both sides.protocol/scenario-walkthrough.mdxverified agent binding asSHA256(requester.id + ":" + requester.domain)— which binds nothing an attacker cannot recompute. The page's own caution block said exactly that, two screens above.None of the three contained the string
HMAC, so a grep for the defect would have missed all of them.Akamai
No longer documented as a supported delivery target. EdgeAuth verifies with a secret shared with the CDN, which is the model this change removes; documenting it as supported would reproduce the defect. One paragraph says it is unsupported and why.
Denylist
Each new pattern was verified by breaking the tree and watching the gate go red, not by observing that it passes. That caught a real defect in this PR: the canonicalization pattern was written with one character too many and matched nothing, so it would have shipped as a guard that could never fire.
Two patterns are deliberately narrower than the defect. The bare token
HMACstays legal — report tokens genuinely are HMAC-hashed. The bare tokentxn_idstays legal too, found by running the check rather than by reasoning about it: three Go samples use it as aslogfield name.Verification
./scripts/ci-local.sh— PASS, all ten steps, with the pinned buf 1.66.1.npm run buildinwebsite/green, all internal links valid.check-doc-conformance.shclean. The conformance corpus is unchanged, which confirms only comments moved.Companion change
The reference implementation carries the same claim in two published ADRs and a dead
tenants.hmac_secret_refcolumn. That lands separately, as appended ADR amendments rather than edits to the ADR bodies. This PR does not depend on it.