feat(backend-federation): outbound OIDC→backend STS exchange (oidc-provider delegates to it)#57
Merged
Merged
Conversation
|
🚀 Latest commit deployed to https://multistore-proxy-pr-57.development-seed.workers.dev
|
…backend STS New `multistore-federation` crate: the symmetric outbound half of multistore's existing inbound federation (multistore-sts / multistore-oidc-provider). Lets the proxy present its own OIDC identity to a backend cloud and assume a role there, so it can serve a private bucket without the operator holding long-lived keys. - aws: runtime-agnostic AssumeRoleWithWebIdentity request builder + XML response parser (caller owns HTTP), surfacing AWS <ErrorResponse> as a typed error. - FederatedCredentials::apply_to(&mut BucketConfig): inject temp creds (access_key_id/secret_access_key/token), clear skip_signature, disable anonymous_access. Debug redacts secret + session token. No core changes; additive crate only. Tests cover parse, error path, redaction, request shape, and BucketConfig integration. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
alukach
force-pushed
the
feat/backend-federation
branch
from
0d8deb5 to
df827c3
Compare
…c-provider delegation
Resolves the duplication between this crate and multistore-oidc-provider's
existing (wired-in) outbound exchange.
- Rename multistore-federation -> multistore-backend-federation: the outbound
AssumeRoleWithWebIdentity *client*, conceptual peer to multistore-sts's
inbound *server*.
- Make multistore-oidc-provider delegate its AWS STS exchange to this crate
(request build via form_pairs() + parse_response), deleting its hand-rolled
extract_xml_value parser. Wired-in get_credentials + AwsBackendAuth
middleware behavior is preserved (all existing tests pass).
- Carry the primitive's improvements through delegation: DurationSeconds (now
Option, omitted when None), inline session policy, typed StsError
{code,message}, and secret-redacting Debug on CloudCredentials.
- Fix apply_to to leave inbound anonymous_access untouched (it governs only
outbound backend signing).
- Add an env-gated live functional test against real AWS STS + S3
(tests/live_sts.rs) plus a CI job to run it, a documented federation bucket
in the cf-workers wrangler.toml, and a crate README.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
alukach
changed the title
… not green) The live-federation job was reporting success while the test self-skipped (MULTISTORE_TEST_ROLE_ARN unset), giving a misleading green check that implied the real AWS STS path was exercised when it wasn't. - Gate the live-federation job with `if: vars.MULTISTORE_TEST_ROLE_ARN != ''` so it shows as *skipped* (neutral), not *passed*, when unconfigured. - Mark the test `#[ignore]` so the ordinary unit-test job reports it as *ignored* rather than *passed*; the gated job runs it with `-- --ignored`. The crate's encoding/parsing/redaction logic remains covered by the unit tests; only the real network round-trip is gated on a configured role + bucket. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…onfigured Per request: the live-federation job must fail — not skip — when MULTISTORE_TEST_ROLE_ARN is unset, so a green checks list genuinely means the real AWS STS path was exercised. - Drop the job's `if:` guard so it always runs. - The test now panics (instead of returning early) on missing MULTISTORE_TEST_ROLE_ARN, failing the `-- --ignored` run. - Kept `#[ignore]` so ordinary `cargo test` (and the unit-test CI job, and local dev) does not run it and is unaffected. CI's live-federation check will be RED until the repo Actions variables MULTISTORE_TEST_ROLE_ARN / MULTISTORE_TEST_BUCKET / MULTISTORE_TEST_REGION are set and the IAM role trusts this repo's GitHub OIDC subject. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…s and staging
Adds an end-to-end test of the outbound federation path (proxy mints its own
OIDC assertion -> AssumeRoleWithWebIdentity -> SigV4 read of a private bucket),
run by the shared smoke-test job on every preview (PR) and staging deploy.
To avoid creating/tearing down a per-PR AWS identity provider + role, PR
previews reuse the STABLE staging OIDC issuer: all deployments share
OIDC_PROVIDER_KEY, so a preview mints tokens whose `iss` is the staging URL and
AWS validates them against the single, already-registered staging IAM OIDC
provider. preview.yml and staging.yml both set oidc_issuer_override to
${{ vars.STAGING_OIDC_ISSUER }}; the rationale is commented in both workflows.
- tests/smoke/test_federation.py: anonymous GET of the `federated-test` bucket
must return the private object (no silent skip — fails if federation is
unconfigured).
- wrangler.deploy.toml: `federated-test` bucket (auth_type=oidc) with
placeholder role ARN / bucket to replace with real test resources.
- deploy.yml: pass FEDERATION_TEST_KEY to the smoke-test job.
Requires (one-time, no per-PR IAM): set the STAGING_OIDC_ISSUER repo variable,
register an AWS IAM OIDC provider for it, and create a role trusting it
(sub=multistore, aud=sts.amazonaws.com) with read on the private bucket.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
alukach
added a commit
that referenced
this pull request
Jun 3, 2026
The backend-federation crate (added in #57) was missing from the release-please extra-files list, so its $.workspace.dependencies entry wasn't bumped in lockstep with the workspace version. On the 0.5.0 release PR this left the dep pinned at ^0.4.0 while the crate built as 0.5.0, breaking workspace resolution and failing every CI job. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
alukach
added a commit
that referenced
this pull request
Jun 4, 2026
…ive refresh) Roadmap item 2 of #56. Caches short-lived FederatedCredentials per credential identity so a proxy doesn't re-mint on every request: - proactive refresh once within a configurable lead window of expiry (so a credential never expires mid-use), - single-flight: concurrent callers for the same key await one in-flight fetch via a per-key futures::lock::Mutex, - runtime-agnostic: the caller passes `now` (no clock dep) and no async runtime is required. Closure-based get_or_fetch(key, now, fetch) keeps it flexible; invalidate(key) supports drop-and-refetch on backend rejection. Tested: miss/hit/refresh/ invalidate/key-isolation and concurrent single-flight. Rebased onto main: the federation crate landed on main as backend-federation (#57), so this folds the cache into that crate instead of re-adding a duplicate. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Refs #56 (roadmap item 1: the federation exchange primitive).
What
Adds
multistore-backend-federation— the runtime-agnostic outbound STS-exchange primitive: the proxy presents its own OIDC identity to a backend cloud, assumes a role there (AWSAssumeRoleWithWebIdentity), and signs backend requests with the temporary credentials, so the operator never holds long-lived backend keys.It is the client-side counterpart to the existing
multistore-sts(the inboundAssumeRoleWithWebIdentityserver, which mints proxy credentials for callers). The naming now reflects that symmetry:sts= inbound server,backend-federation= outbound client.Relationship to
multistore-oidc-provider(important)multistore-oidc-provideralready performed the outbound AWS exchange (inexchange/aws.rs), with its own caching (CredentialCache) andAwsBackendAuthmiddleware wired into dispatch. Rather than ship a second, parallel implementation, this PR makesoidc-providerdelegate its AWS exchange tobackend-federation:AwsExchange::exchangenow builds the request viabackend-federation'sAssumeRoleWithWebIdentity::form_pairs()and parses it viaparse_response; the hand-rolledextract_xml_valueXML scanner is deleted.OidcCredentialProvider::get_credentials(mint → exchange → cache) andAwsBackendAuth/MaybeOidcAuthmiddleware are unchanged and still pass their existing tests.oidc-providerkeeps its real responsibilities: minting the proxy's identity (JWT signing, JWKS, discovery), caching, and middleware.So
oidc-provideris the batteries-included orchestrator;backend-federationis the dependency-light mechanism it (and any "bring-your-own-token" caller) builds on. Dependency direction:oidc-provider → backend-federation → multistore(no cycles; verified for native and wasm).Contents
aws— runtime-agnosticAssumeRoleWithWebIdentityrequest builder.form_pairs()returns unencoded key/value pairs (for HTTP layers that urlencode themselves, e.g. reqwest.form());body()returns a ready-to-POST urlencoded string.parse_responseparses the XML; an AWS<ErrorResponse>becomes a typedFederationError::Sts { code, message }.FederatedCredentials::apply_to(&mut BucketConfig)— injectsaccess_key_id/secret_access_key/tokenand clearsskip_signature.Debugredacts the secret + session token.DurationSeconds(omitted whenNone), inlinesession_policy, the typedStsError, and a secret-redactingDebugonoidc-provider'sCloudCredentials(it previously derivedDebugand could log secrets).cf-workers example
The example already wires OIDC backend auth (
MaybeOidcAuth+AwsBackendAuth+ aFetchHttpExchange), so it exercises this crate transitively. This PR adds a documentedfederated-privatebucket towrangler.toml(auth_type=oidc+oidc_role_arn) showing the setup, and the wasm build is verified.Functional tests
tests/live_sts.rs— an env-gated functional test that runs the full primitive against real AWS STS (build request → exchange a real OIDC token → parse) and then proves the credentials work by reading the private bucket viaobject_store, exactly how multistore signs backend requests. It self-skips whenMULTISTORE_TEST_ROLE_ARNis unset, so ordinarycargo testis a no-op.live-federationruns it with GitHub Actions OIDC (id-token: write); it activates once the repo Actions variablesMULTISTORE_TEST_ROLE_ARN/MULTISTORE_TEST_BUCKET/MULTISTORE_TEST_REGIONare set and the role's trust policy trusts this repo's GitHub OIDC subject (audiencests.amazonaws.com).cargo test -p multistore-backend-federation— 11 unit tests + 1 doctest + the (skipping) live test. Existingoidc-providertests all pass against the delegated path.Unit tests
Response parse, STS error path, secret redaction (creds + applied
BucketConfig), request shape (incl. session policy,DurationSecondspresent/omitted, unencodedform_pairs), regional endpoint, andBucketConfiginjection.Notes
sts+oidc-providerpair is also "federation"); the crate is named backend-federation and its docs reserve bare "federation" for the concept.Cargo.lockis updated to reflect the crate rename and the newoidc-provider → backend-federationedge (required for--lockedCI).Follow-ups (tracked in #56)
oidc-providerhas feature-gated Azure/GCP exchange stubs today; extracting them intobackend-federation(and modelling bearer-only credentials) is future work.