feat: AFD-first-party Layer-7 GLB + WAF + Private Link (SFI-NS253)#373
Draft
rchincha wants to merge 36 commits into
Draft
feat: AFD-first-party Layer-7 GLB + WAF + Private Link (SFI-NS253)#373rchincha wants to merge 36 commits into
rchincha wants to merge 36 commits into
Conversation
…NS253 Adds docs/first-party/ folder with an index README and proposal 001 outlining how to extend fleet-networking beyond ATM to satisfy SFI-NS253 for first-party AKS workloads via Azure Front Door Standard/Premium with an attached WAF policy and Private Link origins. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 10282cca-4848-413e-a12a-f6cf963a72ec
…osal 002) Adds docs/first-party/002-afd-implementation-plan.md — the executable companion to proposal 001. Covers guiding conventions extracted from the existing ATM implementation, a complete file-by-file scope table (add/modify/generated), Go type sketches with CEL rules, controller skeletons, cmd/hub-net-controller-manager/main.go diff, a 5-phase delivery plan, backward-compatibility notes, risks, and success criteria. Also updates docs/first-party/README.md to index proposal 002. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 10282cca-4848-413e-a12a-f6cf963a72ec
rchincha
marked this pull request as draft
July 15, 2026 18:53
rchincha
changed the title
Proposal 002: add new section 8 'East-west (MCS) compatibility' covering guarantees for the ServiceExport/ServiceImport/EndpointSlice* data plane, guardrails on the ExportMode transition, interaction with the existing TrafficManagerBackend controller, and required non-regression test coverage. Renumber subsequent sections. Proposals 001 and 002: call out that AFD is L7-only (HTTP/HTTPS + WebSockets-over-HTTPS) and that L4 first-party workloads are not covered by this proposal — future L4 SFI-NS253 compliance likely via Azure Cross-region Load Balancer. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 10282cca-4848-413e-a12a-f6cf963a72ec
Adds docs/first-party/003-pre-implementation-checklist.md — converts the open design questions from proposal 001 and the unstated assumptions in proposal 002 into a trackable list of gates: - open design decisions (profile scope, WAF requiredness, custom domains phase, umbrella CRD, adding Spec to ServiceExport) - external review / sign-off gates (maintainer, SFI, PM, security) - spikes (AKS PLS status annotation, armcdn SDK compat, cross-sub PLS auto-approval, RBAC minimums, dev-sub, chart cloud-config interaction) - overall readiness verdict with per-phase gating rules Also indexes proposal 003 in the folder README. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 10282cca-4848-413e-a12a-f6cf963a72ec
…profile scope Proposal 001: - 2.1: add explicit traffic-isolation property (AFD PoP → Microsoft backbone → PE → ILB → pod, no public internet leg, no public IP on origin) with ASCII flow diagram. - 2.1: state AFD Premium (not Standard) as the required SKU. - 2.3: expand the non-goal on non-Premium SKUs — Standard is out of scope because Private Link origins are Premium-only. - 3.2: annotate the topology description with the backbone-isolation guarantee. Proposal 003: - 1.1: resolve profile scope as **namespaced**, with rationale tying ownership to the PLS/Service/namespace boundary and calling out that the per-flow isolation property does not require a cluster-scoped profile. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 10282cca-4848-413e-a12a-f6cf963a72ec
…equiredness (1.2) Proposal 002: - 3.1: replace SKU enum with single-value Premium (Standard removed per proposal 001 2.3). Add a first-class spec.complianceMode: None | SFI-NS253 field (default None, immutable). CEL requires spec.wafPolicy when complianceMode is SFI-NS253. - 4.2: describe the FrontDoorBackend reconciler's cross-check that refuses privateLink=false when the referenced profile is in SFI mode (surfaced as Accepted=False, Reason=SFIComplianceViolation). Closes the loophole of an SFI-compliant profile with a lax backend. Proposal 003: - 1.2: resolve as 'required when complianceMode == SFI-NS253', with rationale explaining why an explicit Spec field beats an annotation-based marker (auditable, mistype-safe, kubectl- discoverable, drives both profile and backend enforcement). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 10282cca-4848-413e-a12a-f6cf963a72ec
…-frontdoor-feature
Introduces two new namespaced CRDs and their reconcilers in the hub-net-controller-
manager, gated by a new feature flag that defaults to false so existing ATM-only
installs are unaffected. Scope is intentionally a proof-of-concept; charts, envtest /
E2E, and BYOC (Key Vault) TLS are deferred per the decisions recorded in the breadcrumb.
New CRDs (api/v1alpha1, categories={fleet-networking})
* FrontDoorProfile (shortName: afdp)
- Immutable ResourceGroup and Sku (Standard_AzureFrontDoor | Premium_AzureFrontDoor).
- Status: ResourceID, EndpointHostname, Programmed condition
(reasons: Programmed / Invalid / AzureError / Pending).
* FrontDoorCustomDomain (shortName: afdcd)
- Immutable same-namespace ProfileRef and Hostname (D4).
- TLS.Mode enum (Managed | BYOC) with CEL cross-field validation on
keyVaultCertificate; BYOC field shape reserved but reconciliation deferred (D3).
- Status: ResourceID, ValidationState, DNSValidationToken, DNSValidationExpiry,
Programmed condition (reasons: Programmed / Invalid / ProfileNotReady /
AwaitingDNSValidation / ValidationFailed / TLSFailed / AzureError / Pending).
Azure client factory (pkg/common/azurefrontdoor)
* Workload Identity via azidentity.NewWorkloadIdentityCredential (D6);
reads AZURE_TENANT_ID / AZURE_CLIENT_ID / AZURE_FEDERATED_TOKEN_FILE /
AZURE_SUBSCRIPTION_ID with aggregated validation errors.
* Bundles armcdn Profiles, AFDEndpoints, and AFDCustomDomains sub-clients
behind a small NewClients() seam.
* Kept strictly separate from pkg/common/azuretrafficmanager per D7 (no umbrella
GlobalLoadBalancer abstraction).
* Unit tests cover env-var validation and nil-guard paths.
Controllers (pkg/controllers/hub/frontdoorprofile, pkg/controllers/hub/frontdoorcustomdomain)
* Standard controller-runtime pattern: get -> handleDelete | handleUpdate, with the
finalizer registered immediately before the first Azure call to avoid stuck-delete
on persistent 403.
* FrontDoorProfile: ensures profile + default AFD endpoint (Location=Global),
populates status, emits Programmed/AzureAPIError Events. Azure names are
fleet-<UID> so they are stable across CR renames.
* FrontDoorCustomDomain: BYOC rejected up front with a permanent Invalid condition
(D3); waits for parent profile Programmed=True before touching Azure; polls
validation state every 30s while Pending/Submitting; surfaces the DNS TXT record
that the user must create at _dnsauth.<hostname>.
* Both controllers use predicate.GenerationChangedPredicate{} and share the
azureerrors helpers for classification (IsNotFound / IsForbidden / IsClientError /
IsThrottled).
Wiring (cmd/hub-net-controller-manager/main.go)
* New flag --enable-frontdoor-feature (default false) with justifying inline
comment: existing installs do not have WI + AFD-scoped subscription config,
and the ATM branch is untouched.
* New frontDoorFeatureRequiredGVKs CRD-presence check mirroring the ATM one.
* New if-block constructs azurefrontdoor.{LoadConfigFromEnv, NewCredential,
NewClients} and registers both reconcilers on the manager.
* Zero changes to the ATM branch, cloudConfigFile plumbing, or
charts/hub-net-controller-manager/, honouring the D9 zero-diff compatibility
contract. Baseline `helm template` renders + SHA-256 hashes captured under
.github/.copilot/breadcrumbs/baselines/ as a regression fixture.
Finalizer constants (pkg/common/objectmeta/objectmeta.go)
* FrontDoorProfileFinalizer = networking.fleet.azure.com/frontdoor-profile-cleanup
* FrontDoorCustomDomainFinalizer = networking.fleet.azure.com/frontdoor-custom-domain-cleanup
Dependencies
* Adds github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/cdn/armcdn v1.1.1.
Decisions (see breadcrumb 2026-07-18-1717-frontdoor-phase2-custom-domains.md)
D1 Custom domains land in Phase 2, not Phase 5 (mandatory for production).
D2 Separate CRD for custom domains (lifecycle + ownership, not frequency).
D3 Managed TLS only in the POC; BYOC field reserved, reconciliation deferred.
D4 Same-namespace ProfileRef; no cross-namespace refs.
D5 Status + Kubernetes Events; token in status, event emission for token deferred.
D6 Azure AD Workload Identity for AFD auth (not azure.json / MSI).
D7 No umbrella GlobalLoadBalancer CRD; AFD and ATM helpers stay separate.
D8 Do not deviate from upstream mcs-api; carry Fleet-specific config via
annotations under networking.fleet.azure.com/ rather than new Spec fields.
D9 Ship AFD as a future sibling chart; do not modify the existing ATM chart.
ATM `helm template` baselines captured for regression.
Validation
* `go build ./...` and `go vet ./...` both green.
* `go test ./pkg/common/azurefrontdoor/...` passing.
Not in this change (deferred past POC)
* Envtest / E2E tests for the new controllers.
* New charts/hub-afd-controller-manager/ sibling chart with WI ServiceAccount
annotations and AFD-only RBAC.
* BYOC (Key Vault) TLS reconciliation, secret rotation.
* Kubernetes Event emission for the DNS validation token.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 8e159d50-fd15-4d17-b92a-5c5ced921269
…erve mcs-api parity Drop the proposed ServiceExport.Spec.ExportMode field. Instead: - Primary signal: infer L7-FrontDoor from the Service's azure-load-balancer-internal + azure-pls-* annotations. - Opt-in flag: networking.fleet.azure.com/export-mode annotation on ServiceExport (matches the existing weight-annotation precedent). - Precedence: annotation wins if set; mismatch surfaces ServiceExportValid=False, Reason=ExportModeAnnotationServiceMismatch (no silent fallback). - Member controller no longer mutates the tenant-owned Service. Updates proposals 001, 002, and closes checklist item 1.5 in 003. Adds a session breadcrumb documenting rationale and references KEP-1645 for mcs-api parity. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: a8fabc68-e113-49ef-b82d-d99a8502da11
AKS Automatic: - Proposal 001 §3.4: declare AKS Standard and AKS Automatic both supported. Enumerate cluster-side prerequisites (Standard LB, BYO VNet, PLS NAT subnet with privateLinkServiceNetworkPolicies: Disabled, PLS auto-approval for the AFD subscription) and the Automatic-specific considerations (Deployment Safeguards, NAP, cluster-config lockdown, AGC coexistence). - Proposal 001 §2.3: list member-cluster provisioning as a non-goal. - Proposal 002 §6.5: chart hygiene required by Automatic Safeguards (resource limits, runAsNonRoot, readOnlyRootFilesystem, no hostPath, allow-listed registries, PDBs), file table, NAP pod spread guidance, e2e coverage note, doc pointers. - Proposal 002 §9: add risks for Safeguards drift and NAP-driven controller restarts. - Proposal 002 §11: add success criterion 7 (clean install on an AKS Automatic member in Enforcement mode). - Proposal 003 §1.6: resolve — Automatic is first-class. - Proposal 003 §3.7: new spike — Deployment Safeguards install validation (helm template + offline policy check). - Proposal 003 §6 readiness table: add Automatic install row. AFD/ATM coexistence: - Proposal 001 §3.5 (new): coexistence-granularity table (fleet-wide safe, same-namespace-different-Service safe, same-ServiceImport-on-both-surfaces forbidden per Proposal 002 §8.4). Security implications: ATM public IP remains a bypass surface, split identities mandatory, WAF is per-surface, audit split. Tenancy implications: namespace RBAC unchanged, per-Service surface choice, weight-annotation semantics differ, per-surface quotas, cost attribution via per-tenant RGs. Staged ATM→AFD migration playbook (no controller hot swap). Session breadcrumb updated with an addendum documenting both additions. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: a8fabc68-e113-49ef-b82d-d99a8502da11
Reconcile Proposals 001/002/003 with the AFD POC that landed as cb02d14 ('feat(hub): POC for Azure Front Door (AFD) controllers behind --enable-frontdoor-feature'). Explicitly flag incompatibilities and impossibilities uncovered during the reconciliation pass, rather than silently resolving them. Decisions folded in: * Sku: Premium-only (Proposal 001 sec. 2.3 + SFI-NS253). POC currently accepts Standard too; documented as a Phase-4 CRD tightening. * Identity split: sibling binary+chart (cmd/hub-afd-controller-manager + charts/hub-afd-controller-manager) is the target model, required for Proposal 001 sec. 7 to be satisfiable. POC hosts both AFD and ATM controllers in the same pod and shares one Workload-Identity subject; flagged as a hard GA prerequisite in Proposal 003 sec. 2.4. * Custom domains: shipped as a separate CRD FrontDoorCustomDomain (afdcd) in cb02d14; documented as Proposal 001 sec. 4.1.3 and Proposal 002 sec. 3.3. Managed TLS reconciled; BYOC deferred. This is a docs-only commit; no code changes. The sibling binary+chart split lands in a follow-up commit. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: a8fabc68-e113-49ef-b82d-d99a8502da11
Add reader-note callouts at the top of Proposals 001/002/003 pointing
at the POC-vs-target sections and the reconciliation breadcrumb.
Expand code comments (no behavior change) in the six AFD files that
readers most often land on:
* api/v1alpha1/frontdoorprofile_types.go - Sku enum rationale
(POC accepts Standard; SFI-NS253 requires Premium; Phase-4 tightening
tracked in Proposal 002 sec. 9) and why FrontDoorProfileSpec has no
Location field (AFD is global-only).
* api/v1alpha1/frontdoorcustomdomain_types.go - BYOC deferral pointer
to Proposal 002 sec. 9, and a design note that a validated custom
domain does not front traffic without a FrontDoorBackend route.
* pkg/controllers/hub/frontdoorprofile/controller.go - expand the POC
scope note to list the actual gaps (no FrontDoorBackend, permissive
Sku, missing Spec fields) and cross-reference the SFI identity-split
caveat.
* pkg/controllers/hub/frontdoorcustomdomain/controller.go - same
treatment; also spell out how tenants read the DNS validation token.
* pkg/common/azurefrontdoor/client.go - expand the package doc to
explain the Workload-Identity choice (vs. mounted MI / azcloudconfig),
the env-var contract, and the POC-bridge caveat that AFD currently
shares its WI subject with the ATM controller.
* cmd/hub-net-controller-manager/main.go - flag the
--enable-frontdoor-feature block as a POC bridge and describe the
intended sibling cmd/hub-afd-controller-manager + chart migration.
Verified: go vet ./api/v1alpha1/... ./pkg/common/azurefrontdoor/...
./pkg/controllers/hub/frontdoorprofile/...
./pkg/controllers/hub/frontdoorcustomdomain/...
./cmd/hub-net-controller-manager/... passes.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: a8fabc68-e113-49ef-b82d-d99a8502da11
The POC accepted both `Standard_AzureFrontDoor` and
`Premium_AzureFrontDoor`. SFI-NS253 requires Private Link origins,
which are Premium-only, so a Standard profile can never satisfy the
first-party compliance envelope. Tighten the CRD enum so the mismatch
is rejected at admission time rather than surfaced hours later as
`Programmed=False` when a backend attempts to attach a Private
Link origin.
Changes
- api/v1alpha1/frontdoorprofile_types.go:
* Drop the `FrontDoorProfileSkuStandard` constant.
* Restrict `+kubebuilder:validation:Enum` on `Spec.Sku` to
`Premium_AzureFrontDoor`.
* Add `+kubebuilder:default=Premium_AzureFrontDoor` so tenants
can omit the field; the CRD supplies the correct value.
* Rewrite type and field-level doc comments to explain why the
enum is single-valued (retained for additive future SKUs) and
why it is immutable (AFD does not support in-place SKU upgrades).
- pkg/controllers/hub/frontdoorprofile/controller.go:
* Delete the outdated POC-caveat comment that flagged the loose
enum as a Phase-4 gap; that gap is now closed.
- docs/first-party/001-afd-global-load-balancing.md:
* Update the illustrative `FrontDoorProfileSpec` block to
mirror the tightened enum and its rationale.
- docs/first-party/002-afd-implementation-plan.md:
* Drop the POC-status paragraph on `FrontDoorSKUPremium`.
* Mark the §9 risk row for POC Sku permissiveness as resolved
(kept for change-history clarity; will be removed in the final
cleanup commit).
Follow-up (required, done separately in WSL by the user):
- Run `make manifests` to regenerate
`config/crd/bases/networking.fleet.azure.com_frontdoorprofiles.yaml`.
Until the regenerated CRD is applied to a cluster, admission will
still accept `Standard_AzureFrontDoor` even though the Go source
no longer defines it.
Validation
- `go vet ./api/v1alpha1/... ./pkg/controllers/hub/frontdoorprofile/...`
passes.
- `go build ./...` succeeds.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: a8fabc68-e113-49ef-b82d-d99a8502da11
Runs the `make manifests` output that pairs with 2fecf8d (Sku Premium-only). Kept as a separate commit for reviewability: the YAML here is 100% derived from the Go source in the previous commit — no hand edits — so this diff can be reviewed as "controller-gen output" rather than reading it alongside logic changes. Regenerated by: `make manifests` (WSL, controller-gen v0.20.0). Changes in the schema - `spec.sku.default` set to `Premium_AzureFrontDoor`. Users who omit the field get the correct value at admission time; no client-side defaulting is required. - `spec.sku.enum` reduced to a single value (`Premium_AzureFrontDoor`). `Standard_AzureFrontDoor` is now rejected at admission — no more silent-then-fail behavior once a Private-Link backend is attempted. - `spec.sku.description` updated to explain why the enum is intentionally single-valued (see the Go type comment for the full SFI-NS253 rationale). - The `required: [sku]` constraint is preserved but harmless: CRD defaulting runs before schema validation, so tenants can still omit `sku` in their YAML and receive the Premium default. Compatibility note - Any existing `FrontDoorProfile` object already stored with `sku: Standard_AzureFrontDoor` will fail re-validation on next write. There are no such objects in production (the POC is dev-only). If any local dev cluster has one, delete it before applying the new CRD. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: a8fabc68-e113-49ef-b82d-d99a8502da11
Proposal 001 §7 (SFI-NS253) requires the AFD reconcilers to run under a
Workload-Identity federated subject that is distinct from the ATM
reconcilers' subject, so ATM-only tenants do not inherit AFD write
permissions on the shared Azure subscription. A Kubernetes pod projects
exactly one WI token, so the split must happen at the Pod boundary —
two binaries, two Deployments, two ServiceAccounts.
Changes
- cmd/hub-afd-controller-manager/main.go (new): dedicated binary that
hosts only the FrontDoorProfile and FrontDoorCustomDomain reconcilers.
Distinct from the ATM binary in three concrete ways so both can run on
the same hub simultaneously:
* leader-election ID `afd.hub.networking.fleet.azure.com` (vs.
`2bf2b407.hub.networking.fleet.azure.com`).
* metrics port :8082, probe port :8083 (vs. :8080 / :8081).
* Workload Identity from AZURE_* env vars only — no `--cloud-config`
flag, no `azure.json` dependency.
Scheme registration is limited to clientgoscheme + v1alpha1 (the group
that owns the AFD CRDs); v1beta1 and clusterv1beta1 are intentionally
not registered so this binary cannot reconcile ATM or MemberCluster
objects even if their CRDs happen to be installed on the same cluster.
A webhook.NewServer is initialized (though no webhooks are registered
yet) so a future FrontDoorBackend coexistence webhook (Proposal 001
§3.5) can be added without a chart/binary co-change.
- cmd/hub-net-controller-manager/main.go: delete the entire
`if *enableFrontDoorFeature { ... }` block, the `enableFrontDoorFeature`
flag, the `frontDoorFeatureRequiredGVKs` var, and the three AFD-only
imports (`pkg/common/azurefrontdoor`, `pkg/controllers/hub/frontdoorprofile`,
`pkg/controllers/hub/frontdoorcustomdomain`). The v1alpha1 scheme
registration is preserved because InternalServiceExport /
InternalServiceImport / EndpointSliceExport / ServiceImport all live
in v1alpha1 and are still reconciled here.
A short NOTE comment is left where the flag used to be, pointing
operators at the new binary so a future reader is not surprised by the
missing option.
Deployment consequence (intentional)
- No chart wiring is added in this commit. Until
charts/hub-afd-controller-manager lands (next commit in this series),
the AFD binary is not deployable via helm. That is deliberate: the
shared-subject POC deployment path is being CLOSED before the new
identity-split path opens, so an operator cannot silently regress by
keeping the old install. Users who need to test AFD in the interim
can run the new binary manually against a hub kubeconfig with the
`AZURE_*` env vars set.
Validation
- `go vet ./cmd/...` passes.
- `go build ./cmd/hub-net-controller-manager/... ./cmd/hub-afd-controller-manager/...`
succeeds.
- Full `go build ./...` succeeds — the moved packages are still
compiled (via the new binary), so no downstream reference breaks.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: a8fabc68-e113-49ef-b82d-d99a8502da11
Wires up the new AFD binary (added in fcb37f2) to the container image build pipeline, mirroring the hub-net-controller-manager targets so both binaries are treated symmetrically by CVE scanning, release tagging, and the aggregate ``image`` / ``push`` targets. Changes - docker/hub-afd-controller-manager.Dockerfile (new): copy of docker/hub-net-controller-manager.Dockerfile with the binary name swapped. Kept intentionally in sync — same base (mcr.microsoft.com/oss/go/microsoft/golang:1.25.12), same distroless runtime (mcr.microsoft.com/azurelinux/distroless/base:3.0), same non-root UID (65532), same CGO_ENABLED=1 build flag — so both images fall under the same supply-chain policy set. Only main.go from cmd/hub-afd-controller-manager is copied (not the whole cmd/ tree) to keep the image layer minimal. - Makefile: * Add HUB_AFD_CONTROLLER_MANAGER_IMAGE_NAME (default hub-afd-controller-manager) and HUB_AFD_CONTROLLER_MANAGER_IMAGE_VERSION (defaults to $(TAG) like every other image). * Add bin/hub-afd-controller-manager to the build target so "make build" produces both hub binaries. * Add a run-hub-afd-controller-manager convenience target for local dev, with a comment about the required AZURE_* env vars. * Add docker-build-hub-afd-controller-manager mirroring the existing per-image build targets. * Extend the aggregate image and push targets to include the new image so "make image" / "make push" cover both hub binaries without a second command. The docker target is kept as its own recipe (rather than folding into a matrix over an image list) so operators iterating on just AFD during POC can rebuild that one image without also rebuilding the ATM one. Validation (in WSL) - "make build" succeeds and produces all four binaries. - "make -n docker-build-hub-afd-controller-manager" (dry-run) expands to the expected "docker buildx build" invocation with the correct Dockerfile path, tag, and platform args. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: a8fabc68-e113-49ef-b82d-d99a8502da11
Adds the second half of the identity-split wiring (the first half was the binary in fcb37f2 and its image build in 5672313). With this chart the AFD controller is deployable via helm again — restoring the install path that was intentionally closed at commit fcb37f2 — but this time under a distinct Workload-Identity federated subject as required by Proposal 001 §7 (SFI-NS253). Chart layout - Chart.yaml, .helmignore, values.yaml, README.md. - templates/_helpers.tpl — name/label helpers mirroring the hub-net chart so a reader familiar with the ATM chart has zero surprise. - templates/serviceaccount.yaml — ServiceAccount with the azure.workload.identity/use label and azure.workload.identity/client-id + azure.workload.identity/tenant-id annotations. This is the SFI-NS253 §7 payoff: an AAD identity distinct from the ATM chart is bound HERE. - templates/rbac.yaml — least-privilege ClusterRole + ClusterRoleBinding. Verbs mirror the +kubebuilder:rbac markers on the two AFD reconcilers. Deliberately NO verbs on ATM/MCS/MemberCluster CRDs so RBAC also enforces the split at the K8s layer (defense in depth). - templates/deployment.yaml — 2 replicas by default with pod anti- affinity, distinct container ports (:8082 metrics / :8083 probe) so both binaries can co-exist on one node without collision, and only AZURE_SUBSCRIPTION_ID is set explicitly (the other three AZURE_* vars are auto-injected by the workload-identity webhook). - templates/poddisruptionbudget.yaml — PDB (minAvailable=1) so node drains and NAP scaling on AKS Automatic cannot take both replicas down simultaneously. Fail-fast wiring - values.yaml requires azure.tenantId, azure.clientId, and azure.subscriptionID to be non-empty; the templates use `required` so helm install fails at admission with a clear error rather than producing pods that CrashLoopBackOff on token load. Verified: `helm template ... --set azure.tenantId=t --set azure.subscriptionID=s` (omitting clientId) exits with "Error: execution error ... azure.clientId is required for Workload Identity". CRDs - The chart does NOT install CRDs. The shared net-crd-installer job attached to the hub-net-controller-manager chart is authoritative for CRD install; documented in README.md prerequisites. hub-net chart - Unmodified. `helm lint charts/hub-net-controller-manager` still passes with zero errors. Validation - `helm lint charts/hub-afd-controller-manager --set azure.tenantId=t --set azure.clientId=c --set azure.subscriptionID=s` -> 0 failed. - `helm template rel charts/hub-afd-controller-manager --set ...` renders exactly the expected 5 resources with the right ports, annotations, and RBAC verbs. Follow-ups (not this commit) - Wire the chart into any release-automation pipeline that currently packages only hub-net-controller-manager. - Add an e2e assertion that both charts install cleanly on the same hub in the same namespace (planned for Commit 5). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: a8fabc68-e113-49ef-b82d-d99a8502da11
rchincha
changed the title
rchincha
changed the title
Bump github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/cdn/armcdn from v1.1.1 to v2.2.0. This is a prerequisite for the AFD controller tests (next commit): armcdn/v2 ships an armcdn/v2/fake package with ProfilesServerTransport / AFDEndpointsServerTransport / AFDCustomDomainsServerTransport, which mirrors the pattern already established for the ATM controllers in test/common/trafficmanager/fakeprovider. v1.x has no fake subpackage so testing against that version would require hand-rolled fakes and would diverge from the existing convention. Compatibility - All type and constant names used by fleet-networking are unchanged between v1.1.1 and v2.2.0 (ProfilesClient, AFDEndpointsClient, AFDCustomDomainsClient, Profile, ProfileProperties, SKU, SKUName, AFDEndpoint, AFDEndpointProperties, EnabledStateEnabled, AFDDomain, AFDDomainProperties, AFDDomainHTTPSParameters, AfdCertificateTypeManagedCertificate, NewClientFactory). The import alias also stays as `armcdn` (no explicit alias needed because v2 imports as `armcdn "...cdn/armcdn/v2"` by default), so call sites are identical to the v1 versions and no logic changes. - The generated API version moves from 2023-05-01 (v1.1.1) to 2024-02-01 (v2.2.0). For the types we use, the 2024-02-01 API is a strict superset — no request/response schema field we currently read or write has been removed or renamed. - The require directive is promoted from `// indirect` to a direct require. The v1 entry was mislabeled indirect (fleet-networking has been importing armcdn directly since cb02d14); tidy corrects it as part of the bump. Changes - go.mod / go.sum: swap armcdn v1.1.1 for armcdn/v2 v2.2.0. - pkg/common/azurefrontdoor/client.go: import path updated; a comment explains the pin rationale (fake subpackage availability). - pkg/controllers/hub/frontdoorprofile/controller.go: import path only. - pkg/controllers/hub/frontdoorcustomdomain/controller.go: import path only. Validation - `go mod tidy` succeeds; go.sum diff is limited to the armcdn line pair. - `go vet ./pkg/common/azurefrontdoor/... ./pkg/controllers/hub/frontdoor... ./cmd/hub-afd-controller-manager/...` passes. - `go build ./...` succeeds (no code changes were required — the type and function names in armcdn/v2 are identical to v1.1.1 for the surface we consume). Follow-up - Docs (docs/first-party/002 §2.1, 001 §5.4, 003 §3.2) still reference armcdn v1.1.1 by version number in illustrative text. Left alone in this commit for reviewability; a docs-only sweep at the end of the series will update the version numbers uniformly (Commit 11). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: a8fabc68-e113-49ef-b82d-d99a8502da11
TestCollectCRDFileNamesWithActualPath/hub_mode_excludes_MultiClusterService_CRD was failing after the AFD POC (cb02d14) added config/crd/bases/frontdoorprofiles.networking.fleet.azure.com.yaml and .../frontdoorcustomdomains.networking.fleet.azure.com.yaml but did not update the test's whitelisted CRD names. CollectCRDs in hub mode returns every CRD under config/crd/bases with group "networking.fleet.azure.com" (util.go lines 111-113 — no explicit hub allow-list), so those two new CRDs surface as -want +got diffs. Fix: extend the hub-mode expected list to include - frontdoorprofiles.networking.fleet.azure.com - frontdoorcustomdomains.networking.fleet.azure.com Member mode is intentionally left unchanged: AFD is a hub-only concern, and the member allow-list (memberIncludedCRDs in util.go) already correctly omits both AFD CRDs. Design note added in a block comment on the hub test case: AFD CRDs are installed onto the hub even though the AFD controller runs in a sibling pod (cmd/hub-afd-controller-manager, chart charts/hub-afd-controller-manager). The AFD/ATM identity split required by SFI-NS253 §7 is enforced at the pod/ServiceAccount layer via separate least-privilege ClusterRoles per chart; withholding CRDs from the hub would not add security value and would break the AFD reconciler on install. Validation - `go test ./cmd/net-crd-installer/utils/ -run TestCollectCRDFileNamesWithActualPath -v` passes both sub-tests locally. - Full test suite otherwise unaffected (this test file is the only consumer of the expected list). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: a8fabc68-e113-49ef-b82d-d99a8502da11
Introduces the minimal test scaffolding for the FrontDoorProfile and
FrontDoorCustomDomain controllers, mirroring the shape already
established for the ATM controllers under
pkg/controllers/hub/trafficmanagerprofile and
test/common/trafficmanager/fakeprovider. This unblocks incremental spec
growth as the AFD reconcilers mature in commits 6-11 (WAFPolicy,
ComplianceMode, FrontDoorBackend, coexistence guard, etc.).
Deliberate choice: land the SCAFFOLD (fakes + suite bootstrap + one
happy-path spec per package + one BYOC-guard spec) rather than a full
error-injection matrix. Rationale in a design comment on each spec
file: the reconcilers themselves are still POC-quality (deferred error
classification, no metrics, no FrontDoorBackend integration), so
brittle full-matrix specs written now would need extensive rewriting
as the reconcilers evolve. Each spec below can be extended
independently.
New: test/common/azurefrontdoor/fakeprovider
- profile.go — stateful in-memory fake for armcdn.ProfilesClient.
Uses the SDK-shipped armcdn/v2/fake package (unlocked by the
armcdn v1 -> v2 bump in the prior commit). Get/BeginCreate/BeginDelete
keyed by profile name in a mutex-guarded map, so
create-then-get returns the created object and delete-then-get
returns NotFound — matching real AFD semantics closely enough to
exercise the reconciler's "get before create" idempotency pattern.
Magic profile names ConflictErrProfileName /
InternalServerErrProfileName trigger canned LRO errors for future
error-injection specs (unused today).
- endpoint.go — stateful fake for armcdn.AFDEndpointsClient. Synthesizes
a *.azurefd.net HostName on create so the reconciler's
status.endpointHostname population path is exercised (a nil HostName
would silently mask bugs there).
- customdomain.go — stateful fake for armcdn.AFDCustomDomainsClient.
Design tradeoff called out in the package doc comment: returns
DomainValidationStateApproved IMMEDIATELY on create rather than
simulating the real AFD Pending -> Approved DNS-validation dance.
This covers the happy-path status-population code with minimal
scaffolding; a future spec that needs pending-state coverage can
override the beginCreate handler through a functional option.
New: pkg/controllers/hub/frontdoorprofile
- suite_test.go — envtest bootstrap (CRDs from
../../../../config/crd/bases), controller-runtime manager, Reconciler
wired with fake ProfilesClient + AFDEndpointsClient. Same shape as
trafficmanagerprofile/suite_test.go.
- controller_integration_test.go — one spec: create a
Premium FrontDoorProfile, assert Programmed=True with the fake-derived
hostname surfaced in status.endpointHostname, ResourceID shape, and
finalizer added. AfterEach deletes the CR and waits for the finalizer
to run so specs are isolated.
New: pkg/controllers/hub/frontdoorcustomdomain
- suite_test.go — envtest bootstrap wired with fake AFDCustomDomainsClient.
Note: this suite does NOT wire the FrontDoorProfile reconciler; specs
synthesize the parent profile's Programmed=True condition directly via
stampProfileProgrammed helper, keeping the two controller test surfaces
independently isolated.
- controller_integration_test.go — two specs:
1. Happy path — parent Programmed, TLS Mode=Managed, hostname
www.contoso.com. Asserts Programmed=True + ValidationState=Approved
+ DNSValidationToken populated + finalizer.
2. BYOC guard (D3 semantics) — TLS Mode=BYOC with a valid Key Vault
ref. Asserts Programmed=False, Reason=Invalid, and CRUCIALLY that
NO finalizer is added — a BYOC-rejected CR must remain deletable
(there is no Azure resource for the delete path to clean up).
Validation
- Windows: `go vet ./test/common/azurefrontdoor/... ./pkg/controllers/hub/frontdoor...`
passes; `go test -c -o NUL` for both AFD test packages compiles.
- WSL: `make local-unit-test` runs both AFD suites to green as part of
the existing hub controller test set.
Follow-up
- Commits 6-11 will grow each suite additively as reconciler behavior
expands (e.g. a spec for WAFPolicy reference in Commit 7, coexistence
guard in Commit 10). The stateful fake providers already support this
because they key state per-resource-name, so a new spec can simply
choose a fresh profile / custom-domain name.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: a8fabc68-e113-49ef-b82d-d99a8502da11
Extends FrontDoorProfileSpec with the two fields that gate the
SFI-NS253 enforcement flow described in
docs/first-party/001-afd-global-load-balancing.md §2.1 and
docs/first-party/003-pre-implementation-checklist.md §1.2.
Additions
- `FrontDoorProfileComplianceMode` string type with enum values
`"None"` (default) and `"SFI-NS253"`. Immutable: switching a live
profile out of SFI-NS253 would silently weaken the guarantees the
operator relied on at creation time; recreate the profile if the
compliance regime legitimately needs to change.
- `FrontDoorProfileSpec.ComplianceMode` — optional, defaults to `"None"`,
enum-restricted, immutable via CEL `self == oldSelf`.
- `FrontDoorProfileSpec.WAFPolicy` — optional `*FrontDoorWAFPolicyRef`.
Cross-field CEL on the FrontDoorProfile object requires `WAFPolicy` to
be set whenever `ComplianceMode == "SFI-NS253"`, so a misconfigured
SFI-NS253 profile never even reaches the reconciler.
- `FrontDoorWAFPolicyRef` with a single required `ResourceID` field
validated by regex against the
`Microsoft.Network/frontdoorwebapplicationfirewallpolicies` ARM ID
layout. Cross-subscription references are supported at the Azure
level.
- Two new condition reasons on `FrontDoorProfileConditionProgrammed`:
* `WAFPolicyNotFound` — policy ARM ID unresolvable OR the AFD
controller's identity lacks read on the policy scope (the two failure
modes are indistinguishable from the AFD RP; both surface as this
reason with the exact ID in the condition message per
docs/first-party/002-afd-implementation-plan.md §11).
* `WAFPolicyNotInPreventionMode` — policy resolves but its
PolicySettings.Mode is not "Prevention". Only enforced when
ComplianceMode == "SFI-NS253"; Detection mode remains legal for
tenants with ComplianceMode == "None" who want alerts without
blocking. Both reasons are surfaced by the reconciler in Commit 7;
this commit only introduces the constants.
Scope choices called out in code comments
- Inline WAF creation (`Inline *InlineWAFPolicy`) sketched in
Proposal 002 §3.1 is deliberately deferred. Nearly all first-party
services will reference a centrally-managed WAF policy, and exposing
inline creation would duplicate what should ultimately be its own
CRD + reconciler. When/if inline creation lands, it will be added
as an additional optional field on FrontDoorWAFPolicyRef without a
schema break. The FrontDoorWAFPolicyRef type comment records this
rationale.
- HealthProbe and OriginResponseTimeoutSeconds are also from Proposal 002
§3.1 but are separate concerns from the SFI-NS253 flow; they will land
in their own commit alongside the FrontDoorBackend reconciler (which
is where they actually get consumed).
Validation
- `go vet ./api/...` clean.
- `go build ./...` clean (compile of pointer field
`WAFPolicy *FrontDoorWAFPolicyRef` succeeds even without regenerated
DeepCopy because Go allows adding fields without breaking existing
method compilation; runtime correctness of DeepCopy is restored by
the follow-up regen commit).
Follow-up (Commit 6b)
- `make manifests` must run to regenerate:
* api/v1alpha1/zz_generated.deepcopy.go — adds
FrontDoorWAFPolicyRef.DeepCopyInto and extends
FrontDoorProfileSpec.DeepCopyInto to copy the new *FrontDoorWAFPolicyRef
pointer. Without regen, DeepCopy() produces a shallow copy where
both objects share the same pointer — a subtle mutation-safety bug.
* config/crd/bases/networking.fleet.azure.com_frontdoorprofiles.yaml —
adds the new fields, the field-level CEL rules, and the top-level
cross-field CEL rule. Without regen the API-server admission layer
does not know about complianceMode / wafPolicy at all.
- The reconciler wiring for WAFPolicy attach + ComplianceMode enforcement
is Commit 7.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: a8fabc68-e113-49ef-b82d-d99a8502da11
Regenerated artifacts for the API additions in the prior commit (feat(api): add ComplianceMode + WAFPolicy fields to FrontDoorProfile). api/v1alpha1/zz_generated.deepcopy.go (make generate): - FrontDoorProfile.DeepCopyInto now calls Spec.DeepCopyInto instead of the previous `out.Spec = in.Spec` byte copy — required because FrontDoorProfileSpec now has a pointer field (WAFPolicy). - FrontDoorProfileSpec.DeepCopyInto grew a nil-checked `*out = new(FrontDoorWAFPolicyRef); **out = **in` block for WAFPolicy. Without this, a DeepCopy() would produce two objects sharing the same WAFPolicy pointer — a subtle mutation-safety bug. - FrontDoorWAFPolicyRef.DeepCopy / DeepCopyInto pair added (autogenerated, all-string fields today so it's a simple `*out = *in`). config/crd/bases/networking.fleet.azure.com_frontdoorprofiles.yaml (make manifests): - New schema properties: `complianceMode` (enum: None, SFI-NS253; default None) and `wafPolicy` (object with required `resourceID` matched against the AFD WAF ARM ID regex). - New field-level CEL: `complianceMode is immutable`. - New top-level CEL under the FrontDoorProfile object: `spec.wafPolicy is required when spec.complianceMode is SFI-NS253`. This is the admission-time enforcement that keeps a misconfigured SFI-NS253 profile from ever reaching the reconciler. No hand edits — both files are byte-identical to what `make generate` and `make manifests` produce against the API types committed in aeb116c. Validation - `go build ./...` clean (unchanged from the prior commit). - CRD YAML diff confined to complianceMode + wafPolicy schema blocks and one top-level x-kubernetes-validations entry; no other property or existing rule was touched. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: a8fabc68-e113-49ef-b82d-d99a8502da11
…undle
Extends the Clients bundle to cover WAF enforcement, which spans two SDKs:
- armcdn/v2 SecurityPoliciesClient attaches a WAF policy to an AFD profile
(Microsoft.Cdn/profiles/{}/securityPolicies/{}).
- armfrontdoor PoliciesClient reads/writes the referenced WAF policy
(Microsoft.Network/frontdoorwebapplicationfirewallpolicies), which lives
under a different ARM resource provider that armcdn deliberately does
not expose.
Both sub-clients share the caller-supplied credential and arm.ClientOptions
so retry/telemetry policy remains uniform. armfrontdoor is pinned to v1.4.0
because it is the first release shipping a fake subpackage (PoliciesServer)
mirroring the armcdn/v2/fake pattern used elsewhere in this repo.
No behavior change yet -- the profile reconciler still ignores WAFPolicy.
Enforcement (resolve -> validate -> attach) lands in the next commit.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: a8fabc68-e113-49ef-b82d-d99a8502da11
…curityPolicy attach Extends the FrontDoorProfile reconciler with the WAF enforcement path described in docs/first-party/002-afd-implementation-plan.md Azure#11: - spec.wafPolicy nil + ComplianceMode=None -> no WAF; any prior fleet-managed SecurityPolicy is drift-deleted so a user who clears the field does not leave stale WAF attachment behind. Programmed=True. - spec.wafPolicy set -> resolve the referenced Microsoft.Network/ frontdoorwebapplicationfirewallpolicies resource via armfrontdoor PoliciesClient, then upsert a Microsoft.Cdn/profiles/{}/securityPolicies/ fleet-waf-{uid} of type WebApplicationFirewall binding the policy to the default AFD endpoint (path "/*"). Programmed=True. - ComplianceMode=SFI-NS253 -> additionally require PolicySettings.Mode == Prevention; Detection or nil-Mode is rejected with Reason=WAFPolicyNotInPreventionMode so an SFI-audited tenant cannot ship a WAF policy that only observes traffic. Terminal client rejections (WAFPolicyNotFound, WAFPolicyNotInPreventionMode, malformed ID, cross-subscription ref) short-circuit before Programmed=True is set, so a single Programmed condition remains authoritative for readiness. Cross-subscription refs are intentionally rejected in POC scope because the Clients bundle does not build a per-sub armfrontdoor client yet; the message tells the operator exactly why. Reconciler gains WAFPoliciesClient, SecurityPoliciesClient, and SubscriptionID fields (last cached because neither SDK client exposes it and the resolver needs it for cross-sub detection). Test scaffolding: - WAFPolicyFake exposes SetPolicy/DeletePolicy for pre-seeding a policy at a specific PolicyMode; the reconciler only reads so the store is test-owned. - SecurityPolicyFake exposes GetStored so specs can assert on the exact ARM ID, WafPolicy.ID, and Association domains the reconciler wrote. - Endpoint fake now populates .ID on create; a nil ID would silently produce an empty Association and mask bugs. No new envtest specs land in this commit -- suite_test.go only wires the fakes so the existing happy-path spec keeps passing. Specs exercising the three WAF paths (happy / NotFound / NotInPrevention) land next. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: a8fabc68-e113-49ef-b82d-d99a8502da11
…SFI-Detection Exercises the three WAF-enforcement paths added in the previous commit: - Happy path: seed a Prevention-mode WAF policy, create a profile referencing it, assert Programmed=True AND the SecurityPolicy stored in the fake carries the correct WafPolicy.ID plus an Association pointing at the default endpoint's exact ARM ID. Proves both the resolver->attach round-trip and that the reconciler propagates the endpoint Get response through to the SecurityPolicy Association. - NotFound: reference an unseeded WAF policy, assert Programmed=False with Reason=WAFPolicyNotFound. Also asserts the message contains the exact ARM ID so operators can diff spec vs Azure without extra tooling. - SFI-NS253 rejects Detection: seed a valid but Detection-mode policy, create a profile with ComplianceMode=SFI-NS253, assert Programmed=False with Reason=WAFPolicyNotInPreventionMode. Also asserts NO SecurityPolicy was written, guarding against partial-WAF leakage on the reject path. Factors a shared deleteProfile helper (finalizer-aware cleanup) so future WAF specs do not duplicate the teardown block; the pre-existing happy-path Context keeps its own AfterEach because it landed before the helper. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: a8fabc68-e113-49ef-b82d-d99a8502da11
rchincha
changed the title
…erviceExport Introduces the annotation and helper that will drive the ServiceExport split between the L4 Traffic Manager path (today) and the L7 Front Door path (Phase 4). Modeled as an annotation rather than a Spec field so the ServiceExport CRD stays identical to upstream mcs-api (KEP-1645) -- see docs/first-party/002-afd-implementation-plan.md Azure#3.4 and the breadcrumb 2026-07-20-1108-afd-export-mode-mcs-parity.md for the parity rationale. New symbols: - ServiceExportAnnotationExportMode = "networking.fleet.azure.com/export-mode" - ExportModeValueTrafficManager = "L4-TrafficManager" (default) - ExportModeValueFrontDoor = "L7-FrontDoor" - ExtractExportModeFromServiceExport(*v1beta1.ServiceExport) (string, error) Semantics chosen to fail loudly on operator error: - Absent annotation -> default TrafficManager, no error. Preserves current behaviour for every existing manifest. - Explicit valid enum member -> returned verbatim. - Empty string, wrong case, typo, anything else -> error with mode="". Empty is not silently defaulted because a mis-templated Helm value that resolves to "" would otherwise mask the intent; case sensitivity matches the CRD-side enum convention and downstream Azure APIs. No reconciler wiring yet -- that lands in commit 8c. This commit is safe to land alone: only adds symbols and a table-driven unit test. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: a8fabc68-e113-49ef-b82d-d99a8502da11
…LinkServiceResourceID Additive v1alpha1 schema change; both new fields are optional so every existing InternalServiceExport keeps validating and routes through the default L4-TrafficManager path. - ExportMode (enum L4-TrafficManager | L7-FrontDoor) is the resolved export-mode annotation from the member ServiceExport. Populated by the member serviceexport reconciler; hub controllers treat it as read-only. Empty value is equivalent to "L4-TrafficManager" so objects created before this field existed keep their prior behaviour. - PrivateLinkServiceResourceID (*string) is the ARM ID of the PLS provisioned by cloud-provider-azure for the exported Service's internal load balancer. Populated only when ExportMode == "L7-FrontDoor" AND the Service carries the azure-pls-* annotations that trigger PLS creation. The hub FrontDoorBackend reconciler (Phase 4) will wire this ID as an AFD private-link origin so global traffic can reach the ILB without traversing a public IP. CRD-side enum validation lands via the +kubebuilder:validation:Enum marker; regenerated by `make manifests`. Deepcopy regenerated by `make generate` -- the nil-checked pointer copy for the new PrivateLinkServiceResourceID field is the only functional change to zz_generated.deepcopy.go. No reconciler wiring yet -- that lands in commit 8c. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: a8fabc68-e113-49ef-b82d-d99a8502da11
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #373 +/- ##
==========================================
- Coverage 67.06% 64.49% -2.57%
==========================================
Files 33 39 +6
Lines 3783 4783 +1000
==========================================
+ Hits 2537 3085 +548
- Misses 1072 1459 +387
- Partials 174 239 +65 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
…Door Read the "networking.fleet.azure.com/export-mode" annotation on the member ServiceExport, propagate the resolved mode into the hub InternalServiceExport (empty on the wire == L4-TrafficManager for backward-compat), and add a parallel Azure Private Link Service lookup path for the L7-FrontDoor mode: * pkg/common/objectmeta: add PLS annotation consts (azure-pls-create, azure-pls-name, azure-pls-resource-group). We require an explicit azure-pls-name rather than deriving a default from cloud-provider-azure internals so the ARM Get is unambiguous and stable across upstream releases. * pkg/controllers/member/serviceexport: extend Reconciler with AzurePrivateLinkServicesClient, add invalid-export-mode / ExportModeAnnotationServiceMismatch / PrivateLinkServicePending condition reasons, and add setAzureRelatedPrivateLinkInformation mirroring setAzureRelatedInformation (ILB required, "still provisioning" branch on ARM 404 matching the PIP pattern). L4 vs L7 branch lives in exportService. * cmd/member-net-controller-manager: construct + wire the PLS client alongside the PIP client in initAzureNetworkClients. Upstream mcs-api ServiceExport Spec/Status shape is untouched (configuration flows via annotation, per KEP-1645 parity). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: a8fabc68-e113-49ef-b82d-d99a8502da11
Adds unit coverage for the new setAzureRelatedPrivateLinkInformation method (9 table-driven cases: non-LB reject, public LB reject, missing pls-create annotation, missing pls-name annotation, 404 pending, non-404 error bubbles, happy path in default RG, RG override annotation, nil-ID pending) via a new fakePrivateLinkServicesClient. Wires the fake PLS client into the ServiceExport controller suite so the integration environment has a matching PLS resource, then adds an integration spec that exercises the full L7-FrontDoor export path: member ServiceExport annotated with export-mode=L7-FrontDoor, backing Service annotated for PLS provisioning, expects the hub InternalServiceExport to carry ExportMode=L7-FrontDoor and the resolved PrivateLinkServiceResourceID — and asserts the ATM-only fields (Weight, PublicIPResourceID) remain nil so the two hub reconcilers don't overlap. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: a8fabc68-e113-49ef-b82d-d99a8502da11
Hub-scoped CRD parallel to TrafficManagerBackend: binds a
ServiceImport (via its L7-FrontDoor-mode InternalServiceExports
carrying PrivateLinkServiceResourceID) to a FrontDoorProfile,
producing an AFD OriginGroup with one Origin per qualifying
member-cluster export.
The Spec shape mirrors TrafficManagerBackend 1:1 so operators
authoring both surfaces have a single mental model:
* Spec.Profile → FrontDoorProfile ref (same namespace, immutable)
* Spec.Backend → ServiceImport ref (same namespace, immutable)
* Spec.Weight → aggregate 0..1000, distributed across per-export
weights identically to ATM
Status surfaces OriginGroupResourceID + per-origin details
(ResourceID, PrivateLinkServiceResourceID, effective Weight, source
FromCluster) plus a standard Accepted condition with
Accepted/Invalid/Conflict/Pending reasons. The Conflict reason is
reserved for the AFD/ATM coexistence guard landing in the next
commit.
Includes regenerated deepcopy methods and the CRD manifest emitted
by `make generate` + `make manifests`. Updates the net-crd-installer
hub-mode test fixture to expect the new CRD in config/crd/bases.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: a8fabc68-e113-49ef-b82d-d99a8502da11
The FrontDoorBackend reconciler (landing in the next commit) programs one OriginGroup per backend and one Origin per qualifying InternalServiceExport, both under Microsoft.Cdn/profiles/*. Extending the shared Clients bundle keeps the AFD client-construction seam single-sourced and mirrors how SecurityPolicies + WAFPolicies were added when the profile reconciler grew WAF enforcement. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: a8fabc68-e113-49ef-b82d-d99a8502da11
Introduces the POC FrontDoorBackend controller that programs one Azure
Front Door OriginGroup per FrontDoorBackend CR plus one Origin per
eligible InternalServiceExport. Origins are wired to member-cluster
PrivateLinkServices via SharedPrivateLinkResource; per-origin weights
follow the same ceil(backend.Weight * export.Weight / sum) formula as
TrafficManagerBackend, so operators moving between the ATM and AFD
surfaces get identical traffic distributions given identical inputs.
The reconciler:
* writes Accepted=True/Invalid/Pending conditions with
ObservedGeneration.
* registers a finalizer before touching Azure so a 403 cannot leave
the CR undeletable.
* only considers InternalServiceExports with
ExportMode=L7-FrontDoor AND a resolved PrivateLinkServiceResourceID.
* verifies parent FrontDoorProfile is Programmed=True at the current
generation before programming.
Secondary watches (Profile/ServiceImport/InternalServiceExport) and
the AFD/ATM coexistence guard (Conflict reason) are deferred to the
next commits; SetupWithManager is intentionally minimal here.
Also wires the reconciler into hub-afd-controller-manager and adds a
FrontDoorBackendFinalizer const alongside FrontDoorProfileFinalizer.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: a8fabc68-e113-49ef-b82d-d99a8502da11
…ble) Adds an unnumbered "Competitive context (industry parity)" subsection at the top of section 2 (Motivation), documenting the L7 edge + native WAF hook parity gap AKS/Fleet has today relative to GKE's Multi-Cluster Gateway + GCPBackendPolicy and EKS's AWS Load Balancer Controller + WAF annotations. Frames FrontDoorProfile.spec.wafPolicy and FrontDoorBackend as the AKS/Fleet parity closer, with the added SFI-NS253 Private-Link guarantee peer clouds do not provide. Kept as an unnumbered subsection to preserve stable section 2.1 / 2.2 / 2.3 cross-references from docs 002 and 003. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: a8fabc68-e113-49ef-b82d-d99a8502da11
Adds the coexistence check the FrontDoorBackend controller performs
before programming any AFD OriginGroup/Origin: if a
TrafficManagerBackend in the same namespace already claims the same
ServiceImport (Spec.Backend.Name), refuse to program and write
Accepted=False, Reason=Conflict. This prevents the L4 (public IP via
Traffic Manager) and L7 (Private Link via Azure Front Door) surfaces
from being programmed against the same backing service concurrently,
which SFI-NS253 forbids (see docs/first-party/001 section 3.5).
Migration path: a TMB that is being deleted (DeletionTimestamp set)
is not treated as a conflict, so the coexistence guard clears itself
the moment the user starts tearing the ATM twin down. The complementary
TMB watch that re-triggers same-namespace FrontDoorBackends when the
conflicting TMB disappears lands with the envtests in the next commit.
Also:
* setConflictAndUpdate helper (parallel to setInvalidAndUpdate) so
operators can distinguish "nonexistent parent" from "two surfaces
at once".
* RBAC marker to allow read of trafficmanagerbackends.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: a8fabc68-e113-49ef-b82d-d99a8502da11
…up providers
Adds integration coverage for the FrontDoorBackend reconciler and wires
the secondary watches its Pending/Conflict states rely on:
* test/common/azurefrontdoor/fakeprovider/origingroup.go
in-memory AFDOriginGroupsClient with a Has() peek helper so
specs can assert Azure-side state without touching ARM.
* test/common/azurefrontdoor/fakeprovider/origin.go
in-memory AFDOriginsClient with Count() + Get() peek helpers.
* pkg/controllers/hub/frontdoorbackend/suite_test.go
envtest bootstrap; registers both v1alpha1 (FrontDoorBackend +
FrontDoorProfile + InternalServiceExport) and v1beta1
(TrafficManagerBackend, needed by the coexistence guard).
* pkg/controllers/hub/frontdoorbackend/controller_integration_test.go
5 spec contexts:
- Happy path: 2 exports -> 1 OriginGroup + 2 Origins,
Accepted=True.
- Pending -> Programmed: backend converges once the parent
FrontDoorProfile flips Programmed=True (exercises the
new Profile watch).
- Conflict: TrafficManagerBackend claims the same
ServiceImport -> Accepted=False/Reason=Conflict, and
clears to Accepted=True after the TMB is deleted
(exercises the new TMB watch).
- Invalid: missing parent profile ->
Accepted=False/Reason=Invalid.
- Deletion: OriginGroup removed from the fake and CR is
finalized.
Also upgrades SetupWithManager to watch FrontDoorProfile,
TrafficManagerBackend, and InternalServiceExport (same-namespace
enqueue) so the Pending/Conflict specs converge on event-driven
wakeups instead of the 30s requeue.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: a8fabc68-e113-49ef-b82d-d99a8502da11
…eadcrumb Addendum 3 Adds a "Status update (2026-07-20 session, head 629644b)" block at the top of docs/first-party/001, 002, and 003 that lists the POC callouts now resolved by shipped code (WAF + ComplianceMode, FrontDoorBackend CRD + reconciler + coexistence guard + envtests, ExportMode + PLS lookup wiring, extended AFD client bundle, sibling binary + chart). Individual "POC:" / "POC deviation" / "Impossibility flag" blocks inline are intentionally left untouched — they remain accurate to commit cb02d14, and the phase framing in 002 s2 tables is preserved. Readers are directed to the summary block for the current state and to breadcrumb Addendum 3 for the per-commit narrative. Also appends Addendum 3 to the session breadcrumb documenting: * the 21-commit series that landed this session, * design decisions locked during landing (wire contract for ExportMode default, PLS resolution via annotation, backend UID naming, weight math parity with TMB, coexistence guard scope, Origin HostName placeholder, list-based enqueue), * what is intentionally out of scope (FrontDoorRoute, FrontDoorCustomDomain BYOC, POC-bridge removal, e2e), * verification results. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: a8fabc68-e113-49ef-b82d-d99a8502da11
… from PR Azure#373 Extracts an annotationValueTrue const to satisfy goconst on the member serviceexport reconciler, deletes an unused reason const, adds targeted //nolint:gosec pragmas for the CRD-validated int64->int32 weight conversion (G115) and the synthetic AFD DNS validation token used by the fake provider (G101), and re-runs gofmt on the serviceexport controller test to fix alignment in a new const group. Also fixes markdown-link-check errors by replacing an outdated AKS PLS learn.microsoft.com URL with the current internal-lb#create-a-private-link-service anchor and by ignoring eng.ms links in the link-check config (SSO-gated, 403 for anonymous fetchers but still valid Microsoft-internal references). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: a8fabc68-e113-49ef-b82d-d99a8502da11
rchincha
changed the title
Collaborator
|
Hi rchincha! Thanks for submitting the PR. The feature is surely very exciting. A concern I have right now is that, the size of the PR is a bit too large (11K+ lines) and it touches a wide area of the code base. At this moment it might be a bit difficult to review them all together -> if possible, would you mind splitting the PR a bit? We could start with the API definitions and then complete it one controller at a time, just to ensure that we do not miss anything during the flow. I am sure that when @zhiying-lin comes back she could help with this as well. |
5 tasks
rchincha
added a commit
to rchincha/fleet-networking
that referenced
this pull request
Jul 22, 2026
…(SFI-NS253) Part 01 of an 8-patch split of Azure#373. Docs-only patch; no code changes. Contents: - docs/first-party/001-afd-global-load-balancing.md — design proposal - docs/first-party/002-afd-implementation-plan.md — implementation plan - docs/first-party/003-pre-implementation-checklist.md — SFI-NS253 checklist - docs/first-party/README.md — first-party proposals index + competitive context - .github/workflows/markdown.links.config.json — ignore SSO-gated eng.ms links - .github/.copilot/breadcrumbs/** — design breadcrumbs and ATM baselines Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: a8fabc68-e113-49ef-b82d-d99a8502da11
rchincha
added a commit
to rchincha/fleet-networking
that referenced
this pull request
Jul 22, 2026
Part 02 of an 8-patch split of Azure#373. Dependency-only patch. Adds github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/cdn/armcdn v2.2.0 so subsequent patches can call the AFD control plane (profiles, origins, security policies, WAF policies, custom domains). No code in this patch imports the module yet. Verification: `go build ./...` and `go mod verify` both clean. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: a8fabc68-e113-49ef-b82d-d99a8502da11
rchincha
pushed a commit
to rchincha/fleet-networking
that referenced
this pull request
Jul 22, 2026
… InternalServiceExport ExportMode Part 03 of an 8-patch split of PR Azure#373. Independent of the other 7 patches. - Add v1alpha1 API types for the three new AFD-facing CRDs: * FrontDoorProfile * FrontDoorBackend * FrontDoorCustomDomain - Extend InternalServiceExport with ExportMode + PrivateLinkServiceResourceID so member-cluster exports can advertise an L7 export path plus a Private Link Service resource ID for AFD origin bindings. - Regenerate zz_generated.deepcopy.go and the four CRD YAMLs under config/crd/bases (three new + updated InternalServiceExport). No controller logic is registered against these APIs yet — the reconcilers land in PATCH 06/07/08. Verification: go build ./api/... ./cmd/net-crd-installer/... # clean go build ./... # clean Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: a8fabc68-e113-49ef-b82d-d99a8502da11
rchincha
added a commit
to rchincha/fleet-networking
that referenced
this pull request
Jul 22, 2026
Part 02 of an 8-patch split of PR Azure#373. Independent of the other 7 patches. - Add github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/cdn/armcdn/v2 v2.2.0 as a direct dependency; also pulls the armfrontdoor helper. - Refresh go.sum with the transitive checksums. The dependency is unused by application code in this patch — it is introduced ahead of the Azure Front Door client bundle (PATCH 04) so that each subsequent branch can compile against a stable module graph. Verification: go mod verify # "all modules verified" go build ./... # clean Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: a8fabc68-e113-49ef-b82d-d99a8502da11
rchincha
added a commit
to rchincha/fleet-networking
that referenced
this pull request
Jul 22, 2026
…(SFI-NS253) Part 01 of an 8-patch split of Azure#373. Docs-only patch; no code changes. Contents: - docs/first-party/001-afd-global-load-balancing.md — design proposal - docs/first-party/002-afd-implementation-plan.md — implementation plan - docs/first-party/003-pre-implementation-checklist.md — SFI-NS253 checklist - docs/first-party/README.md — first-party proposals index + competitive context - .github/workflows/markdown.links.config.json — ignore SSO-gated eng.ms links - .github/.copilot/breadcrumbs/** — design breadcrumbs and ATM baselines Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: a8fabc68-e113-49ef-b82d-d99a8502da11
rchincha
added a commit
to rchincha/fleet-networking
that referenced
this pull request
Jul 22, 2026
Part 02 of an 8-patch split of Azure#373. Dependency-only patch. Adds github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/cdn/armcdn v2.2.0 so subsequent patches can call the AFD control plane (profiles, origins, security policies, WAF policies, custom domains). No code in this patch imports the module yet. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: a8fabc68-e113-49ef-b82d-d99a8502da11
This was referenced Jul 22, 2026
Author
Split into a 8-patch series. Pls see the new PR links attached to this PR. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What type of PR is this?
feature
What this PR does / why we need it:
This PR closes that gap by adding an AFD-first-party controller family on the hub that reconciles Premium AFD + WAF + Private Link origins from CRs — while preserving strict KEP-1645 mcs-api parity on the user-facing ServiceExport / MultiClusterService shapes and satisfying SFI-NS253 (no public IPs on member workloads).
Which issue(s) this PR fixes:
Fixes #
Requirements:
make reviewablefor basic local testHow has this code been tested
Special notes for your reviewer
Problem statement
Shielding member clusters with a programmable Layer-7 cloud-native edge (Gateway API) paired with internal Zero-Trust micro-segmentation is the gold standard for modern multi-cluster architecture — and today it is exactly the gap in AKS Fleet:
GCPBackendPolicy)This PR closes that gap by adding an AFD-first-party controller family on the hub that reconciles Premium AFD + WAF + Private Link origins from CRs — while preserving strict KEP-1645 mcs-api parity on the user-facing
ServiceExport/MultiClusterServiceshapes and satisfying SFI-NS253 (no public IPs on member workloads).Solution outline
No breaking changes to user-facing types.
ServiceExportandMultiClusterServicekeep their upstream KEP-1645 shapes verbatim — Fleet-specific behaviour is carried via annotations, not new Spec fields.Member cluster (opt-in per Service)
Users add annotations to a Service they already
ServiceExport:networking.fleet.azure.com/export-mode: L7-FrontDoorL4-TrafficManagerpreserves current behaviour.service.beta.kubernetes.io/azure-load-balancer-internal: "true"service.beta.kubernetes.io/azure-pls-create: "true"service.beta.kubernetes.io/azure-pls-name: <name>The member
serviceexportcontroller inspects the Service, resolves the PLS ARM ID, and stamps it onto the internal projection.Hub cluster (operator-managed)
Three new CRs, reconciled by a new sibling
hub-afd-controller-managerbinary/chart:FrontDoorProfileWAFPolicyRef+ComplianceModeby attaching aSecurityPolicy; refuses to acceptStandard_AzureFrontDoor.FrontDoorBackendInternalServiceExport(Programmed=True, ExportMode=L7, PLS ID present), programs one AFDOriginGroupwith oneOriginper member; guards against AFD/ATM double-export with aConflictreason.FrontDoorCustomDomainWire-in points
InternalServiceExportSpecgainsExportMode+PrivateLinkServiceResourceID(internal hub↔member projection only — user-facingServiceExportunchanged).FrontDoorProfileSpeclocked toPremium_AzureFrontDoor(Managed Bot rules + Private Link require Premium).net-crd-installer.Files by area
Design docs & competitive context (
docs/first-party/)README.md,001-afd-global-load-balancing.md,002-afd-implementation-plan.md,003-pre-implementation-checklist.mdAPI / CRDs (
api/v1alpha1/,config/crd/bases/)frontdoorprofile_types.go,frontdoorbackend_types.go,frontdoorcustomdomain_types.gointernalserviceexport_types.go(+ExportMode, +PrivateLinkServiceResourceID),zz_generated.deepcopy.goHub controllers (
pkg/controllers/hub/,cmd/hub-afd-controller-manager/)frontdoorprofile/(withwaf.go),frontdoorbackend/,frontdoorcustomdomain/cmd/hub-afd-controller-manager/main.go(feature-gated split fromhub-net-controller-manager)Member controller (
pkg/controllers/member/serviceexport/)controller.go: addssetAzureRelatedPrivateLinkInformation+ ExportMode dispatchannotationValueTrueCommon libs (
pkg/common/)azurefrontdoor/client.go: SDK bundle addsOriginGroups,Origins,WAFPolicies,SecurityPolicies,AFDCustomDomainsclients (armcdnv1.1.1 → v2.2.0)objectmeta/:ServiceExportAnnotationExportMode+ExtractExportModeFromServiceExportPackaging (
charts/,docker/,Makefile)charts/hub-afd-controller-manager/(Deployment + RBAC + PDB)docker/hub-afd-controller-manager.DockerfileTests (
pkg/controllers/hub/*/,test/common/azurefrontdoor/fakeprovider/)frontdoorprofile,frontdoorbackend,frontdoorcustomdomain,serviceexport(integration)profile.go,endpoint.go,origin.go,origingroup.go,securitypolicy.go,wafpolicy.go,customdomain.goCI / housekeeping
.github/workflows/markdown.links.config.json: ignore SSO-gatedeng.mslinks.github/.copilot/breadcrumbs/