feat: make PQC validation fail closed and AWS-tested

[ethanolivertroy]

Summary

This PR turns QuantumForge's AWS post-quantum cryptography (PQC) path from a syntax-validated proof of concept into a credential-free tested and live-validated reference implementation. Missing, malformed, or contradictory inputs now stop the assessment instead of producing an empty or compliant result.

1. AWS provider compatibility

• Requires AWS Provider ~> 6.2, which contains ML-DSA support for AWS Key Management Service (KMS).
• Commits root, module, and live-test provider lockfiles.
• Removes the unused Azure provider requirement.

2. Terminology and standards

• Renames modules/hybrid-pqc-kms to modules/pqc-kms-signing; ML-DSA is pure post-quantum signing, not a classical/PQC hybrid signature.
• Preserves Terraform state migration with a moved block.
• Renames the root enable flag and keeps both deployment paths opt-in.
• Uses FIPS 204 for ML-DSA and removes the incorrect SP 800-208 association.
• Makes the listener module explicitly Application Load Balancer (ALB)-only (HTTPS). Network Load Balancer (NLB) TLS support remains a separate implementation.
• Uses exact versioned AWS PQ-TLS policy allowlists.

3. Credential-free Terraform tests

Adds native mock_provider "aws" tests for:

• root opt-in/default behavior;
• ML-DSA key configuration, outputs, deletion windows, and IAM principal validation;
• ALB HTTPS configuration, exact approved policies, malformed ARNs, and NLB rejection.

Pull-request tests clear AWS credential variables and disable EC2 metadata access.

4. Fail-closed CI

• Replaces the Terraform-plan inventory collector that could suppress plan failures and emit a false-clean empty inventory.
• Gives PR CI only contents: read; checkout does not persist the GitHub token and no PR-executed job receives OpenID Connect (OIDC) authority.
• Pins all actions by verified commit, scanners by container digest, Conftest by release checksum, and the schema validator wheel by SHA-256.
• Requires Terraform, Open Policy Agent (OPA), Conftest, Checkov, Trivy infrastructure-as-code and secret scans, schema, scoring, Cryptographic Bill of Materials (CBOM), and evidence checks.
• Distinguishes assessment_complete, no_assets_found, collection_failed, and not_assessed.
• Tests missing and malformed plans, incomplete evidence, inconsistent counts/risk IDs, and private environment identifiers.

5. Real AWS ML-DSA lifecycle

Adds an explicitly authorized, account-guarded temporary KMS test deployment that:

1. creates an ML_DSA_65 key;
2. signs and verifies with AWS KMS;
3. retrieves the public key;
4. independently verifies the signature with OpenSSL 3.5.7;
5. destroys Terraform resources and requires PendingDeletion in the expected seven-day window.

6. Real ALB PQ-TLS runtime validation

Adds a temporary ALB test deployment and capable OpenSSL client that proves:

• TLS 1.3 hybrid negotiation uses X25519MLKEM768;
• a classical client still negotiates X25519;
• the listener uses ELBSecurityPolicy-TLS13-1-2-Res-PQ-2025-09;
• Terraform, AWS Certificate Manager (ACM), ALB, target-group, security-group, gateway, and virtual private cloud (VPC) cleanup completes.

Live tests are manual, main-only jobs inside protected GitHub environments, which restrict approval, branch access, and variables. They use job-scoped GitHub OIDC instead of stored AWS access keys. Cleanup is layered: process traps, a post-test job that runs even after failure, and an hourly cleanup workflow that removes expired tagged test resources if a runner is canceled or crashes. The workflow filename and environment retain the legacy internal name “janitor.”

7. Scoring, governance, and evidence

• Keeps inherent risk, migration urgency, remediation effort, and evidence confidence independent.
• Makes risk enrichment schema-compatible with the canonical inventory and rejects incomplete scoring.
• Requires exception owners, approvers, rationale, controls, RFC3339 creation/expiration, temporal consistency, and unique IDs; enforcement uses the actual time when OPA executes.
• Creates provenance attestations only in protected jobs running from the main branch, before optional publication.
• Adds a fail-closed S3 Object Lock publisher and role boundary requiring Compliance mode, at least 2,555 remaining retention days, post-upload retention, version, and checksum verification, and explicit deletion/Object Lock administration denies.
• Excludes account IDs, ARNs, endpoints, IP addresses, credentials, connection strings, private keys, Terraform state, and raw provider logs from live bundles.

8. Discovery before more deployment modules

• Adds a vendor-neutral inventory schema for certificates, protocols, libraries, applications, data flows, and cryptographic resources across cloud, SaaS, on-prem, and application sources.
• Expands Terraform discovery to KMS, ELB listeners, ACM, ACM Private CA, CloudFront, API Gateway domains, and Site-to-Site VPN.
• Preserves unknown as a first-class assessment outcome.
• Adds a non-empty CycloneDX 1.5 Cryptographic Bill of Materials derived from enforced ML-DSA and PQ-TLS allowlists.
• Prioritizes inventory data collectors and evidence-backed measurement before additional deployment modules.

Verification

Credential-free and static

• terraform fmt -check -recursive
• Terraform native mocks: root 2/2, KMS 6/6, ALB 5/5
• OPA 1.18.2: 42/42 passed
• Conftest: generated hybrid plan passed; generated classical-only plan was denied with 2 expected failures; malformed plan member denied
• check-jsonschema 0.37.4: canonical and scoring-enriched inventories passed
• CBOM: 10 components
• Checkov 3.3.8: Terraform 0 failures / 0 parse errors; GitHub Actions 0 failures / 0 parse errors
• Trivy 0.69.2: 0 HIGH/CRITICAL IaC failures; 0 secret findings
• AWS Identity and Access Management (IAM) Access Analyzer: 0 identity-policy errors; 0 trust-policy errors after placeholder substitution
• ShellCheck, Ruff, Python compilation, Actionlint, JSON parsing, immutable action-reference verification, evidence checksum/privacy tests: passed

Live AWS sandbox

Executed locally in an isolated us-east-1 sandbox on 2026-07-20:

• KMS ML_DSA_65 sign/verify: passed
• Independent OpenSSL verification: passed
• KMS cleanup: alias absent and key in PendingDeletion
• ALB hybrid group: X25519MLKEM768
• ALB classical fallback: X25519
• Final KMS and ALB evidence bundles passed semantic validation, archive verification, SHA-256 verification, and private-identifier checks
• No final billable ALB/network/ACM resources remained
• The cleanup safety net passed both targeted-run and expired-resource smoke tests by removing tagged VPC/security-group test resources with zero residuals

Raw sandbox evidence is intentionally not committed because the private source material contains ephemeral resource identifiers. The sanitized bundles contain only the allowlisted runtime assertions.

Upstream Actions status

At PR creation, GitHub reported all three workflows already present on the upstream default branch as disabled_manually. No check suite or status context was created for this PR. A maintainer must enable GitHub Actions before hosted CI can run; the verification above is from the exact pushed commit.

Migration and operator notes

• Root variable rename: enable_hybrid_pqc_kms → enable_pqc_kms_signing.
• Terraform state migration is encoded from module.hybrid_pqc_kms[0] to module.pqc_kms_signing[0].
• Both deployment modules default to disabled.
• Upstream operators must configure the protected live-test environment quantumforge-aws-sandbox and the protected expired-resource cleanup environment quantumforge-aws-sandbox-janitor, plus exact OIDC trust subjects and repository variables QUANTUMFORGE_AWS_ROLE_ARN / QUANTUMFORGE_AWS_ACCOUNT_ID.
• Optional immutable publication additionally requires QUANTUMFORGE_EVIDENCE_BUCKET and the rendered IAM/Object Lock templates under docs/aws/.
• The ALB integration test is expected to cost about $0.05 for a successful one-hour-minimum run in us-east-1; the hourly expired-resource cleanup workflow bounds ordinary orphan duration, but operators should keep an account budget alert.

[ethanolivertroy]

Independent Cursor review

I ran a read-only implementation review of upstream/main...HEAD using the Cursor personal agent at commit 6410686, followed by a terminology and workflow-reference review of the documentation update now included in 1bc987a.

Verdict: PASS. No material release blockers found.

The review independently confirmed:

• AWS Provider 6.x, ML-DSA, and FIPS 204 terminology are aligned.
• Pull-request checks require no AWS credentials and block missing, malformed, or contradictory results instead of treating them as compliant.
• Every Terraform resource_changes member must include a non-empty string change.actions array; the earlier missing-actions concern is resolved in both the Rego policy and Terraform-plan inventory collector.
• Live KMS and ALB tests verify the expected AWS account before changing anything and use shell exit handlers, post-run verification, expiration tags, and cancellation recovery.
• OpenID Connect (OIDC), AWS Identity and Access Management (IAM), and S3 Object Lock controls are narrowly scoped and enforce retention and checksum verification.
• Inventory, scoring, governance, and evidence models remain consistent.
• Evidence privacy controls prevent account IDs, ARNs, endpoints, IP addresses, credentials, private keys, Terraform state, and raw provider logs from entering published bundles.

The follow-up review also confirmed that the public documentation now defines the repository's legacy shorthand, and that renamed workflow display names, job IDs, artifact names, and AWS role-session names do not break in-repository dependencies or change the credential boundary.

What the “janitor” is

“Janitor” is shorthand for the cleanup safety net implemented by .github/workflows/aws-live-pqc-janitor.yml and scripts/aws/cleanup-expired-live-resources.sh.

Normal cleanup happens inside each live test and again in a post-test cleanup job that runs even after failure. If a runner is canceled, crashes, or disappears before either path finishes, the hourly cleanup safety net runs from a separate protected GitHub environment. It only targets resources tagged as QuantumForge integration-test resources whose expires-at timestamp has passed. It deletes those resources in dependency order, schedules any test KMS keys for AWS-managed deletion, and fails if billable test resources remain.

It is not a general AWS cleanup process and does not target production or unrelated resources.

Non-blocking suggestions

• Generate README validation results from workflow artifacts to prevent status drift.
• Optionally split live-test, cleanup, and evidence-publication permissions across separate AWS roles.
• Add a negative test for malformed expires-at tags.

The Cursor run was review-only. It made no file changes, commits, AWS calls, or external mutations.

[IAwiz87]

Reviewed end-to-end with direct test execution, not just trusting the PR description — all claims verified: terraform fmt clean, opa test 42/42, terraform test 2/2 + 6/6 + 5/5, CycloneDX CBOM (10 components), Conftest hybrid-pass (8/8) and classical-fail (exactly 2 expected failures), fail-closed census behavior on empty/malformed plans, and the AWS provider 6.1.0→6.2.0 justification cross-checked against HashiCorp's own withdrawal notice (hashicorp/terraform-provider-aws#43213).

Confirmed improvements: fixes a real fail-open bug in the old census workflow (silently reported "compliant" on plan failure); meaningfully tightens OIDC/id-token: write exposure so PR-triggered runs can no longer assume the AWS role; least-privilege IAM boundary with tag-gated mutations and an explicit anti-tamper Deny; a properly account/tag-scoped sandbox cleanup script; a real exception lifecycle wired into the policy deny logic; and a risk-scoring change (impact as a co-primary dimension) that lines up with the roadmap. Nice work.

Non-blocking items for a follow-up, not this PR (see inline comments):

1. enable_pqc_kms_signing's default flips true→false — confirm intentional.
2. Tag-key list asymmetry between KMS key creation and ACM certificate creation in the IAM boundary.
3. try_delete() in the cleanup script swallows all errors silently — fine given the end-of-script verification, but worth a debug log line eventually.
4. ROADMAP.md is substantially rewritten rather than incrementally updated — worth a maintainer pass to confirm the new framing is the one to keep.
5. Two prior capabilities quietly disappear rather than carrying forward: PR-comment posting from the old census workflow, and the previously-disabled CMVP status-change issue-creation stub. Confirm you're fine losing both.

None of the above block merge — approving.

[IAwiz87]

Default flips from true to false, not just a rename from enable_hybrid_pqc_kms. Confirm this behavior change (KMS signing key no longer provisioned by default) is intentional.

[IAwiz87]

ACM certificate creation's required tag-key list omits owner (and the quantumforge:* keys) that the KMS key creation list requires above. Confirm this asymmetry is intentional.

[IAwiz87]

try_delete() swallows all command errors, not just "already deleted" ones. The end-of-script remaining-resource check catches aggregate failures, but you lose the specific error reason if something unexpected breaks here.

[IAwiz87]

This replaces the roadmap doc wholesale rather than updating it incrementally — worth a maintainer pass to confirm the new prioritization (discovery → governance → gated platform modules) is the direction you want, versus merging with the original framing.