feat(bootstrap): dedicated AWS state backend + scoped IAM role via terraform-aws-template

[JacobPEvans-personal]

Summary

terragrunt plan against this stack was failing with S3 403 because state was pointing at the terraform-proxmox-state-useast2-<account> bucket (cross-stack) and the day-to-day terraform IAM user lacks s3:ListBucket on that bucket. Rather than widen the existing user's S3 grants across stacks, follow the canonical dryvist/terraform-aws-template pattern: one bucket per stack, one scoped IAM role per stack, operator assumes the role with MFA via aws-vault.

What lands

• bootstrap/ — thin module call to dryvist/terraform-aws-template pinned to commit c85894b (the resolved SHA for v0.1.0, satisfying Checkov CKV_TF_1 which forbids tag-pinned module sources). project = "github" yields:

  • S3 bucket tfstate-github-<account> (AES256 SSE-S3, versioning, public-access-block, TLS-only bucket policy, 90-day noncurrent-version expiry, S3 native locking via use_lockfile = true — no DynamoDB).
  • IAM role tf-github with combined trust: GitHub Actions OIDC (repo:<github_org>/<github_repo> on push to main and on pull_request) plus MFA-gated AssumeRole from the operator IAM user.
  • IAM role policy scoped to that one bucket only.
  • Account-wide GitHub Actions OIDC provider (singleton; if it already exists from another stack, tofu import once per the README and re-apply).

  github_org, github_repo, and operator_user_arns are required vars with no defaults — operator supplies via gitignored terraform.tfvars. Source tree stays identity-free.

• bootstrap/README.md — full operator walkthrough: prerequisites, apply, state migration into the new bucket at _bootstrap/terraform.tfstate, ~/.aws/config profile addition shape, verification, the OIDC-already-exists import path, and the don't-touch rule for ongoing changes.

• Root terragrunt.hcl — switched bucket from terraform-proxmox-state-useast2-<acct> to tfstate-github-<acct>, key from terraform-github/terraform.tfstate to github/terraform.tfstate (matches the template's state_key_prefix formula), dropped dynamodb_table. No state to migrate — no apply has ever run against this stack.

• Root versions.tf — bumped required_version to >= 1.10 for use_lockfile support.

• AGENTS.md — new "State backend" section: bucket-per-stack rationale, the operator identity flow (operator IAM user → MFA AssumeRole → STS → terragrunt), the aws-vault profile naming convention (profile name == role name == tf-<project>), future CI via OIDC, and the rule "never run this stack with elevated bootstrap creds — only with tf-github STS".

• Root README.md — replaces the manual tofu init -backend-config=... snippet with aws-vault exec tf-github -- terragrunt … and points to bootstrap/README.md for first-time setup.

• .pre-commit-config.yaml — terraform_validate gains --hook-config=--retry-once-with-cleanup=true, the documented antonbabenko/pre-commit-terraform escape hatch for module sources pinned to commit SHAs. Without it the hook fails on the first call after any module-SHA bump (prior .terraform/modules/ install doesn't match the new SHA, validate fails, hook auto-inits but doesn't retry validate). With it the hook auto-cleans and re-inits before retrying validate, then passes cleanly.

What this does NOT do

• No tofu apply from this PR. The operator runs the one-time bootstrap apply with the elevated iam-user creds after merge (per bootstrap/README.md), then switches to tf-github via aws-vault for everything else.
• No .github/workflows/terragrunt.yml. The role IS OIDC-ready for repo:<github_org>/<github_repo> on push to main and on pull_request, but CI wiring is a future PR.
• Does not touch terraform-proxmox or terraform-aws state. They keep their existing shared bucket. Only this stack carves out its own.

Cost impact

Free. S3 for a < 1 KB state file plus a few noncurrent versions sits well under any free-tier ceiling. AES256 SSE-S3 is free (no KMS per-key or per-API-call charges). S3 native locking is free (no DynamoDB provisioned). IAM role + inline policies + OIDC provider are free. Lifecycle expiry, public-access block, TLS-only bucket policy: all free.

Verification

• tofu init -backend=false && tofu validate in both root and bootstrap/ — green
• pre-commit run --all-files from a cold .terraform/ — green (the --retry-once-with-cleanup config makes terraform_validate recover cleanly from the first-call module-SHA mismatch)
• Checkov CKV_TF_1 satisfied (module source pinned to commit SHA, not tag name)
• Commit GPG-signed
• No identities in source tree (verified by grep; all identity values are required vars supplied via gitignored tfvars)
• Operator runs the bootstrap apply with elevated creds (one-time)
• Operator adds the tf-github profile to ~/.aws/config and verifies aws-vault exec tf-github -- aws sts get-caller-identity returns the assumed-role identity
• Operator runs aws-vault exec tf-github -- terragrunt plan and reviews the full plan (the original goal — should show 2 imports + 1 add for the rulesets in PR feat(rulesets): reverse-engineer org branch protection + add review gate, conventional commits #4)