Provision the minimum HTTPS-origin scaffold runtime on ECS and RDS

[alexeygrigorev]

Parent epic: #9

Normative spec: 08 — Runtime topology
Foundation dependency and reviewed expansion: #67

Runtime integration correction

The #67 foundation used an HTTP custom origin from CloudFront to the ALB. An ALB receiving that request sets X-Forwarded-Proto: http; deployed Django correctly trusts only X-Forwarded-Proto: https while SECURE_SSL_REDIRECT=True. The result would be a redirect/readiness loop even though the viewer connected to CloudFront over HTTPS.

Selected safe solution A is end-to-end edge/origin TLS. This issue owns the reviewed expansion of #67's network/DNS count:

• a workload-only origin.web.dtcdev.click hostname;
• an ALB origin certificate in eu-west-1, with DNS validation in exact zone Z05963572WVWFHDQZH5NE;
• origin.web.dtcdev.click A/AAAA aliases to the ALB;
• a TLS 1.2 HTTPS ALB listener;
• CloudFront connecting to origin.web.dtcdev.click with https-only.

web.dtcdev.click remains the only public/canonical website hostname. The origin hostname is operational plumbing, is always non-indexable, is not accepted as a public application canonical, and never appears in a sitemap. Django's proxy trust and SECURE_SSL_REDIRECT remain unchanged; this issue fixes the transport rather than trusting a spoofable header or disabling redirects.

Scope

Extend the DataTalksClub/aws-infra website stack with the minimum production-shaped runtime required by the #1 scaffold and the HTTPS-origin correction above.

HTTPS origin and DNS

• Add an explicit environment-owned origin_hostname input; sandbox supplies origin.web.dtcdev.click.
• Create the origin ACM certificate with the primary eu-west-1 provider, validate it through records in exact hosted zone Z05963572WVWFHDQZH5NE, and add only the origin A/AAAA aliases to the workload ALB.
• Change CloudFront's custom origin domain to origin.web.dtcdev.click, require https-only, and require TLS 1.2 to the ALB.
• Restrict ALB ingress to the inventoried AWS-managed CloudFront origin-facing prefix list on HTTPS only.
• Use an HTTPS listener with the origin certificate. Its default action denies access; forwarding requires both the expected public host web.dtcdev.click and the generated secret origin-verification header supplied only by this CloudFront distribution.
• Do not export, log, or commit the origin-verification value. A request lacking either the CloudFront-network control or the secret/host rule must not reach Django.
• Keep the public web.dtcdev.click A/AAAA aliases pointing to CloudFront and keep all emitted application canonicals on https://datatalks.club/... as defined by Bootstrap the runnable Django/uv foundation and engineering workflow #1.
• Treat origin.web.dtcdev.click as operational/noindex only: it is absent from sitemaps/navigation/canonicals and is not added as a normal Django allowed public host.

Minimum scaffold runtime

• Create an ECR repository with scan-on-push and a bounded lifecycle policy.
• Create an ECS Fargate cluster and task definitions/services for separate Django web and Django-Q2 worker processes using the same immutable image digest.
• Create a separate one-off migration task definition; web and worker entrypoints never apply migrations.
• Run exactly one sandbox worker service instance so the scaffold has one effective Django-Q2 scheduler owner until later job primitives are implemented.
• Register only the web service with the existing target group and preserve /health/live for ALB liveness plus /health/ready for dependency-aware rollout/readiness verification.
• Provision a small encrypted, non-public PostgreSQL RDS instance in the isolated subnets with automated backups, storage autoscaling, and explicit final-snapshot/deletion behavior.
• Create Secrets Manager containers and least-privilege task execution/application roles for current bootstrap values; values are injected out of band.
• Create CloudWatch log groups with bounded retention/encryption and the minimum ECS/ALB/RDS service-health alarms required for the scaffold.
• Expose production-account inputs for private tasks, Multi-AZ, stronger deletion protection, longer backup retention, scaling, sizing, public hostname, and origin hostname without sandbox coupling.

Non-goals

• Do not disable SECURE_SSL_REDIRECT, weaken SECURE_PROXY_SSL_HEADER, trust arbitrary viewer-supplied forwarding headers, add an application exception that treats external HTTP as HTTPS, or add origin.web.dtcdev.click as a public canonical.
• Do not expose the ALB as an alternate website, forward based on origin hostname alone, place the origin hostname in navigation/sitemaps, or permit direct origin access without both network and secret/host controls.
• No shared/wildcard certificate or DNS ownership beyond the workload's explicit web.dtcdev.click, origin.web.dtcdev.click, and their certificate-validation records. Never discover a hosted zone by name or create a zone.
• No SES identity/configuration set/provider events, recipient safeguards, email/outbox alarms, GitHub content assets, S3 content/export buckets, content freshness alarms, advanced tracing, domain dashboards, or restore drill. Those belong to Build the durable Relay delivery intent and status lifecycle #49, Add observability, service targets, backup/restore, and failure runbooks #66, and domain issues.
• No application deployment workflow; Deploy scaffold images through GitHub OIDC and immutable ECS releases #69 owns it.
• No secret values in Terraform/Git, migration race in web/worker startup, wildcard IAM, public RDS/task ports, broad egress ownership, or production-sized sandbox defaults.

Acceptance criteria

• Terraform explicitly creates/validates an origin.web.dtcdev.click certificate in eu-west-1 and its A/AAAA ALB aliases in hosted zone Z05963572WVWFHDQZH5NE; it creates or discovers no hosted zone and touches no unrelated DNS.
• The reviewed plan identifies the expansion from Create the portable sandbox network, DNS, edge, and TLS foundation #67's three DNS records: only the origin certificate-validation record and origin A/AAAA aliases are added, subject to ACM's exact generated validation record count.
• CloudFront uses origin.web.dtcdev.click with https-only and TLS 1.2; the ALB accepts CloudFront-origin HTTPS, presents the matching origin certificate, and no longer relies on an HTTP CloudFront→ALB hop.
• The ALB security group allows the explicit CloudFront managed prefix list on HTTPS only. The listener defaults to denial and forwards only when both public host web.dtcdev.click and the generated secret origin header match.
• Source/plan guarantee: Terraform requires the CloudFront-origin network source, generated secret header, and expected public host before forwarding, so a request missing any control cannot reach Django; the secret is sensitive state only and is absent from outputs, logs, committed plans, task configuration, and issue/test artifacts. Deploy and validate the scaffold at web.dtcdev.click #70 must exercise every direct-origin denial combination live before deployment acceptance.
• Source/settings guarantee: CloudFront uses HTTPS to the ALB and the ALB supplies forwarded HTTPS state to Django; deployment settings retain SECURE_SSL_REDIRECT=True and the exact existing SECURE_PROXY_SSL_HEADER, with only exact /health/ready exempted for the private direct target probe. Regression tests prove terminal forwarded-HTTPS responses plus preserved non-health redirects and host rejection; Deploy and validate the scaffold at web.dtcdev.click #70 must verify public /, /health/live, and /health/ready have no live redirect loop.
• Source/configuration guarantee: web.dtcdev.click remains the public and allowed application endpoint, production-equivalent pages retain https://datatalks.club/... canonicals, and origin.web.dtcdev.click is operational/noindex only and absent from sitemap, navigation, canonical, and normal Django host configuration. Deploy and validate the scaffold at web.dtcdev.click #70 must probe and crawl the deployed public and origin paths live before deployment acceptance.
• ECR uses scan-on-push and lifecycle rules; task definitions accept one immutable image digest for both web and worker.
• Web and worker are separate ECS services, and migrations run only through a separate one-off task before service rollout.
• Source/plan guarantee: sandbox worker desired count is one and its dedicated 0/100 deployment configuration plus singleton precondition prevents overlap above desired count; scheduler/lease enhancements remain deferred to Add shared service, configuration, operation, and durable-job primitives #31. Deploy and validate the scaffold at web.dtcdev.click #70 must verify one stable running worker and exercise replacement while proving running plus pending worker tasks never exceed one.
• Source/test/plan guarantee: container liveness uses /health/live; the ALB sends direct HTTP to exact /health/ready and accepts only 200; regression tests return dependency-aware 200 ready or sanitized 503 not_ready without redirect. Deploy and validate the scaffold at web.dtcdev.click #70 must exercise readiness live before and after migrations and during database failure, proving failed readiness blocks a bad revision.
• RDS PostgreSQL is encrypted, non-public, task-SG-only, backed up, storage-autoscaling, and has explicit snapshot/deletion settings.
• Secrets Manager containers and task roles are least privilege; no secret values, state, saved plans, backend credentials, or real tfvars enter Git.
• CloudWatch logs have explicit retention/encryption and basic alarms cover web/worker desired-vs-running state, ALB target health/5xx, and RDS availability/storage.
• Production variables enable a distinct origin hostname/certificate, private tasks, Multi-AZ, stronger protection/retention, and scaling without sandbox state/resource dependencies.
• Terraform format/validate/static-policy checks and reviewed sandbox/production-fixture plans pass with no unrelated shared-resource mutation. Policies cover HTTPS origin, explicit DNS ownership, direct-origin denial, public RDS/task ingress, encryption, wildcard IAM, and committed secrets.

Test scenarios

1. Plan the expanded sandbox root and enumerate every DNS/certificate/listener/origin change. Confirm exact delegated zone use; no zone discovery/creation; existing public records remain CloudFront aliases; new origin aliases target only the ALB.
2. Validate the origin certificate is in eu-west-1, CloudFront viewer certificate remains in us-east-1, CloudFront origin policy is https-only/TLS 1.2, and the ALB listener uses the matching origin certificate.
3. Through https://web.dtcdev.click, request /, /health/live, and /health/ready; verify a terminal expected response with no redirect chain/loop and confirm Django observes forwarded HTTPS without any proxy-setting change.
4. Exercise origin denial without exposing the secret: ordinary internet→origin is blocked by the SG; a permitted-network test without the secret is denied; correct-header/wrong-host and correct-host/missing-header are denied; only the configured CloudFront origin path forwards.
5. Request or crawl the operational origin hostname and prove it cannot serve a separately indexable/canonical site: direct access is denied, no sitemap/navigation links reference it, and no application canonical points to it.
6. Confirm the generated origin secret is marked sensitive and absent from Terraform outputs, application environment, logs, committed artifacts, and test reports.
7. Inspect IAM and plan ownership for exact ECR/ECS/RDS/Secrets/Logs resources and reject wildcard/unrelated permissions.
8. Run migration successfully, then deploy web and worker from one digest; make migration fail and verify services do not roll forward.
9. Verify web/worker processes are non-root, separate, receive only required secrets, and exactly one worker is desired.
10. Confirm target liveness and public dependency-aware readiness before/after migrations and with database unavailable; no case creates a redirect loop.
11. Stop/replace each task and verify ECS recovery plus basic alarms. Verify encrypted automated backups and final-snapshot/deletion configuration; full restore/RPO/RTO remains in Add observability, service targets, backup/restore, and failure runbooks #66.
12. Plan the production fixture with a different public/origin domain, account, region, hosted zone, network/private-task/NAT inputs, and backend without sandbox references.

No product UI is implemented here. Deployed browser/header/redirect verification is completed by #70; #68 must still provide infrastructure-level HTTPS-origin and denial evidence before handoff.

Dependencies

• Depends only on scaffold Bootstrap the runnable Django/uv foundation and engineering workflow #1 and accepted network/edge foundation Create the portable sandbox network, DNS, edge, and TLS foundation #67.
• This issue intentionally expands Create the portable sandbox network, DNS, edge, and TLS foundation #67's workload-owned DNS/TLS resources to correct runtime protocol integration; it does not reopen shared-zone ownership or weaken the accepted network boundary.
• Blocks deployment pipeline Deploy scaffold images through GitHub OIDC and immutable ECS releases #69 and scaffold deployment Deploy and validate the scaffold at web.dtcdev.click #70.
• Add shared service, configuration, operation, and durable-job primitives #31, Build the durable Relay delivery intent and status lifecycle #49, and Add observability, service targets, backup/restore, and failure runbooks #66 enhance jobs, email, alarms, and recovery after the first scaffold is online; they do not block this issue.

[alexeygrigorev]

Re-groomed for first-scaffold deployment

The runtime is reduced to what the current scaffold needs: ECR, separate immutable web/worker tasks, one-off migrations, private PostgreSQL, bootstrap secret containers, logs, automated backups, and basic service-health alarms. SES, S3 content/export assets, outbox/content alarms, advanced operations, and restore drills are deferred to #49/#66 and domain infrastructure. #68 now depends only on #1 and #67.

[alexeygrigorev]

Grooming amendment — HTTPS origin integration

Runtime integration review found that #67's HTTP CloudFront→ALB hop would make the ALB set X-Forwarded-Proto: http. The accepted Django scaffold correctly enables SECURE_SSL_REDIRECT and trusts only forwarded https, so that design would redirect instead of producing a stable readiness/public response.

Selected solution A is now normative in #68: create workload-only origin.web.dtcdev.click, validate an eu-west-1 ACM certificate in exact zone Z05963572WVWFHDQZH5NE, alias that origin hostname to the ALB, terminate HTTPS at an ALB TLS listener, and configure CloudFront https-only to that hostname. The CloudFront prefix-list SG restriction and generated secret-header plus expected-public-host listener rule both remain mandatory.

This is a reviewed expansion from #67's original three DNS records. #68 may add only the origin certificate-validation record plus origin A/AAAA aliases. web.dtcdev.click remains the public endpoint; the origin is operational/noindex, is absent from sitemap/navigation/canonicals, and must not be added as a normal public Django host. No Django proxy-trust or SSL-redirect weakening is permitted.

[alexeygrigorev]

Software engineer report

Implemented the uncommitted DataTalksClub/aws-infra runtime extension on branch issue-68-website-runtime. No apply, import, state write, secret-value write/read, saved plan, real tfvars, commit, or push was performed.

Implementation

• Added the approved end-to-end TLS correction: regional origin ACM certificate for origin.web.dtcdev.click (with the forwarded public host SAN), exact-zone DNS validation, origin A/AAAA aliases to a dual-stack ALB, TLS 1.2 HTTPS listener, and CloudFront https-only origin transport.
• Preserved direct-origin denial: CloudFront prefix-list network ingress, deny-by-default listener, and an AND rule for public Host: web.dtcdev.click plus the generated secret header. The secret is sensitive state only and absent from outputs, task configuration, logs, source examples, and this report.
• Split the weight-55 IPv4 and IPv6 CloudFront managed-prefix-list rules across two SGs attached to the ALB. The refreshed account SG rule quota is 60, so combining both weights (110) would fail apply; the split avoids a quota mutation while retaining dual-stack restriction.
• Added KMS-encrypted immutable/scanned/lifecycle-bounded ECR; one ECS cluster; digest-pinned non-root web, worker, and one-off migration definitions; separate web/worker services; exactly one sandbox worker; and circuit-breaker rollback. Only web joins the target group. Migration overrides the image entrypoint and is never a service.
• Kept health semantics separate: ECS container /health/live; ALB dependency-aware /health/ready, so database/unapplied-migration failures prevent traffic.
• Added encrypted non-public PostgreSQL 16.14 RDS in isolated subnets, RDS-managed master credentials, backups, storage autoscaling, and explicit final-snapshot/deletion behavior.
• Added six KMS-encrypted secret containers with no Terraform secret versions. Current tasks receive only DATABASE_URL and DJANGO_SECRET_KEY; execution IAM is scoped to the exact repository/log groups/two secrets/key, and the application role has no current AWS API grant.
• Added encrypted bounded ECS/RDS log groups and six basic ECS/ALB/RDS service-health alarms.
• Expanded the production fixture for a different account/region/public+origin hostnames/zone/network/backend, private tasks + NAT, scaling, Multi-AZ, stronger database protection/retention, sizing, and alarm actions.
• Documented the HTTPS boundary, liveness/readiness split, staged first bootstrap, migration-before-rollout contract, secret handling, portability, inventory, and plan-review procedure.

Read-only inventory and plan evidence

Fresh inventory still found no website state object and no matching ECR, ECS, RDS, secret, log, alarm, KMS alias, ALB, or target-group resource. PostgreSQL 16.14 remains orderable for db.t4g.micro; no collision was found.

The final sandbox remote-state plan was run with -refresh=false -lock=false -input=false, without -out:

• Plan: 87 to add, 0 to change, 0 to destroy.
• Exactly six Route 53 resources: existing Create the portable sandbox network, DNS, edge, and TLS foundation #67 public validation + public A/AAAA, and the reviewed Provision the minimum HTTPS-origin scaffold runtime on ECS and RDS #68 expansion of one origin validation subject + origin A/AAAA.
• Both certificate validation resources, the HTTPS listener/rule, two quota-safe ALB SGs, and no hosted-zone resource/lookup or unrelated mutation.
• Terraform redacted the sensitive CloudFront origin/listener values in displayed plan blocks.

Verification

Passed:

• terraform fmt -check -recursive
• uvx ruff check tests/policy/check_terraform_policy.py tests/policy/test_terraform_policy.py
• uv run python -m unittest discover -s tests/policy -p 'test_*.py' -v — 16/16 negative fixtures
• uv run python tests/policy/check_terraform_policy.py .
• sandbox terraform validate
• production fixture terraform validate
• production mock terraform test — 1/1
• final sandbox plan review — 87 add / 0 change / 0 destroy
• git diff --check

Policy coverage now includes HTTPS origin/TLS, exact DNS ownership, direct-origin controls, IPv4/IPv6 prefix-list quota separation, public ingress/database denial, encrypted/bounded logs and RDS controls, immutable/scanned ECR, one digest across tasks, non-root processes, one-off-only migration, liveness/readiness path ownership, scoped IAM, Terraform-managed secret-value rejection, and forbidden state/plan/tfvars artifacts.

Tester handoff / intentionally not claimed

No AWS resources are deployed by this issue run. Live endpoint/redirect/header probes, permitted-network wrong-header/wrong-host denial probes, task replacement/alarm activation, successful/failing migration rollout, and backup behavior require the separately authorized deployment/test stages (including #69/#70). The tracked example digest is deliberately non-existent and secret containers are deliberately empty; operator docs require staged repository/image/secret/migration/service bootstrap. No screenshots apply to this infrastructure-only un-applied change.

[alexeygrigorev]

Tester QA — FAIL

The Terraform implementation and plans pass their static gates, but the configured ALB readiness probe cannot return the required 200 from the current Django scaffold. This is a release blocker for #68 because every web task would remain unhealthy.

Blocking defect: the HTTP ALB readiness probe is redirected or rejected

aws_lb_target_group.web sends HTTP directly to the target on /health/ready and accepts only 200. This health-check request does not traverse the HTTPS listener. AWS documents that target-group health checks use their configured health-check protocol directly against each target and that the generated Host is the target private IP (plus the non-default port): target-group health checks, ALB troubleshooting.

The application uses website.settings.development, where SECURE_SSL_REDIRECT=True, SECURE_PROXY_SSL_HEADER=("HTTP_X_FORWARDED_PROTO", "https"), ALLOWED_HOSTS=["web.dtcdev.click"], and no readiness exemption exists.

Exact local reproductions against the unchanged scaffold:

# Insecure target request, with the private target host temporarily allowed
GET /health/ready Host: 10.0.0.10:8000
=> 301 Location: https://10.0.0.10:8000/health/ready

# Same request with the actual sandbox ALLOWED_HOSTS contract
GET /health/ready Host: 10.0.0.10:8000
=> 400

Commands used:

DJANGO_SETTINGS_MODULE=website.settings.test uv run python manage.py shell -c "... override_settings(SECURE_SSL_REDIRECT=True, SECURE_PROXY_SSL_HEADER=('HTTP_X_FORWARDED_PROTO','https'), ALLOWED_HOSTS=['10.0.0.10']) ... Client().get('/health/ready', HTTP_HOST='10.0.0.10:8000') ..."
DJANGO_SETTINGS_MODULE=website.settings.test uv run python manage.py shell -c "... override_settings(SECURE_SSL_REDIRECT=True, SECURE_PROXY_SSL_HEADER=('HTTP_X_FORWARDED_PROTO','https'), ALLOWED_HOSTS=['web.dtcdev.click']) ... Client().get('/health/ready', HTTP_HOST='10.0.0.10:8000') ..."

Required safe correction: add an exact readiness-path exemption for the private ALB-to-task health probe while keeping SECURE_SSL_REDIRECT=True and the existing exact SECURE_PROXY_SSL_HEADER unchanged. Tests must prove /health/ready can answer the direct HTTP health-check shape, non-health HTTP still redirects, public requests still use the HTTPS CloudFront-to-ALB path, and external/direct-origin bypass remains blocked by the existing SG plus host/header controls. Do not broaden the exemption or treat arbitrary external HTTP as HTTPS.

Acceptance criterion 11 remains unchecked. No live endpoint criterion was claimed.

Passing evidence

• terraform fmt -recursive -check -diff — pass.
• uvx ruff check tests/policy/check_terraform_policy.py tests/policy/test_terraform_policy.py — pass.
• uv run --no-project python -m unittest discover -s tests/policy -p 'test_*.py' -v — 16/16 negative fixtures pass.
• uv run --no-project python tests/policy/check_terraform_policy.py . — pass.
• Sandbox terraform init -backend=false -lockfile=readonly and terraform validate — pass.
• Production fixture terraform init -backend=false -lockfile=readonly, terraform validate, and mock terraform test — pass, 1/1.
• git diff --check — pass.
• Remote-state sandbox plan used -refresh=false -lock=false -input=false and no -out: Plan: 87 to add, 0 to change, 0 to destroy.
• The plan contains exactly six intended Route 53 resources: public validation plus public A/AAAA, and origin validation plus origin A/AAAA. All use explicit zone input; no hosted-zone resource or lookup exists.
• Static source/plan review confirms regional origin ACM plus web.dtcdev.click SAN, ALB HTTPS/TLS policy, CloudFront https-only/TLS 1.2, public aliases to CloudFront, origin aliases to the ALB, and generated header values redacted as sensitive.
• Live account inventory confirms both AWS-managed CloudFront origin lists, and the account SG-rule quota is 60. AWS documents weight 55 for each list, so the two attached ALB SGs avoid an impossible 110-weight single SG: AWS-managed prefix-list weights.
• ECR is immutable, KMS-encrypted, scan-on-push, and lifecycle-bounded. Web/worker are separate services with one desired sandbox worker; migration is task-definition-only; all three use one digest and non-root user in source. Only web has a target-group registration.
• RDS is PostgreSQL 16.14 on an orderable db.t4g.micro/gp3 combination, KMS-encrypted, non-public, task-SG-only, backed up, storage-autoscaling, and final-snapshot explicit.
• Six KMS-encrypted secret containers and no secret-version resources are planned. Tasks inject only DATABASE_URL and DJANGO_SECRET_KEY; execution IAM is scoped to the workload repository, ECS log groups, those two secrets, and the workload key. The application role has no permissions.
• Five encrypted log groups with 14-day retention and six ECS/ALB/RDS alarms are planned.
• /health/live is configured as container liveness and /health/ready as ALB dependency readiness; the blocker above is the missing safe HTTP health-probe exception in Django, not path ownership.
• The production mock proves independent account/region/zone/hostnames, private tasks/NAT, scaling, Multi-AZ, stronger deletion/backup/retention, sizing, and alarm inputs without sandbox state.

AWS/state safety and checklist disposition

Before and after the live read-only plan, 19 narrow inventory probes showed no website state object and no matching DNS, ECR, ECS, RDS, secret, log, alarm, KMS alias, ALB, target-group, ACM, or CloudFront resource. The only textual difference was the rotating STS assumed-role session name; the account remained 817685572750. No apply, import, state mutation, secret-value read/write, binary saved plan, commit, or push occurred.

The already-unchecked live/deployment criteria 5, 6, 7, 10, and 11 remain unchecked. Criterion 5 has source-level secret non-leakage evidence, but its combined direct-request probes require deployment. No screenshots apply to this un-applied infrastructure change.