Skip to content

Stop cloud-init from tracing demo secrets#87

Merged
ChelseaKR merged 1 commit into
mainfrom
agent/cloud-init-secret-tracing
Jul 12, 2026
Merged

Stop cloud-init from tracing demo secrets#87
ChelseaKR merged 1 commit into
mainfrom
agent/cloud-init-secret-tracing

Conversation

@ChelseaKR

Copy link
Copy Markdown
Owner

What changed

Root cause

The first-boot template used set -euxo pipefail. Later secret retrieval was added without removing xtrace, allowing expanded secret assignments to reach the IAM-restricted EC2 console log.

Incident response

Both values were rotated immediately, the synthetic archive volume was destroyed and reseeded under the replacement vault key, and health returned to 200. No real contributor data existed and the values were not public.

Impact

The change affects only first-boot command logging. It does not alter Terraform resources, archive formats, or application behavior.

Rollback

Reverting would reintroduce the exposure and is not a safe rollback. If deployment fails, keep the current healthy instance and diagnose without restoring xtrace; infra/aws/destroy.sh remains the full infrastructure rollback.

Validation

  • make verify — 981 tests and all portable gates green
  • targeted infrastructure regression test
  • Terraform formatting and validation
  • secrets rotated and live /healthz restored before this PR

Closes #86

@ChelseaKR
ChelseaKR merged commit 20304dc into main Jul 12, 2026
16 checks passed
@ChelseaKR
ChelseaKR deleted the agent/cloud-init-secret-tracing branch July 12, 2026 05:52
@ChelseaKR

Copy link
Copy Markdown
Owner Author

Production verification completed for merged hotfix 20304dc.\n\n- Replaced EC2 instance: i-007a3d8eaf71e0e68\n- Retained EIP/DNS: https://ledger.chelseakr.com\n- EC2 instance and system checks: ok/ok\n- SSM: Online\n- Console audit: 0 occurrences for VAULT_KEY=, CLAIM_SECRET=, LEDGER_VAULT_KEY=, and LEDGER_CLAIM_SECRET=\n- Intended public routes: 200; disabled /contribute remains 404\n- TLS: valid Let's Encrypt certificate\n- Security headers: HSTS, CSP, no-referrer, and nosniff present\n- Terraform detailed exit code: 0 (no drift)\n- Stability watch: 30/30 samples returned 200 for both / and /healthz\n\nThe exposed synthetic-demo secret values were rotated during containment before this replacement. No real or user data was involved. Rollback remains available through infra/aws/destroy.sh.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

SEV2: cloud-init tracing exposed synthetic demo secrets to EC2 console

1 participant