[AWS] Story 1: CDK Foundation & Project Setup

[mfittko]

Summary

Initialize the AWS CDK project and establish core infrastructure foundations: VPC, secrets management, and configuration CLI.

Epic: #174
Architecture: docs/architecture/planned/aws-ecs-cdk.md

---

Tasks

CDK Project Setup

• Initialize CDK project with TypeScript (infra/ directory)
• Configure cdk.json with context defaults
• Set up TypeScript config and linting
• Add AWS SDK dependencies

VPC Configuration

• Implement VPC with public/private subnets (2 AZs)
• Configure single NAT Gateway (cost optimization)
• Support importing existing VPC via existingVpcId prop
• Create security groups skeleton

Secrets & Config Management

• Create Secrets Manager resources (see mapping below)
• Create SSM Parameter Store structure (see mapping below)
• Implement CLI wrapper (scripts/config.ts) - see commands below
• Add api_providers.example.yaml template to repo
• Add api_providers.yaml to .gitignore

Configuration Props Interface

• Define LlmProxyStackProps with all configurable options
• Document props in README

---

Configuration Mapping

How Config Gets Into Containers

Config Type	Source	Delivery Method
Secrets (tokens, passwords)	Secrets Manager	ECS native secrets injection
Env vars (LOG_LEVEL, etc.)	SSM Parameter Store	ECS native secrets injection
api_providers.yaml	SSM Parameter Store	Init container → shared volume

No .env file needed - ECS handles env vars natively!

Secrets Manager (Sensitive Values)

Injected by ECS as environment variables:

Secret Name	Env Var	Description
llm-proxy/management-token	MANAGEMENT_TOKEN	Admin API authentication
llm-proxy/db-credentials	DATABASE_URL	Aurora credentials (auto-generated)
llm-proxy/redis-auth	REDIS_URL	Redis auth token + URL

Note: Per-provider API keys (OpenAI, Anthropic) are stored per-project in the database, not as global secrets.

SSM Parameter Store (Configuration)

Injected by ECS as environment variables:

Parameter Path	Env Var	Default	Description
/llm-proxy/log-level	LOG_LEVEL	info	Logging verbosity
/llm-proxy/listen-addr	LISTEN_ADDR	:8080	API server address
/llm-proxy/admin-listen-addr	ADMIN_LISTEN_ADDR	:8081	Admin server address
/llm-proxy/db-driver	DB_DRIVER	postgres	Database driver
/llm-proxy/observability-buffer-size	OBSERVABILITY_BUFFER_SIZE	1000	Event bus buffer
/llm-proxy/rate-limit-rpm	RATE_LIMIT_RPM	60	Default rate limit

SSM Parameter Store (YAML Config)

Fetched by init container, written to shared volume:

Parameter Path	File Path	Description
/llm-proxy/api-providers	/config/api_providers.yaml	Full API provider config

---

ECS Task Definition (How It Works)

// CDK - Secrets & Env Vars (ECS native injection)
const container = taskDefinition.addContainer('proxy', {
  secrets: {
    // From Secrets Manager
    'MANAGEMENT_TOKEN': ecs.Secret.fromSecretsManager(managementTokenSecret),
    'DATABASE_URL': ecs.Secret.fromSecretsManager(dbCredentialsSecret),
    'REDIS_URL': ecs.Secret.fromSecretsManager(redisAuthSecret),
    
    // From SSM Parameter Store
    'LOG_LEVEL': ecs.Secret.fromSsmParameter(logLevelParam),
    'DB_DRIVER': ecs.Secret.fromSsmParameter(dbDriverParam),
    // ... other params
  },
  environment: {
    'LISTEN_ADDR': ':8080',
    'API_PROVIDERS_PATH': '/config/api_providers.yaml',
  },
});

// Shared volume for api_providers.yaml (written by init container)
taskDefinition.addVolume({ name: 'config' });
container.addMountPoints({ containerPath: '/config', sourceVolume: 'config' });

Result: Container starts with all env vars populated by ECS, config file on shared volume.

---

Config Loading Flow

┌─────────────────────────────────────────────────────────────┐
│ ECS Task Start                                              │
│                                                             │
│  ECS injects env vars:                                      │
│    MANAGEMENT_TOKEN ← Secrets Manager                       │
│    DATABASE_URL ← Secrets Manager                           │
│    LOG_LEVEL ← SSM Parameter                                │
│    ...                                                      │
└─────────────────────────────────────────────────────────────┘
                          ↓
┌─────────────────────────────────────────────────────────────┐
│ Init Container                                              │
│  1. Acquire DynamoDB lock                                   │
│  2. Fetch /llm-proxy/api-providers from SSM                 │
│  3. Write to /config/api_providers.yaml (shared volume)     │
│  4. Run migrations                                          │
│  5. Release lock                                            │
└─────────────────────────────────────────────────────────────┘
                          ↓
┌─────────────────────────────────────────────────────────────┐
│ Main Container (Proxy)                                      │
│  - Env vars already set by ECS ✓                            │
│  - Reads /config/api_providers.yaml from volume ✓           │
│  - NO app changes needed!                                   │
└─────────────────────────────────────────────────────────────┘

---

API Providers Config

File Structure

config/
├── api_providers.example.yaml  ← Committed (template with placeholders)
└── api_providers.yaml          ← Gitignored (real production config)

Initial Setup Workflow

# 1. Copy example to real config
cp config/api_providers.example.yaml config/api_providers.yaml

# 2. Edit with production values
vim config/api_providers.yaml

# 3. Upload to SSM
cd infra
npm run config:upload-api-providers ../config/api_providers.yaml

# 4. Verify
npm run config:get-api-providers

---

CLI Wrapper Commands

# Interactive setup wizard
npm run setup

# Secrets (Secrets Manager)
npm run secret:set <name> <value>
npm run secret:get <name>

# Config (SSM Parameter Store)
npm run config:set <name> <value>
npm run config:get <name>
npm run config:list

# API Providers (special handling)
npm run config:upload-api-providers <path-to-yaml>
npm run config:get-api-providers

---

Deliverables

infra/
├── bin/app.ts
├── lib/
│   ├── llm-proxy-stack.ts
│   └── constructs/
│       └── config.ts         # Secrets + SSM construct
├── scripts/
│   └── config.ts             # CLI wrapper
├── cdk.json
├── package.json
└── README.md

config/
├── api_providers.example.yaml  ← New template file
└── .gitignore update           ← Ignore api_providers.yaml

---

Acceptance Criteria

• cdk synth generates valid CloudFormation
• cdk deploy creates VPC, secrets, and SSM parameters
• CLI wrapper can set/get all secrets and config
• API providers YAML uploadable to SSM
• api_providers.example.yaml committed as template
• api_providers.yaml gitignored
• Existing VPC can be imported via props
• All configuration documented in README

---

Dependencies

• None (first story)

---

Estimated Effort

Medium-Large - 3-4 days

---

Notes

• SSM Parameter Store standard tier is free (up to 10K parameters)
• Secrets Manager costs $0.40/secret/month → 3 secrets = $1.20/mo
• ECS natively injects secrets/params as env vars - no .env file
• Init container only handles api_providers.yaml (and migrations)
• No LLM Proxy code changes needed