[Gemini] 5.3: Update EC2 worker for project-based targets

[taariq]

Parent: #28

Workflow Instructions for Gemini

Repo: seren-replicator (/Users/taariqlewis/Projects/Seren_Projects/seren-replicator)

When to Start: After completing Task 5.2 (#43)

Dependencies: #43 (Lambda must store project identifiers in DynamoDB first)

---

Task

Update EC2 worker script to fetch connection string from SerenDB Console API when using project-based targets.

File to Create/Modify

Worker script (location may vary - check existing worker scripts in the repo)

What to Change

The worker needs to:

1. Check if target_project_id is present in the DynamoDB job record
2. If yes (API-key flow):
   • Decrypt the seren_api_key from DynamoDB
   • Call SerenDB Console API to get connection string
   • Use returned connection string as TARGET_URL
3. If no (traditional flow):
   • Decrypt target_url directly (existing behavior)

Pseudocode

# Fetch job details from DynamoDB
JOB_DATA=$(aws dynamodb get-item --table-name $TABLE --key '{"job_id":{"S":"'$JOB_ID'"}}')

# Check if using project-based target
TARGET_PROJECT_ID=$(echo $JOB_DATA | jq -r '.Item.target_project_id.S // empty')

if [ -n "$TARGET_PROJECT_ID" ]; then
    # Decrypt API key
    SEREN_API_KEY=$(aws kms decrypt --ciphertext-blob ... | jq -r '.Plaintext' | base64 -d)

    # Fetch connection string from SerenDB Console API
    TARGET_BRANCH_ID=$(echo $JOB_DATA | jq -r '.Item.target_branch_id.S')

    CONN_STRING=$(curl -s -H "Authorization: Bearer $SEREN_API_KEY" \
        "https://console.serendb.com/api/projects/$TARGET_PROJECT_ID/branches/$TARGET_BRANCH_ID/connection-string" \
        | jq -r '.connection_string')

    TARGET_URL="$CONN_STRING"
else
    # Use decrypted target_url directly (existing behavior)
    TARGET_URL=$(aws kms decrypt ... )
fi

# Run replication with resolved TARGET_URL
./database-replicator $COMMAND --source "$SOURCE_URL" --target "$TARGET_URL" ...

See docs/plans/api-key-flow-implementation.md Task 5.3 for details.

How to Test

Manual testing with a real job submission after Lambda changes are deployed.

Commit Message

feat(worker): Fetch connection string from API for project-based targets

[taariq]

Completed Task 5.3 by updating aws/ec2/worker.sh: the worker now detects project-based jobs, decrypts the Seren API key, retrieves the latest connection string from the Seren Console API, and logs whether it used the project identifiers or a direct target URL. Added helper fail_job_and_exit, new env defaults for the console endpoint/timeout, and plumbing for target_project_id, target_branch_id, target_databases, and seren_api_key_encrypted from DynamoDB.