Repository navigation
Staging & Demo EC2 Access and Management Guide
This page provides guidance for connecting to and managing the staging and demo EC2 instances.
It is intended as a type of notes document, initially capturing setup notes, with more details to be added over time.
- Purpose
- Prerequisites
- Step 1: Configure Environment Variables
- Step 2: Connect to EC2 Instance via SSM
- Step 3: Retrieve and Configure k3s.yaml
- Step 4: Setup Port Forwarding Script
- Step 5: Switch Kubernetes Context
- Step 6: Refresh ECR Secret
- Step 7: Troubleshooting
- Step 8: Helm Scripts
- Validation
- Provide repeatable steps for accessing and managing staging and demo EC2 instances.
- Document environment setup, port forwarding, and context switching.
- Capture troubleshooting commands for pods and secrets.
- Track Helm script usage for deployments.
Install the following tools locally:
Set required AWS environment variables:
export AWS_ACCESS_KEY_ID=<your_key_id>
export AWS_SECRET_ACCESS_KEY=<your_secret_key>
export AWS_ACCOUNT_ID=<your_account_id>
export AWS_DEFAULT_REGION=us-west-2Example command to connect to staging or demo EC2 node:
AWS_DEFAULT_REGION=us-west-2 aws ssm start-session --target <instance_id>-
Request a copy of the
k3s.yamlfile from Rich Salas, or locate it at:/etc/rancher/k3s/k3s.yaml⚠️ Do not modify this file—it will break the k3s instance. -
Open
~/.kube/configand merge in thek3s.yamlcontent.- Rename the context (e.g., use
stgdmoinstead ofdefault).
- Rename the context (e.g., use
Create a script to enable port forwarding:
File: ~/.local/bin/stgdmo_port_forward.sh
#!/bin/sh
AWS_DEFAULT_REGION=us-east-1 aws ssm start-session --target <instance_id> --document-name AWS-StartPortForwardingSession --parameters '{"portNumber":["6443"],"localPortNumber":["6443"]}'Run this script before locally using kubectl.
Switch to the proper context before running commands:
kubectl config use-context <stgdmo>If pods fail due to an outdated ecr-secret, refresh it (Usually happens on ErrImagePull errors):
kubectl delete secret --ignore-not-found ecr-secret -n staging && kubectl create secret docker-registry ecr-secret --docker-username=AWS --docker-email=none --docker-password=$(aws ecr get-login-password --region us-west-2) --docker-server=239442854891.dkr.ecr.us-west-2.amazonaws.com --namespace=stagingkubectl delete secret --ignore-not-found ecr-secret -n demo && kubectl create secret docker-registry ecr-secret --docker-username=AWS --docker-email=none --docker-password=$(aws ecr get-login-password --region us-west-2) --docker-server=239442854891.dkr.ecr.us-west-2.amazonaws.com --namespace=demoCommon commands:
kubectl logs server-xxxxxx -n staging
kubectl describe pod server-xxxxxx -n stagingFor demo, behavior is similar to staging, but deployment tags may differ.
Helm scripts define application metadata (e.g., env vars, mounts, image paths).
Manual intervention may be required to update them.
git clone git@github.com:UnlockedLabs/staging_helmfile.git
cd staging_helmfile
git checkout carddev81/staging_restructuring
helm upgrade unlockedv2 . --namespace staginggit clone git@github.com:UnlockedLabs/v2_deployment_definitions.git
cd v2_deployment_definitions/k3s/v2-cluster
git checkout carddev81/demo_restucturing
helm upgrade unlockedv2 . --namespace demo- Confirm port forwarding is active before using
kubectl. - Run
kubectl config current-contextto ensure you are in the correct context. - Use
kubectl get pods -n stagingorkubectl get pods -n demoto verify pod health. - Ensure secrets are refreshed if images fail to pull.