Skip to content

Staging & Demo EC2 Access and Management Guide

Richard Salas edited this page Sep 26, 2025 · 1 revision

Development EC2 Instances (Staging & Demo)

This page provides guidance for connecting to and managing the staging and demo EC2 instances.
It is intended as a type of notes document, initially capturing setup notes, with more details to be added over time.


Table of Contents


Purpose

  • Provide repeatable steps for accessing and managing staging and demo EC2 instances.
  • Document environment setup, port forwarding, and context switching.
  • Capture troubleshooting commands for pods and secrets.
  • Track Helm script usage for deployments.

Prerequisites

Install the following tools locally:


Step 1: Configure Environment Variables

Set required AWS environment variables:

export AWS_ACCESS_KEY_ID=<your_key_id>
export AWS_SECRET_ACCESS_KEY=<your_secret_key>
export AWS_ACCOUNT_ID=<your_account_id>
export AWS_DEFAULT_REGION=us-west-2

Step 2: Connect to EC2 Instance via SSM

Example command to connect to staging or demo EC2 node:

AWS_DEFAULT_REGION=us-west-2 aws ssm start-session --target <instance_id>

Step 3: Retrieve and Configure k3s.yaml

  1. Request a copy of the k3s.yaml file from Rich Salas, or locate it at:

    /etc/rancher/k3s/k3s.yaml
    

    ⚠️ Do not modify this file—it will break the k3s instance.

  2. Open ~/.kube/config and merge in the k3s.yaml content.

    • Rename the context (e.g., use stgdmo instead of default).

Step 4: Setup Port Forwarding Script

Create a script to enable port forwarding:

File: ~/.local/bin/stgdmo_port_forward.sh

#!/bin/sh
AWS_DEFAULT_REGION=us-east-1 aws ssm start-session --target <instance_id>   --document-name AWS-StartPortForwardingSession   --parameters '{"portNumber":["6443"],"localPortNumber":["6443"]}'

Run this script before locally using kubectl.


Step 5: Switch Kubernetes Context

Switch to the proper context before running commands:

kubectl config use-context <stgdmo>

Step 6: Refresh ECR Secret

If pods fail due to an outdated ecr-secret, refresh it (Usually happens on ErrImagePull errors):

Staging

kubectl delete secret --ignore-not-found ecr-secret -n staging && kubectl create secret docker-registry ecr-secret   --docker-username=AWS   --docker-email=none   --docker-password=$(aws ecr get-login-password --region us-west-2)   --docker-server=239442854891.dkr.ecr.us-west-2.amazonaws.com   --namespace=staging

Demo

kubectl delete secret --ignore-not-found ecr-secret -n demo && kubectl create secret docker-registry ecr-secret   --docker-username=AWS   --docker-email=none   --docker-password=$(aws ecr get-login-password --region us-west-2)   --docker-server=239442854891.dkr.ecr.us-west-2.amazonaws.com   --namespace=demo

Step 7: Troubleshooting

Common commands:

kubectl logs server-xxxxxx -n staging
kubectl describe pod server-xxxxxx -n staging

For demo, behavior is similar to staging, but deployment tags may differ.


Step 8: Helm Scripts

Helm scripts define application metadata (e.g., env vars, mounts, image paths).
Manual intervention may be required to update them.

Staging Helm Scripts

git clone git@github.com:UnlockedLabs/staging_helmfile.git
cd staging_helmfile
git checkout carddev81/staging_restructuring
helm upgrade unlockedv2 . --namespace staging

Demo Helm Scripts

git clone git@github.com:UnlockedLabs/v2_deployment_definitions.git
cd v2_deployment_definitions/k3s/v2-cluster
git checkout carddev81/demo_restucturing
helm upgrade unlockedv2 . --namespace demo

Validation

  • Confirm port forwarding is active before using kubectl.
  • Run kubectl config current-context to ensure you are in the correct context.
  • Use kubectl get pods -n staging or kubectl get pods -n demo to verify pod health.
  • Ensure secrets are refreshed if images fail to pull.

Clone this wiki locally