Create staging environment

[ohaiwalt]

We need a staging environment.

MVP can include manual deployment, and must be accessible from the internet, and not using the production database.

Involved in this:

• Verifying a operationcode-staging namespace exists in the cluster
• Creating a staging database, whether RDS or container. Standard Helm postgres deployment works fine, as long as the backend deployment has the appropriate hostname specified.
• Deploying appropriately adjusted backend-secrets Secret object
• Deploying Kubernetes services for frontend and backend
• Deploying OperationCode frontend and backend
• Create Ingress resources for staging deployments. TLS resources are handled by kube-lego
• Point staging domains at ingress controller load balancer in DNSimple (api.staging.operationcode.org for backend and staging.operationcode.org for frontend I believe)

Along the way, thinking about how to make this repeatable, hurdles for secrets management across humans and machines, and making code deployment easier both in the base case and in the update case.

[ohaiwalt]

RDS database created, credentials in 1Password.

Backend secrets object is created, but requires updating and untangling the appropriate environment variables questions here before this will work.

Kubernetes services for frontend and backend were created, as well as an additional service for the redis instance to standardize across default Helm charts, as well as an externalName service so that postgres can be referred to in the same way in both prod and staging environments. (thanks @johnharris85!) PRs for the deployment changes shortly.

Ingress resources up, PR coming up for this shortly as well.

Issues, and will create issues in other repos to address:

• Frontend (https://staging.operationcode.org) is hardcoded to api.operationcode.org, so cannot see staging backend
• Staging backend (https://api.staging.operationcode.org) needs an analysis of which environment variables are safe to leave to production values and which need to be changed

[desnudopenguino]

you might be able to do something with Terraform/Ansible to automate this then pass the infra repo around to devs. then you don't have to maintain a 24/7 live dev environment eating up resources.

[apex-omontgomery]

Marking this as done, there might be some occasional secrets issues but I managed to finish up and verify the staging works a few months back.