Important
This project is designed to manage my offsite setup, which is specific to my use cases, so it might not be directly useful to you. For a ready-to-use solution, please refer to my homelab project.
.
├── flake.nix # Contains dependencies required by this project for both local and CI/CD
├── Makefile # Entry point for all manual actions
├── compose.yaml # Servers required for running locally
├── infra # Infrastructure definition
│ ├── modules # Terraform modules
│ │ ├── network
│ │ ├── instance
│ │ ├── cluster
│ │ └── ...
│ ├── local # Terragrunt configuration for the local environment
│ │ └── ...
│ └── ${ENV} # Terragrunt configuration for the ${ENV} environment
│ ├── root.hcl # Root config used by other Terragrunt files
│ ├── secrets.yaml # Encrypted secrets
│ ├── tfstate # Bootstrap Terraform state
│ ├── ${CLOUD}
│ │ └── ${REGION}
│ │ └── ${MODULE}
│ │ └── terragrunt.hcl
│ ├── metal
│ │ └── vn-south-1
│ │ ├── bootstrap
│ │ │ └── terragrunt.hcl
│ │ └── cluster
│ │ └── terragrunt.hcl
│ └── ...
├── platform # Highly privileged platform components
│ └── ${ENV}
│ ├── grafana.yaml
│ ├── temporal.yaml
│ ├── wireguard.yaml
│ └── ...
├── apps # User applications, standardized with strict controls
│ ├── ${NAMESPACE}
│ │ └── ${APP}
│ │ └── ${ENV}.yaml
│ └── khuedoan
│ └── blog
│ ├── local.yaml
│ └── production.yaml
├── controller # Automation controller for the entire project - think GitHub Actions, but better
│ ├── activities # Temporal activities (git clone, terragrunt apply, etc.)
│ │ ├── git.go
│ │ ├── terragrunt.go
│ │ └── ...
│ ├── workflows # Temporal workflows, define a sequence of activities
│ │ ├── infra.go
│ │ ├── app.go
│ │ └── ...
│ ├── worker # Worker process that executes the workflows
│ └── Dockerfile # Builds the image for the controller, can run locally or on a cluster
└── test # High level tests
- Unified hybrid cloud platform
- Temporal is used as the automation engine, providing the reliability and performance that generic CI/CD engines can only dream of.
- Infra aka IaaS:
- Essentially
cd "infra/${ENV}" && terragrunt apply --all - Includes some graph pruning based on changed files for performance
- Bootstrap ArgoCD to apply the remaining
- Essentially
- Platform aka PaaS:
- Essentially
kubectl apply -f "platform/${ENV}" - However, the runtime doesn’t have access to Git - all manifests are pulled from an OCI registry
- Essentially
- Apps aka SaaS:
- Strict and standardized
- Uses the rendered manifests pattern, essentially
helm template && oras push
| Provider | Service | Usage | Pricing |
|---|---|---|---|
| Cloudflare | R2 Bucket (Terraform state) | 2 | Free |
| Cloudflare | Domain | 2 | 1.67$/month |
| Cloudflare | Load Balancer | 1 | 5$/month |
| Cloudflare | Tunnel | 2 | Free |
| Oracle Cloud | Virtual Cloud Network | 1 | Free |
| Oracle Cloud | VM.Standard.A1.Flex (ARM) |
4 cores, 24GB mem | Free (yes, you read it right!) |
| Oracle Cloud | Block Storage | 200GB | Free |
| Metal | Hardware depreciation | 6.36$/month | |
| Metal | Electricity | 3$/month | |
| Total | 16.03$/month |
- Fork this repository because you will need to customize it for your needs.
- A credit/debit card to register for the accounts.
- Basic knowledge on Terraform, Ansible and Kubernetes (optional, but will help a lot)
Configuration files:
Terraform Cloud
- Create a Terraform Cloud account at https://app.terraform.io
Oracle Cloud
- Create an Oracle Cloud account at https://cloud.oracle.com
- Generate an API signing key:
- Profile menu (User menu icon) -> User Settings -> API Keys -> Add API Key
- Select Generate API Key Pair, download the private key to
~/.oci/private.pemand click Add - Copy the Configuration File Preview to
~/.oci/configand changekey_fileto~/.oci/private.pem
If you see a warning like this, try to avoid those regions:
⚠️ Because of high demand for Arm Ampere A1 Compute capacity in the Foo and Bar regions, A1 instance availability in these regions is limited. If you plan to create A1 instances, we recommend choosing another region as your home region
Install the following packages:
That's it! Run the following command to open the Nix shell:
nix developBuild the infrastructure:
make- Fix OCI plain HTTP for local development
- Config git username and email
- Credentials for the worker (SSH priv + pub + knowhosts?)
