-
Notifications
You must be signed in to change notification settings - Fork 1.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: Example of AWS CodeCommit as a repository for ArgoCD workloads #1509
Changes from all commits
0ab3b50
71a37f1
ec0bef2
43a6f88
5a35273
3b99531
7044035
ea2962e
d60b0e5
c0ebf85
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,170 @@ | ||
# EKS Cluster with ArgoCD and Workloads in private AWS CodeCommit repository | ||
|
||
This example shows how to provision an EKS cluster with: | ||
|
||
- ArgoCD | ||
- Workloads and addons deployed by ArgoCD (main.ft, locals.tf, providers.tf, outputs.tf, data.tf) | ||
- Private AWS CodeCommit repository and AWS Lambda stack provisioned by included Terraform script (repo-cc.tf) and used for workloads | ||
- AWS Lambda triggers ArgoCD projects sync using ArgoCD webhook after each push to the AWS CodeCommit repository | ||
|
||
![Architectural diagram](images/argocd-cc.png) | ||
|
||
To better understand how ArgoCD works with EKS Blueprints, read the EKS Blueprints ArgoCD [Documentation](https://aws-ia.github.io/terraform-aws-eks-blueprints/latest/add-ons/argocd/) | ||
|
||
## Reference Documentation | ||
|
||
- [Documentation](https://aws-ia.github.io/terraform-aws-eks-blueprints/latest/add-ons/argocd/) | ||
- [EKS Blueprints Add-ons Repo](https://github.com/aws-samples/eks-blueprints-add-ons) | ||
- [EKS Blueprints Workloads Repo](https://github.com/aws-samples/eks-blueprints-workloads) | ||
- [EKS Blueprints for Terraform Workshop](https://catalog.workshops.aws/eks-blueprints-terraform/en-US) | ||
|
||
## Prerequisites | ||
|
||
Main Terraform files (main.ft, locals.tf, providers.tf, outputs.tf, data.tf) were created using instructions from [EKS Blueprints for Terraform Workshop](https://catalog.workshops.aws/eks-blueprints-terraform/en-US) | ||
Solution specific script is repo-cc.tf | ||
|
||
Ensure that you have the following tools installed locally: | ||
|
||
1. [aws cli](https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html) | ||
2. [kubectl](https://Kubernetes.io/docs/tasks/tools/) | ||
3. [terraform](https://learn.hashicorp.com/tutorials/terraform/install-cli) | ||
|
||
Change the link in the examples/gitops/argocd-codecommit/locals.tf of GitHub repository for workloads from [EKS Blueprints Workloads Repo](https://github.com/aws-samples/eks-blueprints-workloads) to AWS CodeCommit repository provisioned by the stack in repo-cc.tf | ||
|
||
```sh | ||
#--------------------------------------------------------------- | ||
# ARGOCD WORKLOAD APPLICATION | ||
#--------------------------------------------------------------- | ||
# workload_repo = "https://github.com/aws-samples/eks-blueprints-workloads.git" | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I don't quite follow what is happening here - what should users do, what sequence of events, etc. |
||
workload_repo = aws_codecommit_repository.workloads_repo_cc.clone_url_http | ||
``` | ||
|
||
Update main.tf and enable workloads and addons (if not enabled yet). | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Why do they need to change this, shouldn't this just be set in the pattern/example? |
||
|
||
```sh | ||
argocd_applications = { | ||
addons = local.addon_application # YOU NEED TO KEEP ADDONS | ||
workloads = local.workload_application # Comment to remove workloads | ||
} | ||
``` | ||
|
||
## Deploy | ||
|
||
- Provision this example in the solution directory: | ||
|
||
```sh | ||
cd examples/gitops/argocd-codecommit | ||
terraform init | ||
terraform plan | ||
terraform apply | ||
``` | ||
|
||
Enter `yes` at command prompt to apply | ||
|
||
- Configure kubectl using output | ||
|
||
```sh | ||
terraform output configure_kubectl | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Copy boiler plate text from other examples - this looks a bit different here |
||
``` | ||
|
||
- Clone [EKS Blueprints Workloads Repo](https://github.com/aws-samples/eks-blueprints-workloads) from Github to AWS CodeCommit (clone2cc.sh) | ||
|
||
```sh | ||
pushd ../../../.. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
|
||
git clone https://github.com/aws-samples/eks-blueprints-workloads.git | ||
git clone https://git-codecommit.$AWS_REGION.amazonaws.com/v1/repos/eks-blueprints-workloads-cc | ||
cd eks-blueprints-workloads-cc | ||
git checkout -b main | ||
cd .. | ||
rsync -av eks-blueprints-workloads/ eks-blueprints-workloads-cc --exclude .git | ||
cd eks-blueprints-workloads-cc | ||
git add . && git commit -m "initial commit" && git push --set-upstream origin main | ||
popd | ||
``` | ||
|
||
- Wait some time until Elastic Load Balancer will be associated with ArgoCD service (External IP = ELB Url) | ||
|
||
```sh | ||
kubectl get svc argo-cd-argocd-server -n argocd | ||
``` | ||
|
||
``` | ||
NAME TYPE CLUSTER-IP EXTERNAL-IP | ||
argo-cd-argocd-server LoadBalancer 172.20.228.171 ...elb.amazonaws.com ... | ||
``` | ||
|
||
- Set ArgoCD url to environment variable | ||
|
||
```sh | ||
export TF_VAR_argocd_url=`kubectl get svc argo-cd-argocd-server -n argocd -o json | jq --raw-output '.status.loadBalancer.ingress[0].hostname'` | ||
``` | ||
|
||
- Update stack to set ArgoCD Url for AWS CodeCommit webhook notifications on Git push | ||
|
||
```sh | ||
terraform plan | ||
terraform apply | ||
``` | ||
|
||
Enter `yes` at command prompt to apply | ||
|
||
- Use ArgoCD Url, "admin" user and password below to logging to ArgoCD UI | ||
|
||
```sh | ||
echo "ArgoCD URL: https://$TF_VAR_argocd_url" | ||
echo "ArgoCD server user: admin" | ||
echo "ArgoCD admin password: $(kubectl -n argocd get secret argocd-initial-admin-secret -o jsonpath="{.data.password}" | base64 -d)" | ||
``` | ||
|
||
The following environment should be deployed. | ||
![Workloads in ArgoCD](images/argocd-cc-workloads-tf.png) | ||
|
||
## Destroy | ||
|
||
To teardown and [remove](https://catalog.workshops.aws/eks-blueprints-terraform/en-US/050-cleanup) the resources created in this example: | ||
|
||
- Delete the Workloads | ||
|
||
Update main.tf with the removal of workloads except addons. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. See other examples, we do targetted applies/destroys to ensure proper deploy and clean-up |
||
|
||
```sh | ||
argocd_applications = { | ||
addons = local.addon_application # YOU NEED TO KEEP ADDONS | ||
#workloads = local.workload_application # Comment to remove workloads | ||
} | ||
``` | ||
|
||
```sh | ||
terraform plan | ||
terraform apply | ||
``` | ||
|
||
Enter `yes` at command prompt to apply | ||
|
||
- Remove Kubernetes AddOns modules | ||
|
||
```sh | ||
terraform destroy -target=module.kubernetes_addons | ||
``` | ||
|
||
Enter `yes` at command prompt to apply | ||
|
||
- delete EKS Cluster | ||
|
||
```sh | ||
terraform destroy -target=module.eks_blueprints | ||
``` | ||
|
||
Enter `yes` at command prompt to apply | ||
|
||
- Delete the remaining modules from the main.tf | ||
|
||
```sh | ||
terraform destroy | ||
``` | ||
|
||
Enter `yes` at command prompt to apply | ||
|
||
- Delete EKS Blueprint Log Groups in CloudWatch. | ||
|
||
![Log Groups in CloudWatch](images/argocd-cc-loggroup.png) |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,10 @@ | ||
pushd ../../../.. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. See comments above - in the README lets just reference the helper script so we don't have duplicate info |
||
git clone https://github.com/aws-samples/eks-blueprints-workloads.git | ||
git clone https://git-codecommit.$AWS_REGION.amazonaws.com/v1/repos/eks-blueprints-workloads-cc | ||
cd eks-blueprints-workloads-cc | ||
git checkout -b main | ||
cd .. | ||
rsync -av eks-blueprints-workloads/ eks-blueprints-workloads-cc --exclude .git | ||
cd eks-blueprints-workloads-cc | ||
git add . && git commit -m "initial commit" && git push --set-upstream origin main | ||
popd |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,18 @@ | ||
# Find the user currently in use by AWS | ||
data "aws_caller_identity" "current" {} | ||
|
||
# Region in which to deploy the solution | ||
# data "aws_region" "current" {} | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Remove any commented out code that is not used please |
||
|
||
# Availability zones to use in our solution | ||
data "aws_availability_zones" "available" { | ||
state = "available" | ||
} | ||
|
||
# data "aws_eks_cluster" "cluster" { | ||
# name = module.eks_blueprints.eks_cluster_id | ||
# } | ||
|
||
data "aws_eks_cluster_auth" "this" { | ||
name = module.eks_blueprints.eks_cluster_id | ||
} |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,52 @@ | ||
/* eslint-env node */ | ||
const https = require('https'); // eslint-disable-line | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Lets remove the lambda function which acts like a webhook. ArgoCD should be able to sync as part of its reconciliation process There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Sorry for the intrusion. If that's the case, I would say you are missing a big chance to bring the same capabilities that GitHub and GitLab already offer. This is the feature that brought me to this PR in the first place, to be honest :) |
||
|
||
/* webhook payload | ||
{ | ||
"ref": "refs/heads/master", | ||
"repository": { | ||
"html_url": "https://git-codecommit.us-west-2.amazonaws.com/v1/repos/eks-blueprints-workloads-cc", | ||
"default_branch": "master" | ||
} | ||
} | ||
*/ | ||
|
||
exports.handler = async function(event) { | ||
const eventSourceARNarray = event.Records[0].eventSourceARN.split(':'); | ||
const repoName = eventSourceARNarray[eventSourceARNarray.length - 1]; | ||
const ref = event.Records[0].codecommit.references[0].ref; | ||
const refArray = ref.split('/'); | ||
const branch = refArray[refArray.length - 1]; | ||
const data = JSON.stringify({ | ||
"ref": ref, | ||
"repository": { | ||
"html_url": "https://git-codecommit." + event.Records[0].awsRegion + ".amazonaws.com/v1/repos/" + repoName, | ||
"default_branch": branch | ||
} | ||
}); | ||
console.log(data); | ||
|
||
const options = { | ||
hostname: event.Records[0].customData, | ||
path: '/api/webhook', | ||
method: 'POST', | ||
port: 443, | ||
headers: { | ||
'Content-Type': 'application/json', | ||
'X-GitHub-Event': 'push', | ||
'Content-Length': data.length, | ||
}, | ||
}; | ||
|
||
const promise = new Promise(function(resolve, reject) { | ||
process.env["NODE_TLS_REJECT_UNAUTHORIZED"] = 0; | ||
const req = https.request(options, (res) => { | ||
resolve(res.statusCode); | ||
}).on('error', (e) => { | ||
reject(Error(e)); | ||
}); | ||
req.write(data); | ||
req.end(); | ||
}); | ||
return promise; | ||
}; |
Original file line number | Diff line number | Diff line change | ||||
---|---|---|---|---|---|---|
@@ -0,0 +1,55 @@ | ||||||
locals { | ||||||
|
||||||
name = basename(path.cwd) | ||||||
# region = data.aws_region.current.name | ||||||
cluster_version = "1.23" | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Lets use the latest - please ensure its validated with this version
Suggested change
|
||||||
|
||||||
vpc_cidr = "10.0.0.0/16" | ||||||
azs = slice(data.aws_availability_zones.available.names, 0, 3) | ||||||
|
||||||
node_group_name = "managed-ondemand" | ||||||
|
||||||
env = "dev" | ||||||
|
||||||
#--------------------------------------------------------------- | ||||||
# ARGOCD ADD-ON APPLICATION | ||||||
#--------------------------------------------------------------- | ||||||
|
||||||
addon_application = { | ||||||
path = "chart" | ||||||
repo_url = "https://github.com/aws-samples/eks-blueprints-add-ons.git" | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. should this be a codecommit URL? |
||||||
add_on_application = true | ||||||
} | ||||||
|
||||||
#--------------------------------------------------------------- | ||||||
# ARGOCD WORKLOAD APPLICATION | ||||||
#--------------------------------------------------------------- | ||||||
# workload_repo = "https://github.com/aws-samples/eks-blueprints-workloads.git" | ||||||
workload_repo = aws_codecommit_repository.workloads_repo_cc.clone_url_http | ||||||
|
||||||
workload_application = { | ||||||
path = "envs/dev" | ||||||
repo_url = local.workload_repo | ||||||
add_on_application = false | ||||||
values = { | ||||||
labels = { | ||||||
env = local.env | ||||||
myapp = "myvalue" | ||||||
} | ||||||
spec = { | ||||||
source = { | ||||||
repoURL = local.workload_repo | ||||||
} | ||||||
blueprint = "terraform" | ||||||
clusterName = local.name | ||||||
#karpenterInstanceProfile = "${local.name}-${local.node_group_name}" # Activate to enable Karpenter manifests (only when Karpenter add-on will be enabled in the Karpenter module) | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. remove commented out code please |
||||||
env = local.env | ||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. do we need this, whats it used for? |
||||||
} | ||||||
} | ||||||
} | ||||||
|
||||||
tags = { | ||||||
Blueprint = local.name | ||||||
GithubRepo = "github.com/aws-ia/terraform-aws-eks-blueprints" | ||||||
} | ||||||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Lets add in why this pattern is useful to users - what would be the primary motivation to use CodeCommit, why any additional steps are necessary to integrate with CodeCommit, etc.