Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Example of AWS CodeCommit as a repository for ArgoCD workloads #1509

Closed
wants to merge 10 commits into from
Closed
170 changes: 170 additions & 0 deletions examples/gitops/argocd-codecommit/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,170 @@
# EKS Cluster with ArgoCD and Workloads in private AWS CodeCommit repository

This example shows how to provision an EKS cluster with:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lets add in why this pattern is useful to users - what would be the primary motivation to use CodeCommit, why any additional steps are necessary to integrate with CodeCommit, etc.


- ArgoCD
- Workloads and addons deployed by ArgoCD (main.ft, locals.tf, providers.tf, outputs.tf, data.tf)
- Private AWS CodeCommit repository and AWS Lambda stack provisioned by included Terraform script (repo-cc.tf) and used for workloads
- AWS Lambda triggers ArgoCD projects sync using ArgoCD webhook after each push to the AWS CodeCommit repository

![Architectural diagram](images/argocd-cc.png)

To better understand how ArgoCD works with EKS Blueprints, read the EKS Blueprints ArgoCD [Documentation](https://aws-ia.github.io/terraform-aws-eks-blueprints/latest/add-ons/argocd/)

## Reference Documentation

- [Documentation](https://aws-ia.github.io/terraform-aws-eks-blueprints/latest/add-ons/argocd/)
- [EKS Blueprints Add-ons Repo](https://github.com/aws-samples/eks-blueprints-add-ons)
- [EKS Blueprints Workloads Repo](https://github.com/aws-samples/eks-blueprints-workloads)
- [EKS Blueprints for Terraform Workshop](https://catalog.workshops.aws/eks-blueprints-terraform/en-US)

## Prerequisites

Main Terraform files (main.ft, locals.tf, providers.tf, outputs.tf, data.tf) were created using instructions from [EKS Blueprints for Terraform Workshop](https://catalog.workshops.aws/eks-blueprints-terraform/en-US)
Solution specific script is repo-cc.tf

Ensure that you have the following tools installed locally:

1. [aws cli](https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html)
2. [kubectl](https://Kubernetes.io/docs/tasks/tools/)
3. [terraform](https://learn.hashicorp.com/tutorials/terraform/install-cli)

Change the link in the examples/gitops/argocd-codecommit/locals.tf of GitHub repository for workloads from [EKS Blueprints Workloads Repo](https://github.com/aws-samples/eks-blueprints-workloads) to AWS CodeCommit repository provisioned by the stack in repo-cc.tf

```sh
#---------------------------------------------------------------
# ARGOCD WORKLOAD APPLICATION
#---------------------------------------------------------------
# workload_repo = "https://github.com/aws-samples/eks-blueprints-workloads.git"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't quite follow what is happening here - what should users do, what sequence of events, etc.

workload_repo = aws_codecommit_repository.workloads_repo_cc.clone_url_http
```

Update main.tf and enable workloads and addons (if not enabled yet).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do they need to change this, shouldn't this just be set in the pattern/example?


```sh
argocd_applications = {
addons = local.addon_application # YOU NEED TO KEEP ADDONS
workloads = local.workload_application # Comment to remove workloads
}
```

## Deploy

- Provision this example in the solution directory:

```sh
cd examples/gitops/argocd-codecommit
terraform init
terraform plan
terraform apply
```

Enter `yes` at command prompt to apply

- Configure kubectl using output

```sh
terraform output configure_kubectl
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy boiler plate text from other examples - this looks a bit different here

```

- Clone [EKS Blueprints Workloads Repo](https://github.com/aws-samples/eks-blueprints-workloads) from Github to AWS CodeCommit (clone2cc.sh)

```sh
pushd ../../../..
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. Why do we need to go up to root?
  2. We should ensure any local copies are cleaned up

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. Why only copy over the workloads repo? What about the repository where the Terraform code is defined (i.e. - eks-blueprints), and the repository where the ArgoCD manifests are located (i.e. - eks-addons)?

git clone https://github.com/aws-samples/eks-blueprints-workloads.git
git clone https://git-codecommit.$AWS_REGION.amazonaws.com/v1/repos/eks-blueprints-workloads-cc
cd eks-blueprints-workloads-cc
git checkout -b main
cd ..
rsync -av eks-blueprints-workloads/ eks-blueprints-workloads-cc --exclude .git
cd eks-blueprints-workloads-cc
git add . && git commit -m "initial commit" && git push --set-upstream origin main
popd
```

- Wait some time until Elastic Load Balancer will be associated with ArgoCD service (External IP = ELB Url)

```sh
kubectl get svc argo-cd-argocd-server -n argocd
```

```
NAME TYPE CLUSTER-IP EXTERNAL-IP
argo-cd-argocd-server LoadBalancer 172.20.228.171 ...elb.amazonaws.com ...
```

- Set ArgoCD url to environment variable

```sh
export TF_VAR_argocd_url=`kubectl get svc argo-cd-argocd-server -n argocd -o json | jq --raw-output '.status.loadBalancer.ingress[0].hostname'`
```

- Update stack to set ArgoCD Url for AWS CodeCommit webhook notifications on Git push

```sh
terraform plan
terraform apply
```

Enter `yes` at command prompt to apply

- Use ArgoCD Url, "admin" user and password below to logging to ArgoCD UI

```sh
echo "ArgoCD URL: https://$TF_VAR_argocd_url"
echo "ArgoCD server user: admin"
echo "ArgoCD admin password: $(kubectl -n argocd get secret argocd-initial-admin-secret -o jsonpath="{.data.password}" | base64 -d)"
```

The following environment should be deployed.
![Workloads in ArgoCD](images/argocd-cc-workloads-tf.png)

## Destroy

To teardown and [remove](https://catalog.workshops.aws/eks-blueprints-terraform/en-US/050-cleanup) the resources created in this example:

- Delete the Workloads

Update main.tf with the removal of workloads except addons.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See other examples, we do targetted applies/destroys to ensure proper deploy and clean-up


```sh
argocd_applications = {
addons = local.addon_application # YOU NEED TO KEEP ADDONS
#workloads = local.workload_application # Comment to remove workloads
}
```

```sh
terraform plan
terraform apply
```

Enter `yes` at command prompt to apply

- Remove Kubernetes AddOns modules

```sh
terraform destroy -target=module.kubernetes_addons
```

Enter `yes` at command prompt to apply

- delete EKS Cluster

```sh
terraform destroy -target=module.eks_blueprints
```

Enter `yes` at command prompt to apply

- Delete the remaining modules from the main.tf

```sh
terraform destroy
```

Enter `yes` at command prompt to apply

- Delete EKS Blueprint Log Groups in CloudWatch.

![Log Groups in CloudWatch](images/argocd-cc-loggroup.png)
10 changes: 10 additions & 0 deletions examples/gitops/argocd-codecommit/clone2cc.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
pushd ../../../..
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See comments above - in the README lets just reference the helper script so we don't have duplicate info

git clone https://github.com/aws-samples/eks-blueprints-workloads.git
git clone https://git-codecommit.$AWS_REGION.amazonaws.com/v1/repos/eks-blueprints-workloads-cc
cd eks-blueprints-workloads-cc
git checkout -b main
cd ..
rsync -av eks-blueprints-workloads/ eks-blueprints-workloads-cc --exclude .git
cd eks-blueprints-workloads-cc
git add . && git commit -m "initial commit" && git push --set-upstream origin main
popd
18 changes: 18 additions & 0 deletions examples/gitops/argocd-codecommit/data.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
# Find the user currently in use by AWS
data "aws_caller_identity" "current" {}

# Region in which to deploy the solution
# data "aws_region" "current" {}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove any commented out code that is not used please


# Availability zones to use in our solution
data "aws_availability_zones" "available" {
state = "available"
}

# data "aws_eks_cluster" "cluster" {
# name = module.eks_blueprints.eks_cluster_id
# }

data "aws_eks_cluster_auth" "this" {
name = module.eks_blueprints.eks_cluster_id
}
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
52 changes: 52 additions & 0 deletions examples/gitops/argocd-codecommit/lambda/index.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
/* eslint-env node */
const https = require('https'); // eslint-disable-line
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lets remove the lambda function which acts like a webhook. ArgoCD should be able to sync as part of its reconciliation process

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for the intrusion.
Are you suggesting, @bryantbiggs , to wait until Argo CD's scheduled reconciliation happens?

If that's the case, I would say you are missing a big chance to bring the same capabilities that GitHub and GitLab already offer.
In practice, I would dare to say that users are not willing to wait the 3 minutes of scheduled pull that comes by default.

This is the feature that brought me to this PR in the first place, to be honest :)


/* webhook payload
{
"ref": "refs/heads/master",
"repository": {
"html_url": "https://git-codecommit.us-west-2.amazonaws.com/v1/repos/eks-blueprints-workloads-cc",
"default_branch": "master"
}
}
*/

exports.handler = async function(event) {
const eventSourceARNarray = event.Records[0].eventSourceARN.split(':');
const repoName = eventSourceARNarray[eventSourceARNarray.length - 1];
const ref = event.Records[0].codecommit.references[0].ref;
const refArray = ref.split('/');
const branch = refArray[refArray.length - 1];
const data = JSON.stringify({
"ref": ref,
"repository": {
"html_url": "https://git-codecommit." + event.Records[0].awsRegion + ".amazonaws.com/v1/repos/" + repoName,
"default_branch": branch
}
});
console.log(data);

const options = {
hostname: event.Records[0].customData,
path: '/api/webhook',
method: 'POST',
port: 443,
headers: {
'Content-Type': 'application/json',
'X-GitHub-Event': 'push',
'Content-Length': data.length,
},
};

const promise = new Promise(function(resolve, reject) {
process.env["NODE_TLS_REJECT_UNAUTHORIZED"] = 0;
const req = https.request(options, (res) => {
resolve(res.statusCode);
}).on('error', (e) => {
reject(Error(e));
});
req.write(data);
req.end();
});
return promise;
};
55 changes: 55 additions & 0 deletions examples/gitops/argocd-codecommit/locals.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
locals {

name = basename(path.cwd)
# region = data.aws_region.current.name
cluster_version = "1.23"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lets use the latest - please ensure its validated with this version

Suggested change
cluster_version = "1.23"
cluster_version = "1.25"


vpc_cidr = "10.0.0.0/16"
azs = slice(data.aws_availability_zones.available.names, 0, 3)

node_group_name = "managed-ondemand"

env = "dev"

#---------------------------------------------------------------
# ARGOCD ADD-ON APPLICATION
#---------------------------------------------------------------

addon_application = {
path = "chart"
repo_url = "https://github.com/aws-samples/eks-blueprints-add-ons.git"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should this be a codecommit URL?

add_on_application = true
}

#---------------------------------------------------------------
# ARGOCD WORKLOAD APPLICATION
#---------------------------------------------------------------
# workload_repo = "https://github.com/aws-samples/eks-blueprints-workloads.git"
workload_repo = aws_codecommit_repository.workloads_repo_cc.clone_url_http

workload_application = {
path = "envs/dev"
repo_url = local.workload_repo
add_on_application = false
values = {
labels = {
env = local.env
myapp = "myvalue"
}
spec = {
source = {
repoURL = local.workload_repo
}
blueprint = "terraform"
clusterName = local.name
#karpenterInstanceProfile = "${local.name}-${local.node_group_name}" # Activate to enable Karpenter manifests (only when Karpenter add-on will be enabled in the Karpenter module)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove commented out code please

env = local.env
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we need this, whats it used for?

}
}
}

tags = {
Blueprint = local.name
GithubRepo = "github.com/aws-ia/terraform-aws-eks-blueprints"
}
}
Loading