devops-stack-module-argocd

A DevOps Stack module to deploy and configure Argo CD.

The Argo CD chart used by this module is shipped in this repository as well, in order to avoid any unwanted behaviors caused by unsupported versions.

Current Chart Version	Original Repository	Default Values
5.27.1	Chart	values.yaml

Usage

Important

The root of this repository contains the final Argo CD module to be deployed, which uses a Terraform provider for Argo CD to deploy the Argo CD chart. On the first deployment of a cluster, you’ll want to use the bootstrap module instead. Check the bootstrap module’s documentation for more information.

To deploy the final Argo CD module, you’ll need to add the following declaration on your Terraform configuration:

module "argocd" {
  source = "git::https://github.com/camptocamp/devops-stack-module-argocd.git?ref=<RELEASE>"

  cluster_name   = local.cluster_name
  base_domain    = local.base_domain
  cluster_issuer = local.cluster_issuer

  admin_enabled            = "true"
  namespace                = module.argocd_bootstrap.argocd_namespace
  accounts_pipeline_tokens = module.argocd_bootstrap.argocd_accounts_pipeline_tokens
  server_secretkey         = module.argocd_bootstrap.argocd_server_secretkey

  dependency_ids = {
    argocd                = module.argocd_bootstrap.id
    traefik               = module.traefik.id
    cert-manager          = module.cert-manager.id
    oidc                  = module.oidc.id
    kube-prometheus-stack = module.kube-prometheus-stack.id
  }
}

A more complex declaration, that includes a OIDC configuration for the login (this way you avoid having to login using the admin password found on a Kubernetes secret) and the configuration of some other source repositories, would be:

module "argocd" {
  source = "git::https://github.com/camptocamp/devops-stack-module-argocd.git?ref=<RELEASE>"

  ...

  oidc = {
    name         = "OIDC"
    issuer       = module.oidc.oidc.issuer_url
    clientID     = module.oidc.oidc.client_id
    clientSecret = module.oidc.oidc.client_secret
    requestedIDTokenClaims = {
      groups = {
        essential = true
      }
    }
    requestedScopes = [
      "openid", "profile", "email"
    ]
  }

  repositories = {
    cluster-apps = {
      ...
    }
    user-apps = {
      ...
    }
  }

  ...
}

Tip	The sources can also be configured using the Application and ApplicationSet modules.

You can also overload the policy.csv as shown in the following example:

module "argocd" {
  source = "git::https://github.com/camptocamp/devops-stack-module-argocd.git?ref=<RELEASE>"

  ...

  rbac = {
    policy_csv = <<-EOT
      g, pipeline, role:admin
      g, argocd-admin, role:admin
      p, role:read-sync, applications, get, */*, allow
      p, role:read-sync, applications, sync, */*, allow
      p, role:read-sync, repositories, get, *, allow
      p, role:read-sync, projects, get, *, allow
    EOT
  }

  ...
}

Note	By default, this module has a policy.csv that is configured to give administrator access to any user belonging to the groups argocd-admin or devops-stack-admins.

Custom Plugins

This module installs two custom plugins:

1. "kustomized-helm" is just a combination of Kustomize and Helm that allows using Helm charts and then applying Kustomize overrides to the result.

2. "helmfile-sops" adds support for Helmfile managed applications, and integrates the support for SOPS. This is a custom plugin developed by Camptocamp (source code available here).

Note

When using SOPS, you will want to give it access to an external secrets management or encryption service. For this, you will want to pass an AWS IAM role, and Azure Workload Identity Client-ID, or an Azure AAD Pod Identity using the variables repo_server_iam_role_arn, repo_server_azure_workload_identity_clientid`, or repo_server_aadpodidbinding.

Troubleshooting

connection_error during terraform apply

When deploying this module of Argo CD you may experience connection errors (which is normal given that argocd-server pod could have been redeployed).

│ Error: Error while waiting for application argocd to be created
│
│   with module.argocd.argocd_application.this,
│   on .terraform/modules/argocd/main.tf line 55, in resource "argocd_application" "this":
│   55: resource "argocd_application" "this" {
│
│ error while waiting for application argocd to be synced and healthy: rpc error: code = Unavailable desc = connection error: desc = "transport: error while dialing: dial tcp 127.0.0.1:44461: connect:
│ connection refused"

When bootstrapping a cluster for the first time, you can simply run terraform apply again and the deployment should finish correctly.

Note

There are some settings that force the automatic redeployment of the argocd-server pod during the first bootstrap of the cluster. A good example is the admin_enabled variable which is set as true for the bootstrap but not for the final Argo CD module. If you pass the admin_enabled = true to the final Argo CD module you will most likely not encounter this error on the first deployment of you cluster, however you could run into the error of the interface looping whenever you click login (see the other troubleshoot sections).

However, on some cases (notably when upgrading the Argo CD module), this error could leave the Terraform resource tainted.

terraform plan
  # module.devops_stack_blue.module.argocd.argocd_application.this is tainted, so must be replaced
-/+ resource "argocd_application" "this" {
      ~ id      = "argocd:argocd" -> (known after apply)
        # (2 unchanged attributes hidden)

      ~ metadata {
        ...
        }

      ~ spec {
        ...
        }
    }

Plan: 1 to add, 1 to change, 1 to destroy.

Untainting the resource with the command terraform untaint module.argocd.argocd_application.this should solve the issue.

Argo CD interface reload loop when clicking on login

If you encounter a loop when clicking on the login button on the Argo CD interface, you can try to delete the Argo CD server pod and let it be recreated.

This error usually happens because there are some unapplied settings between the bootstrap Argo CD and the final Argo CD module. If you already encountered the error from the previous step, then this error should not happen, as it means the pod was already recreated.

Technical Documentation

Dependencies

module.argocd_bootstrap.id

Obviously, this module needs an already working Argo CD (the bootstrap), so it depends on module.argocd_bootstrap.

module.traefik.id

Since there is an ingress deployed with this module, it needs to be deployed after Traefik so it depends on module.ingress.

module.traefik.id

For the same reason as the previous dependency, it needs to be deployed after cert-manager so it depends on module.cert-manager.

module.oidc.id

Only for the platforms that deploy a OIDC module, such as EKS, KinD or SKS, there is also a the dependency on module.oidc.id.

module.kube-prometheus-stack.id

Finally, the kube-prometheus-stack is a requirement because this Argo CD module requires the ServiceMonitor CRD so it depends on module.kube-prometheus-stack.

Requirements

The following requirements are needed by this module:

• terraform (>= 1.2)

• argocd (>= 6)

• htpasswd (>= 1)

• jwt (>= 1.1)

• null (>= 3)

• random (>= 3)

• time (>= 0.9)

• utils (>= 1.6)

Providers

The following providers are used by this module:

• jwt (>= 1.1)

• time (>= 0.9)

• random (>= 3)

• argocd (>= 6)

• utils (>= 1.6)

• null (>= 3)

Resources

The following resources are used by this module:

• argocd_application.this (resource)

• argocd_project.this (resource)

• jwt_hashed_token.tokens (resource)

• null_resource.dependencies (resource)

• null_resource.this (resource)

• random_uuid.jti (resource)

• time_static.iat (resource)

• utils_deep_merge_yaml.values (data source)

Required Inputs

The following input variables are required:

cluster_name

Description: Name given to the cluster. Value used for the ingress' URL of the application.

Type: string

base_domain

Description: Base domain of the cluster. Value used for the ingress' URL of the application.

Type: string

accounts_pipeline_tokens

Description: API token for pipeline account.

Type: string

server_secretkey

Description: Signature key for session validation. Must reuse the bootstrap output containing the secretkey.

Type: string

Optional Inputs

The following input variables are optional (have default values):

subdomain

Description: Subdomain of the cluster. Value used for the ingress' URL of the application.

Type: string

Default: "apps"

argocd_project

Description: Name of the Argo CD AppProject where the Application should be created. If not set, the Application will be created in a new AppProject only for this Application.

Type: string

Default: null

argocd_labels

Description: Labels to attach to the Argo CD Application resource.

Type: map(string)

Default: {}

target_revision

Description: Override of target revision of the application chart.

Type: string

Default: "v7.1.0"

cluster_issuer

Description: SSL certificate issuer to use. Usually you would configure this value as letsencrypt-staging or letsencrypt-prod on your root *.tf files. You can use ca-issuer when using the self-signed variant of cert-manager.

Type: string

Default: "selfsigned-issuer"

helm_values

Description: Helm chart value overrides. They should be passed as a list of HCL structures.

Type: any

Default: []

app_autosync

Description: Automated sync options for the Argo CD Application resource.

Type:

object({
    allow_empty = optional(bool)
    prune       = optional(bool)
    self_heal   = optional(bool)
  })

Default:

{
  "allow_empty": false,
  "prune": true,
  "self_heal": true
}

dependency_ids

Description: n/a

Type: map(string)

Default: {}

resources

Description: Resource limits and requests for Argo CD’s components. Follow the style on official documentation to understand the format of the values.

Note	The repo_server requests and limits will be applied to all the extra containers that are deployed with the argocd-repo-server component (each container has the same requests and limits as the main container, so it is cumulative).

Note	If you enable the HA mode using the high_availability variable, the values for Redis will be applied to the Redis HA chart instead of the default one.

Important

These are not production values. You should always adjust them to your needs.

Type:

object({

    application_set = optional(object({
      requests = optional(object({
        cpu    = optional(string, "100m")
        memory = optional(string, "128Mi")
      }), {})
      limits = optional(object({
        cpu    = optional(string)
        memory = optional(string)
      }), {})
    }), {})

    controller = optional(object({
      requests = optional(object({
        cpu    = optional(string, "500m")
        memory = optional(string, "512Mi")
      }), {})
      limits = optional(object({
        cpu    = optional(string)
        memory = optional(string)
      }), {})
    }), {})

    notifications = optional(object({
      requests = optional(object({
        cpu    = optional(string, "100m")
        memory = optional(string, "128Mi")
      }), {})
      limits = optional(object({
        cpu    = optional(string)
        memory = optional(string)
      }), {})
    }), {})

    repo_server = optional(object({
      requests = optional(object({
        cpu    = optional(string, "200m")
        memory = optional(string, "128Mi")
      }), {})
      limits = optional(object({
        cpu    = optional(string)
        memory = optional(string)
      }), {})
    }), {})

    kustomized_helm_cmp = optional(object({
      requests = optional(object({
        cpu    = optional(string, "100m")
        memory = optional(string, "128Mi")
      }), {})
      limits = optional(object({
        cpu    = optional(string)
        memory = optional(string)
      }), {})
    }), {})

    helmfile_cmp = optional(object({
      requests = optional(object({
        cpu    = optional(string, "100m")
        memory = optional(string, "128Mi")
      }), {})
      limits = optional(object({
        cpu    = optional(string)
        memory = optional(string)
      }), {})
    }), {})

    server = optional(object({
      requests = optional(object({
        cpu    = optional(string, "50m")
        memory = optional(string, "128Mi")
      }), {})
      limits = optional(object({
        cpu    = optional(string)
        memory = optional(string)
      }), {})
    }), {})

    redis = optional(object({
      requests = optional(object({
        cpu    = optional(string, "200m")
        memory = optional(string, "256Mi")
      }), {})
      limits = optional(object({
        cpu    = optional(string)
        memory = optional(string)
      }), {})
    }), {})

  })

Default: {}

high_availability

Description: Argo CD High Availability settings. By default, the HA is disabled.

To enable HA using the default replicas, simply set the value high_availability.enabled to true. This will deploy Argo CD in HA without autoscaling.

You can enable autoscaling of the argocd-server and argocd-repo-server components by setting the high_availability.server.autoscaling.enabled and high_availability.repo_server.autoscaling.enabled values to true. You can also configure the minimum and maximum replicas desired or leave the default values.

Important

Activating the HA mode automatically enables the Redis HA chart which requires at least 3 worker nodes, as this chart enforces Pods to run on separate nodes.

Note	Since this variable uses the optional argument to forcing the user to define all the values, there is a side effect you can pass any other bogus value and Terraform will accept it, but they won’t be used in the chart behind the module.

Type:

object({
    enabled = bool

    controller = optional(object({
      replicas = optional(number, 1)
    }), {})

    application_set = optional(object({
      replicas = optional(number, 2)
    }), {})

    server = optional(object({
      replicas = optional(number, 2)
      autoscaling = optional(object({
        enabled      = bool
        min_replicas = optional(number, 2)
        max_replicas = optional(number, 5)
        }), {
        enabled = false
      })
    }), {})

    repo_server = optional(object({
      replicas = optional(number, 2)
      autoscaling = optional(object({
        enabled      = bool
        min_replicas = optional(number, 2)
        max_replicas = optional(number, 5)
        }), {
        enabled = false
      })
    }), {})

  })

Default:

{
  "enabled": false
}

oidc

Description: OIDC settings for the log in to the Argo CD web interface.

Type: any

Default: null

rbac

Description: RBAC settings for the Argo CD users.

Type:

object({
    scopes         = optional(string, "[groups, cognito:groups, roles]")
    policy_default = optional(string, "")
    policy_csv = optional(string, <<-EOT
                                    g, pipeline, role:admin
                                    g, argocd-admin, role:admin
                                    g, devops-stack-admins, role:admin
                                  EOT
    )
  })

Default: {}

repositories

Description: List of repositories to add to Argo CD.

Type: map(map(string))

Default: {}

ssh_known_hosts

Description: List of SSH known hosts to add to Argo CD.

Check the official values.yaml to get the format to pass this value.

Important

If you set this variable, the default known hosts will be overridden by this value, so you might want to consider adding the ones you need here."

Type: string

Default: null

exec_enabled

Description: Flag to enable the web-based terminal on Argo CD. Do not forget to set the appropriate RBAC configuration to your users/groups.

Type: bool

Default: false

admin_enabled

Description: Flag to indicate whether to enable the administrator user.

Type: bool

Default: false

extra_accounts

Description: List of accounts for which tokens will be generated.

Type: list(string)

Default: []

repo_server_iam_role_arn

Description: IAM role ARN to associate with the argocd-repo-server ServiceAccount. This role can be used to give SOPS access to AWS KMS.

Type: string

Default: null

repo_server_azure_workload_identity_clientid

Description: Azure AD Workload Identity Client-ID to associate with argocd-repo-server. This role can be used to give SOPS access to a Key Vault.

Type: string

Default: null

repo_server_aadpodidbinding

Description: Azure AAD Pod Identity to associate with the argocd-repo-server Pod. This role can be used to give SOPS access to a Key Vault.

Type: string

Default: null

helmfile_cmp_version

Description: Version of the helmfile-cmp plugin.

Type: string

Default: "0.1.1"

helmfile_cmp_env_variables

Description: List of environment variables to attach to the helmfile-cmp plugin, usually used to pass authentication credentials. Use an explicit format or take the values from a Kubernetes secret.

Type:

list(object({
    name  = optional(string)
    value = optional(string)
    valueFrom = optional(object({
      secretKeyRef = optional(object({
        name = optional(string)
        key  = optional(string)
      }))
    }))
  }))

Default: []

Outputs

The following outputs are exported:

id

Description: ID to pass other modules in order to refer to this module as a dependency.

extra_tokens

Description: Map of extra accounts that were created and their tokens.

Reference in table format

Show tables

= Requirements

Name	Version
terraform	>= 1.2
argocd	>= 6
htpasswd	>= 1
jwt	>= 1.1
null	>= 3
random	>= 3
time	>= 0.9
utils	>= 1.6

= Providers

Name	Version
jwt	>= 1.1
time	>= 0.9
random	>= 3
argocd	>= 6
utils	>= 1.6
null	>= 3

= Resources

Name	Type
argocd_application.this	resource
argocd_project.this	resource
jwt_hashed_token.tokens	resource
null_resource.dependencies	resource
null_resource.this	resource
random_uuid.jti	resource
time_static.iat	resource
utils_deep_merge_yaml.values	data source

= Inputs

Name	Description	Type	Default	Required
cluster_name	Name given to the cluster. Value used for the ingress' URL of the application.	string	n/a	yes
base_domain	Base domain of the cluster. Value used for the ingress' URL of the application.	string	n/a	yes
subdomain	Subdomain of the cluster. Value used for the ingress' URL of the application.	string	"apps"	no
argocd_project	Name of the Argo CD AppProject where the Application should be created. If not set, the Application will be created in a new AppProject only for this Application.	string	null	no
argocd_labels	Labels to attach to the Argo CD Application resource.	map(string)	{}	no
target_revision	Override of target revision of the application chart.	string	"v7.1.0"	no
cluster_issuer	SSL certificate issuer to use. Usually you would configure this value as letsencrypt-staging or letsencrypt-prod on your root *.tf files. You can use ca-issuer when using the self-signed variant of cert-manager.	string	"selfsigned-issuer"	no
helm_values	Helm chart value overrides. They should be passed as a list of HCL structures.	any	[]	no
app_autosync	Automated sync options for the Argo CD Application resource.	object({ allow_empty = optional(bool) prune = optional(bool) self_heal = optional(bool) })	{ "allow_empty": false, "prune": true, "self_heal": true }	no
dependency_ids	n/a	map(string)	{}	no
resources	Resource limits and requests for Argo CD’s components. Follow the style on official documentation to understand the format of the values. Note The repo_server requests and limits will be applied to all the extra containers that are deployed with the argocd-repo-server component (each container has the same requests and limits as the main container, so it is cumulative). Note If you enable the HA mode using the high_availability variable, the values for Redis will be applied to the Redis HA chart instead of the default one. Important These are not production values. You should always adjust them to your needs.	object({ application_set = optional(object({ requests = optional(object({ cpu = optional(string, "100m") memory = optional(string, "128Mi") }), {}) limits = optional(object({ cpu = optional(string) memory = optional(string) }), {}) }), {}) controller = optional(object({ requests = optional(object({ cpu = optional(string, "500m") memory = optional(string, "512Mi") }), {}) limits = optional(object({ cpu = optional(string) memory = optional(string) }), {}) }), {}) notifications = optional(object({ requests = optional(object({ cpu = optional(string, "100m") memory = optional(string, "128Mi") }), {}) limits = optional(object({ cpu = optional(string) memory = optional(string) }), {}) }), {}) repo_server = optional(object({ requests = optional(object({ cpu = optional(string, "200m") memory = optional(string, "128Mi") }), {}) limits = optional(object({ cpu = optional(string) memory = optional(string) }), {}) }), {}) kustomized_helm_cmp = optional(object({ requests = optional(object({ cpu = optional(string, "100m") memory = optional(string, "128Mi") }), {}) limits = optional(object({ cpu = optional(string) memory = optional(string) }), {}) }), {}) helmfile_cmp = optional(object({ requests = optional(object({ cpu = optional(string, "100m") memory = optional(string, "128Mi") }), {}) limits = optional(object({ cpu = optional(string) memory = optional(string) }), {}) }), {}) server = optional(object({ requests = optional(object({ cpu = optional(string, "50m") memory = optional(string, "128Mi") }), {}) limits = optional(object({ cpu = optional(string) memory = optional(string) }), {}) }), {}) redis = optional(object({ requests = optional(object({ cpu = optional(string, "200m") memory = optional(string, "256Mi") }), {}) limits = optional(object({ cpu = optional(string) memory = optional(string) }), {}) }), {}) })	{}	no
high_availability	Argo CD High Availability settings. By default, the HA is disabled. To enable HA using the default replicas, simply set the value high_availability.enabled to true. This will deploy Argo CD in HA without autoscaling. You can enable autoscaling of the argocd-server and argocd-repo-server components by setting the high_availability.server.autoscaling.enabled and high_availability.repo_server.autoscaling.enabled values to true. You can also configure the minimum and maximum replicas desired or leave the default values. Important Activating the HA mode automatically enables the Redis HA chart which requires at least 3 worker nodes, as this chart enforces Pods to run on separate nodes. Note Since this variable uses the optional argument to forcing the user to define all the values, there is a side effect you can pass any other bogus value and Terraform will accept it, but they won’t be used in the chart behind the module.	object({ enabled = bool controller = optional(object({ replicas = optional(number, 1) }), {}) application_set = optional(object({ replicas = optional(number, 2) }), {}) server = optional(object({ replicas = optional(number, 2) autoscaling = optional(object({ enabled = bool min_replicas = optional(number, 2) max_replicas = optional(number, 5) }), { enabled = false }) }), {}) repo_server = optional(object({ replicas = optional(number, 2) autoscaling = optional(object({ enabled = bool min_replicas = optional(number, 2) max_replicas = optional(number, 5) }), { enabled = false }) }), {}) })	{ "enabled": false }	no
oidc	OIDC settings for the log in to the Argo CD web interface.	any	null	no
rbac	RBAC settings for the Argo CD users.	object({ scopes = optional(string, "[groups, cognito:groups, roles]") policy_default = optional(string, "") policy_csv = optional(string, <<-EOT g, pipeline, role:admin g, argocd-admin, role:admin g, devops-stack-admins, role:admin EOT ) })	{}	no
repositories	List of repositories to add to Argo CD.	map(map(string))	{}	no
ssh_known_hosts	List of SSH known hosts to add to Argo CD. Check the official values.yaml to get the format to pass this value. Important If you set this variable, the default known hosts will be overridden by this value, so you might want to consider adding the ones you need here."	string	null	no
exec_enabled	Flag to enable the web-based terminal on Argo CD. Do not forget to set the appropriate RBAC configuration to your users/groups.	bool	false	no
admin_enabled	Flag to indicate whether to enable the administrator user.	bool	false	no
accounts_pipeline_tokens	API token for pipeline account.	string	n/a	yes
server_secretkey	Signature key for session validation. Must reuse the bootstrap output containing the secretkey.	string	n/a	yes
extra_accounts	List of accounts for which tokens will be generated.	list(string)	[]	no
repo_server_iam_role_arn	IAM role ARN to associate with the argocd-repo-server ServiceAccount. This role can be used to give SOPS access to AWS KMS.	string	null	no
repo_server_azure_workload_identity_clientid	Azure AD Workload Identity Client-ID to associate with argocd-repo-server. This role can be used to give SOPS access to a Key Vault.	string	null	no
repo_server_aadpodidbinding	Azure AAD Pod Identity to associate with the argocd-repo-server Pod. This role can be used to give SOPS access to a Key Vault.	string	null	no
helmfile_cmp_version	Version of the helmfile-cmp plugin.	string	"0.1.1"	no
helmfile_cmp_env_variables	List of environment variables to attach to the helmfile-cmp plugin, usually used to pass authentication credentials. Use an explicit format or take the values from a Kubernetes secret.	list(object({ name = optional(string) value = optional(string) valueFrom = optional(object({ secretKeyRef = optional(object({ name = optional(string) key = optional(string) })) })) }))	[]	no

= Outputs

Name	Description
id	ID to pass other modules in order to refer to this module as a dependency.
extra_tokens	Map of extra accounts that were created and their tokens.