Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unable to continue with install: existing resource conflict when creating multiple Helm charts #7592

Closed
arjanschaaf opened this issue Apr 24, 2020 · 2 comments
Assignees

Comments

@arjanschaaf
Copy link

@arjanschaaf arjanschaaf commented Apr 24, 2020

When adding multiple Helm charts to an EKS cluster I get an error. The error is probably triggered because the helm install actions are performed in parallel: I install the same Helm chart twice with a different configuration and both installs start by trying to create the same Kubernetes CustomResourceDefinition's.
I can't find any documentation or examples on how to force a sequential installation of these Helm charts using the CDK.

Reproduction Steps

  1. Add two Helm charts to an EKS cluster in the CDK with the following snippet:
    // install external facing NGINX Ingress Controller
    cluster.addChart('external-ingress-controller', {
      chart: 'nginx-ingress',
      namespace: 'ingress-controller',
      release: 'external-ingress-controller',
      repository: 'https://helm.nginx.com/stable',
      values: {
        "controller": {
          "config": {
            "proxy-body-size": "512m"
          },
          "metrics": {
            "enabled": true,
            "service": {
              "type": "NodePort"
            }
          },
          "replicaCount": 1,
          "service": {
            "externalTrafficPolicy": "Local",
            "type": "LoadBalancer"
          }
        }
      }
    });
    // install internal facing NGINX Ingress Controller
    cluster.addChart('internal-ingress-controller', {
      chart: 'nginx-ingress',
      namespace: 'ingress-controller',
      release: 'internal-ingress-controller',
      repository: 'https://helm.nginx.com/stable',
      values: {
        "controller": {
          "config": {
            "proxy-body-size": "512m"
          },
          "ingressClass": "internal-nginx",
          "metrics": {
            "enabled": true,
            "service": {
              "type": "NodePort"
            }
          },
          "replicaCount": 1,
          "service": {
            "annotations": {
              "service.beta.kubernetes.io/aws-load-balancer-internal": "0.0.0.0/0"
            },
            "externalTrafficPolicy": "Local",
            "type": "LoadBalancer"
          }
        }
      }
    });

Error Log

  4/11 | 11:50:20 AM | CREATE_FAILED        | Custom::AWSCDK-EKS-HelmChart                | eks-cloud-infra/chart-internal-ingress-controller/Resource/Default (ekscloudinfrachartinternalingresscontroller6054650E) Failed to create resource. Error: b'Release "internal-ingress-controller" does not exist. Installing it now.\nError: rendered manifests contain a resource that already exists. Unable to continue with install: existing resource conflict: kind: CustomResourceDefinition, namespace: , name: virtualservers.k8s.nginx.org\n'
    at invokeUserFunction (/var/task/framework.js:85:19)
    at process._tickCallback (internal/process/next_tick.js:68:7)
	new CustomResource (~/git/cloud-infrastructure/node_modules/@aws-cdk/aws-cloudformation/lib/custom-resource.ts:163:21)
	\_ new HelmChart (~/git/cloud-infrastructure/node_modules/@aws-cdk/aws-eks/lib/helm-chart.ts:83:5)
	\_ Cluster.addChart (~/git/cloud-infrastructure/node_modules/@aws-cdk/aws-eks/lib/cluster.ts:650:12)
	\_ new CloudInfrastructureStack (~/git/cloud-infrastructure/lib/cloud-infrastructure-stack.ts:114:13)
	\_ Object.<anonymous> (~/git/cloud-infrastructure/bin/cloud-infrastructure.ts:7:1)
	\_ Module._compile (internal/modules/cjs/loader.js:1123:30)
	\_ Module.m._compile (~/git/cloud-infrastructure/node_modules/ts-node/src/index.ts:839:23)
	\_ Module._extensions..js (internal/modules/cjs/loader.js:1143:10)
	\_ Object.require.extensions.<computed> [as .ts] (~/git/cloud-infrastructure/node_modules/ts-node/src/index.ts:842:12)
	\_ Module.load (internal/modules/cjs/loader.js:972:32)
	\_ Function.Module._load (internal/modules/cjs/loader.js:872:14)
	\_ Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:71:12)
	\_ main (~/git/cloud-infrastructure/node_modules/ts-node/src/bin.ts:227:14)
	\_ Object.<anonymous> (~/git/cloud-infrastructure/node_modules/ts-node/src/bin.ts:512:3)
	\_ Module._compile (internal/modules/cjs/loader.js:1123:30)
	\_ Object.Module._extensions..js (internal/modules/cjs/loader.js:1143:10)
	\_ Module.load (internal/modules/cjs/loader.js:972:32)
	\_ Function.Module._load (internal/modules/cjs/loader.js:872:14)
	\_ Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:71:12)
	\_ /usr/local/lib/node_modules/npm/node_modules/libnpx/index.js:268:14

Environment

  • **CLI Version : aws-cli/2.0.5
  • **Framework Version: cdk 1.34.1
  • **OS : OSX
  • **Language : Typescript

Other


This is 🐛 Bug Report

@arjanschaaf
Copy link
Author

@arjanschaaf arjanschaaf commented Apr 26, 2020

I worked around this by declaring a dependency between these two actions:

    const externalIngressController = new eks.HelmChart(this, 'external-ingress-controller', {
      chart: 'nginx-ingress',
      cluster: cluster,
      namespace: 'ingress-controller',
      release: 'external-ingress-controller',
      repository: 'https://kubernetes-charts.storage.googleapis.com',
      values: {
        "controller": {
          "config": {
            "proxy-body-size": "512m"
          },
          "metrics": {
            "enabled": true,
            "service": {
              "type": "NodePort"
            }
          },
          "replicaCount": 1,
          "service": {
            "externalTrafficPolicy": "Local",
            "type": "LoadBalancer"
          }
        },
        wait: true
      }
    });

    const internalIngressController = new eks.HelmChart(this, 'internal-ingress-controller', {
      chart: 'nginx-ingress',
      cluster: cluster,
      namespace: 'ingress-controller',
      release: 'internal-ingress-controller',
      repository: 'https://kubernetes-charts.storage.googleapis.com',
      values: {
        "controller": {
          "config": {
            "proxy-body-size": "512m"
          },
          "ingressClass": "internal-nginx",
          "metrics": {
            "enabled": true,
            "service": {
              "type": "NodePort"
            }
          },
          "replicaCount": 1,
          "service": {
            "annotations": {
              "service.beta.kubernetes.io/aws-load-balancer-internal": "0.0.0.0/0"
            },
            "externalTrafficPolicy": "Local",
            "type": "LoadBalancer"
          }
        },
        wait: true
      }
    });
    // declare a dependency between the two helm charts
    internalIngressController.node.addDependency(externalIngressController);

I think it would be very helpful to document how to add dependencies using the CDK, it is hard to find documentation on this topic right now

@eladb
Copy link
Contributor

@eladb eladb commented Jun 24, 2020

We will add some documentation about dependencies as part of #4291

@eladb eladb closed this Jun 24, 2020
mergify bot pushed a commit that referenced this issue Jul 1, 2020
…ts (#8780)

- Added README content describing how to define deps for resources and helm charts.
- Updated integ test to define a k8s namespace and install nginx inside it.

Fixes #7592
Fixes #6806


----

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

3 participants
You can’t perform that action at this time.