Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multiple S3SourceActions returns "already a Construct with name" error #4237

Closed
joekiller opened this issue Sep 25, 2019 · 13 comments · Fixed by #4481
Closed

Multiple S3SourceActions returns "already a Construct with name" error #4237

joekiller opened this issue Sep 25, 2019 · 13 comments · Fixed by #4481
Assignees
Labels
@aws-cdk/aws-codepipeline Related to AWS CodePipeline bug This issue is a bug.

Comments

@joekiller
Copy link

When adding more than one S3SourceAction to a pipeline I get a duplicate Construct name error.

jsii.errors.JSIIError: There is already a Construct with name 'joetestintegrationpipelinePipeline5832AB7ESourceEventRule' in Bucket [Bucket]

Reproduction Steps

from aws_cdk import (
    aws_codepipeline_actions as actions,
    aws_codepipeline as pipe,
    aws_s3 as s3,
    core
)

class MyStack(core.Stack):
    def __init__(self, scope: core.Construct, id: str, **kwargs) -> None:
        super().__init__(scope, id, **kwargs)

        pipeline = pipe.Pipeline(self, "Pipeline")
        pipe_bucket = s3.Bucket(self, "Bucket", versioned=True)
        action1 = actions.S3SourceAction(
            bucket=pipe_bucket,
            bucket_key='test1',
            output=pipe.Artifact(),
            action_name='test1',
            trigger=actions.S3Trigger.EVENTS
        )
        action2 = actions.S3SourceAction(
            bucket=pipe_bucket,
            bucket_key='test2',
            output=pipe.Artifact(),
            action_name='test2',
            trigger=actions.S3Trigger.EVENTS
        )

        source_stage = pipeline.add_stage(
            stage_name='Source',
            actions=[action1, action2])

Error Log

jsii.errors.JavaScriptError: 
  Error: There is already a Construct with name 'joetestintegrationpipelinePipeline5832AB7ESourceEventRule' in Bucket [Bucket]
      at ConstructNode.addChild (/tmp/jsii-kernel-iAn6en/node_modules/@aws-cdk/core/lib/construct.js:420:19)
      at new ConstructNode (/tmp/jsii-kernel-iAn6en/node_modules/@aws-cdk/core/lib/construct.js:33:24)
      at new Construct (/tmp/jsii-kernel-iAn6en/node_modules/@aws-cdk/core/lib/construct.js:460:21)
      at new Resource (/tmp/jsii-kernel-iAn6en/node_modules/@aws-cdk/core/lib/resource.js:13:9)
      at new Rule (/tmp/jsii-kernel-iAn6en/node_modules/@aws-cdk/aws-events/lib/rule.js:14:9)
      at Bucket.onCloudTrailEvent (/tmp/jsii-kernel-iAn6en/node_modules/@aws-cdk/aws-s3/lib/bucket.js:49:22)
      at Bucket.onCloudTrailPutObject (/tmp/jsii-kernel-iAn6en/node_modules/@aws-cdk/aws-s3/lib/bucket.js:73:27)
      at S3SourceAction.bound (/tmp/jsii-kernel-iAn6en/node_modules/@aws-cdk/aws-codepipeline-actions/lib/s3/source-action.js:43:31)
      at S3SourceAction.bind (/tmp/jsii-kernel-iAn6en/node_modules/@aws-cdk/aws-codepipeline-actions/lib/action.js:23:21)
      at Kernel._wrapSandboxCode (/home/jlawson/src/codecommit/pipeline-cdk/.env/lib/python3.7/site-packages/jsii/_embedded/jsii/jsii-runtime.js:7253:19)
      at /home/jlawson/src/codecommit/pipeline-cdk/.env/lib/python3.7/site-packages/jsii/_embedded/jsii/jsii-runtime.js:6631:25
      at Kernel._ensureSync (/home/jlawson/src/codecommit/pipeline-cdk/.env/lib/python3.7/site-packages/jsii/_embedded/jsii/jsii-runtime.js:7226:20)
      at Kernel.invoke (/home/jlawson/src/codecommit/pipeline-cdk/.env/lib/python3.7/site-packages/jsii/_embedded/jsii/jsii-runtime.js:6630:26)
      at KernelHost.processRequest (/home/jlawson/src/codecommit/pipeline-cdk/.env/lib/python3.7/site-packages/jsii/_embedded/jsii/jsii-runtime.js:6327:28)
      at KernelHost.run (/home/jlawson/src/codecommit/pipeline-cdk/.env/lib/python3.7/site-packages/jsii/_embedded/jsii/jsii-runtime.js:6270:14)
      at Immediate._onImmediate (/home/jlawson/src/codecommit/pipeline-cdk/.env/lib/python3.7/site-packages/jsii/_embedded/jsii/jsii-runtime.js:6273:37)

Environment

  • CLI Version : 1.9.0 (build 30f158a)
  • Framework Version: 1.9.0
  • OS : Arch Linux
  • Language : Python 3.7

Other

N/A


This is 🐛 Bug Report

@joekiller joekiller added bug This issue is a bug. needs-triage This issue or PR still needs to be triaged. labels Sep 25, 2019
@joekiller
Copy link
Author

This appears to be an issue with trying to use S3Trigger.EVENTS.

this.props.bucket.onCloudTrailPutObject(stage.pipeline.node.uniqueId + 'SourceEventRule', {

@skinny85 skinny85 self-assigned this Sep 25, 2019
@skinny85 skinny85 added @aws-cdk/aws-codepipeline Related to AWS CodePipeline and removed needs-triage This issue or PR still needs to be triaged. labels Sep 25, 2019
@skinny85
Copy link
Contributor

skinny85 commented Sep 25, 2019

Thanks for opening the issue @joekiller . You're correct; a quick way to unblock yourself is to change one the of those actions.S3Trigger.EVENTS to actions.S3Trigger.POLL.

I'll see what I can do here - the edge case here is that you're using the same bucket twice in the pipeline, with different paths, which we didn't anticipate.

@skyrpex
Copy link
Contributor

skyrpex commented Sep 27, 2019

I can reproduce this same error on a different context: I'm reusing lambdas when assigning lambdaFunctionAssociations to different cloudfront.CloudFrontWebDistributions. The thing is that I have two distributions that share a few lambdas. The lambdas live on a stack, and the two cloudfront distributions are on separate stacks as well.

I can provide a repro example if necessary.

BTW: My current workaround is to just remove the shared lambda stack and declare each lambda within the cloudformation stacks.

@skinny85
Copy link
Contributor

skinny85 commented Sep 30, 2019

I can provide a repro example if necessary.

That would be helpful @skyrpex .

@skyrpex
Copy link
Contributor

skyrpex commented Sep 30, 2019

I managed to move it into a single file. All the cdk commands throw error, but I found cdk synth to be the fastest one throwing it.

In this case, there's a SharedLambdas stack which contains a lambda used in the other two PWA stacks, but I guess it'll throw the same error reusing the same lambda in multiple distributions on the same stack.

import * as cdk from "@aws-cdk/core";
import * as s3 from "@aws-cdk/aws-s3";
import * as lambda from "@aws-cdk/aws-lambda";
import * as cloudfront from "@aws-cdk/aws-cloudfront";

class SharedPwaLambdasStack extends cdk.Stack {
  addCspHeadersFunction: lambda.Function;

  constructor(scope: cdk.Construct, name: string, props?: cdk.StackProps) {
    super(scope, name, props);

    this.addCspHeadersFunction = new lambda.Function(this, "AddCspHeaders", {
      runtime: lambda.Runtime.NODEJS_8_10,
      handler: "index.handler",
      code: lambda.Code.fromInline("irrelevant"),
    });
  }
}

interface PwaStackProps extends cdk.StackProps {
  addCspHeadersFunction: lambda.Function;
}

class PwaStack extends cdk.Stack {
  constructor(scope: cdk.Construct, name: string, props: PwaStackProps) {
    super(scope, name, props);

    const bucket = new s3.Bucket(this, `${name}Bucket`);

    const distribution = new cloudfront.CloudFrontWebDistribution(this, `${name}Distribution`, {
      originConfigs: [
        {
          s3OriginSource: {
            s3BucketSource: bucket,
          },
          behaviors: [
            {
              isDefaultBehavior: true,
              lambdaFunctionAssociations: [
                {
                  eventType: cloudfront.LambdaEdgeEventType.VIEWER_RESPONSE,
                  lambdaFunction: props.addCspHeadersFunction.latestVersion,
                },
              ],
            },
          ],
        },
      ],
    });
  }
}

const app = new cdk.App();

const sharedPwaLambdasStack = new SharedPwaLambdasStack(app, "SharedPwaLambdas");

const websitePwa = new PwaStack(app, "WebsitePwa", {
  addCspHeadersFunction: sharedPwaLambdasStack.addCspHeadersFunction,
});

// If you comment the following stack, it will work.
const adminPwa = new PwaStack(app, "AdminPwa", {
  addCspHeadersFunction: sharedPwaLambdasStack.addCspHeadersFunction,
});

@skinny85
Copy link
Contributor

skinny85 commented Oct 2, 2019

And what is the error that you get? Because you don't share buckets in this case (each instance of PwaStack has its own bucket), so I'm surprised it would be the same error...

@skyrpex
Copy link
Contributor

skyrpex commented Oct 4, 2019

Oh sure, here it is: Error: There is already a Construct with name '$LATEST' in Function [AddCspHeaders]

Here's the full error stack.

cdk synth
yarn run v1.19.0
$ /code/node_modules/.bin/ts-node index.ts

/code/node_modules/@aws-cdk/core/lib/construct.ts:512
      throw new Error(`There is already a Construct with name '${childName}' in ${typeName}${name.length > 0 ? ' [' + name + ']' : ''}`);
            ^
Error: There is already a Construct with name '$LATEST' in Function [AddCspHeaders]
    at ConstructNode.addChild (/code/node_modules/@aws-cdk/core/lib/construct.ts:512:13)
    at new ConstructNode (/code/node_modules/@aws-cdk/core/lib/construct.ts:144:18)
    at new Construct (/code/node_modules/@aws-cdk/core/lib/construct.ts:563:17)
    at new Resource (/code/node_modules/@aws-cdk/core/lib/resource.ts:61:5)
    at new FunctionBase (/code/node_modules/@aws-cdk/aws-lambda/lib/function-base.ts:139:1)
    at new LatestVersion (/code/node_modules/@aws-cdk/aws-lambda/lib/function-base.ts:336:5)
    at Function.get latestVersion [as latestVersion] (/code/node_modules/@aws-cdk/aws-lambda/lib/function-base.ts:233:12)
    at new PwaStack (/code/cloud/index.ts:42:63)
    at Object.<anonymous> (/code/cloud/index.ts:62:18)
    at Module._compile (internal/modules/cjs/loader.js:936:30)
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

@skinny85
Copy link
Contributor

Got it. I think that's a completely different error - the message is the same, but the root cause (where is the duplication happening) is totally different in your case @skyrpex .

@skyrpex
Copy link
Contributor

skyrpex commented Oct 10, 2019

Then I'm so glad I gave you the reproduction code. Shall we create a new issue, then?

@skinny85
Copy link
Contributor

Sorry about that. I've opened a new issue, and copied your information there: #4459

skinny85 added a commit to skinny85/aws-cdk that referenced this issue Oct 11, 2019
…ket multiple times

If you attempted to add two source S3 actions using the same bucket into the same pipeline,
and both using CloudWatch events as triggers,
you would get an error about 'duplicate construct with id PipelineSourceEventRule'.
This is a valid use-case, though,
as each action might point to a different path in the bucket.

Fixes aws#4237
skinny85 added a commit to skinny85/aws-cdk that referenced this issue Oct 14, 2019
…ket multiple times

If you attempted to add two source S3 actions using the same bucket into the same pipeline,
and both using CloudWatch events as triggers,
you would get an error about 'duplicate construct with id PipelineSourceEventRule'.
This is a valid use-case, though,
as each action might point to a different path in the bucket.

Fixes aws#4237
@mergify mergify bot closed this as completed in #4481 Oct 15, 2019
mergify bot pushed a commit that referenced this issue Oct 15, 2019
…ket multiple times (#4481)

If you attempted to add two source S3 actions using the same bucket into the same pipeline,
and both using CloudWatch events as triggers,
you would get an error about 'duplicate construct with id PipelineSourceEventRule'.
This is a valid use-case, though,
as each action might point to a different path in the bucket.

Fixes #4237
@joekiller
Copy link
Author

Thanks @skinny85 I'll get an example for my other issue.

@joekiller
Copy link
Author

joekiller commented Oct 15, 2019

I can confirm this works now so thank you @skinny85 and @rix0rrr

@skinny85
Copy link
Contributor

NP @joekiller , glad it's working now!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
@aws-cdk/aws-codepipeline Related to AWS CodePipeline bug This issue is a bug.
Projects
None yet
3 participants