Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CodePipeline Artifact validation fails on valid use case #2549

Closed
ghidi opened this issue May 15, 2019 · 10 comments · Fixed by #2558
Assignees

Comments

@ghidi
Copy link

@ghidi ghidi commented May 15, 2019

Describe the bug
The validation mechanism for CodePipeline artifacts fails in certain cases that are valid.

Error: Validation failed with the following errors:
Artifact 'buildArtifact1' was used as input before being used as output
at Synthesizer.synthesize (/.../node_modules/@aws-cdk/cdk/lib/synthesis.js:22:23)
at App.run (/.../node_modules/@aws-cdk/cdk/lib/app.js:44:31)
at Object. (/.../bin/infra.js:8:5)
at Module._compile (internal/modules/cjs/loader.js:759:30)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:770:10)
at Module.load (internal/modules/cjs/loader.js:628:32)
at Function.Module._load (internal/modules/cjs/loader.js:555:12)
at Function.Module.runMain (internal/modules/cjs/loader.js:824:10)
at internal/main/run_main_module.js:17:11

To Reproduce
Create a pipeline that has the following stages and dependencies between actions:
buildAction1 has run order 1, input sourceArtifact1 and output buildArtifact1
buildAction2 has run order 2, input sourceArtifact2, extra inputs buildArtifact1 and output buildArtifact2

stages: [
{
name: 'Source',
actions: [sourceAction1, sourceAction2]
},
{
name: 'Build',
actions: [buildAction1, buildAction2]
},
{
....
}
]

Expected behavior
The validation should pass. This is a valid pipeline that worked correctly before codepipeline.Artifact construct was introduced.

Version:

  • OS MacOS 10.13
  • Programming Language TypeScript
  • CDK Version 0.31.0
@ghidi ghidi added the bug label May 15, 2019
@skinny85 skinny85 self-assigned this May 16, 2019
@skinny85

This comment has been minimized.

Copy link
Contributor

@skinny85 skinny85 commented May 16, 2019

Hey @ghidi ,

thanks for reporting this. I confirm I was able to reproduce the issue locally, and it is indeed a bug. I'll be working on a fix today.

Thanks,
Adam

@skinny85

This comment has been minimized.

Copy link
Contributor

@skinny85 skinny85 commented May 16, 2019

PR with the fix submitted here: #2558

@skinny85

This comment has been minimized.

Copy link
Contributor

@skinny85 skinny85 commented May 16, 2019

BTW, until we release the fix in the linked PR, you can work around the issue by moving buildAction2 to a new Stage after Build, something like:

new codepipeline.Pipeline(this, 'Pipeline', {
  stages: [
    {
      name: 'Source',
      actions: [sourceAction1, sourceAction2],
    },
    {
      name: 'Build',
      actions: [buildAction1],
    },
    {
      name: 'Build_temp',
      actions: [buildAction2],
    },
    {
      // ....
    }
  ],
  // ...
});
skinny85 added a commit that referenced this issue May 16, 2019
…e same Stage. (#2558)

The bug was: we were incorrectly iterating though the Actions of a given Stage. First we were iterating through all of the Actions looking at their inputs, and then we looped through the Actions again to look at the outputs. But that won't work if the input of an Action is the output of a different Action in the same Stage with a lower runOrder. The fix is to iterate through all of the inputs of a given Action, and then the outputs, and only then move on to the next Action in increasing runOrder sequence in the same Stage.

Fixes #2549
@stefanfreitag

This comment has been minimized.

Copy link

@stefanfreitag stefanfreitag commented May 18, 2019

@skinny85 I was running into this issue and the workaround helped me out. But I still have this issue when trying to setup two code pipelines in one stack with v.0.31.0 of aws-cdk. If required I can provide more details on my setup.

@skinny85

This comment has been minimized.

Copy link
Contributor

@skinny85 skinny85 commented May 19, 2019

@stefanfreitag please do.

@stefanfreitag

This comment has been minimized.

Copy link

@stefanfreitag stefanfreitag commented May 19, 2019

@skinny85 I tried to reduce the noise in my code a bit and uploaded it here https://gist.github.com/stefanfreitag/ab2c8a905d1f42ad210ca396f7452f83

@skinny85

This comment has been minimized.

Copy link
Contributor

@skinny85 skinny85 commented May 22, 2019

@stefanfreitag are you using the same Artifact in both of your Pipelines? (What I mean, re-using the same Artifact object in the 2 Pipelines?)

@stefanfreitag

This comment has been minimized.

Copy link

@stefanfreitag stefanfreitag commented May 22, 2019

@skinny85 I updated the gist by attaching the buildspec I used in the last stage/ action of the first pipeline.

To answer you question:
To my understanding yes. If you look at lines 42 and 43 of the gist there are the calls to the two methods for setting up the pipelines. I pass the same artifact "imageDefinitionArtifact" to both methods.

  • In the first pipeline I use the artifact to store the output of the createContainerAction (line 92). Based on the attached buildspec_container.yml the artifact contains the imagedefinitions.json file.
  • In the second pipeline I need the imagedefinitions.json for the ECSDeployAction (line 147), so I use the artifact as input here.
@skinny85

This comment has been minimized.

Copy link
Contributor

@skinny85 skinny85 commented May 22, 2019

That's not really supported in CodePipeline. You can't share artifacts between Pipelines. So I believe the validation error you've received is correct, and not related to the bug fixed in this issue.

@stefanfreitag

This comment has been minimized.

Copy link

@stefanfreitag stefanfreitag commented May 23, 2019

@skinny85 Thanks for looking to this! I will then try to handle this differently.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.