-
Notifications
You must be signed in to change notification settings - Fork 3.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
CodeCommit and CodePipeline in different stacks leads to circular dependency #8042
Comments
I've run into similar problems. This is a dangerous hole. I think it cost me ~6 months of kicking until it works. Any variables used in the pipeline stack, must only reference the pipeline stack and are not allowed to be assigned to the main App. I believe you want something closer to this:
|
@guywilsonjr This would probably work without any issue. But I'd rather try to avoid nested stacks. I also want to allow multiple pipelines (from different stacks) to access the same repository (e.g. separate pipelines for develop, staging, production - or - an infrastructure pipeline and a website pipeline). My feature request is more about supporting cross-references within the same region/account in a similar way than cross-references between different regions/accounts. |
I see. I think I messed it up anyways. As far as I know all Pipeline related resources must be bound to the same cdk.Stack. Though you can define all of the resources in any file you want.
if the repository is already created. Where 'Scope' is your pipeline stack. You should be able to use that in all of your different pipelines |
I've also hit this issue in the past few days with more or less the same configuration as in the original post while attempting to migrate a plain CloudFormation implementation that also had these resources split into separate stacks. I dug into the CDK code a bit to try to understand the root cause and one of the causes appears to be the auto-created IAM resources such as the statement created in CodeCommitSourceAction.bound() as well as the auto-created CW Event - these end up being dependencies of the repo. Agree with the comment by @asterikx that this is probably a generalizable issue. Anywhere that a construct makes changes to a resource that it does not directly own, there is potential for this to arise. There should be a clear rule enforced that constructs can only make changes to their own resources otherwise the dependencies will never be manageable. If you search for "cyclic" in the open issues you'll see a number of variants of this problem. In this case it's convenient that the Pipeline constructs try to auto-create policies for me but I'd rather it fail fast and tell me I have to compose in another construct that creates the role so that I have more control and more transparency and so we don't get phantom dependencies. |
I will add in complete agreement with @brendonmatheson I can honestly say I've spent at least 50 wasted hours(on the low end) on this problem. The true question is who is the most gracious and honorable typescript expert with time, energy, and enthusiasm to make it happen? 😉😉😜😜 |
Yeah ,this is a bug, in root_stack: new pipelineActions.CloudFormationCreateUpdateStackAction({
actionName: 'deploy_child_stack',
stackName: 'child_stack'
}), will be recognized as a dependency root_stack -> child_stack, throw Action:
- 'cloudformation:CreateStack'
....
- 'cloudformation:ValidateTemplate'
Effect: Allow
Resource: !Join
- ''
- - 'arn:'
- !Ref 'AWS::Partition'
- ':cloudformation:'
- !Ref 'AWS::Region'
- ':'
- !Ref 'AWS::AccountId'
- ':stack/child_stack/*'` then you can't reference from child_stack to root_stack anymore This is definitely a bug, because root_stack does NOT really depend on child_stack here! |
Ran into this bug myself. The easiest work around I've come up with is to create the CodeCommit repository using the AWS web console, and then import the CodeCommit repository directly in the same CDK stack that I use to create the pipeline. Interestingly, if you import the CodeCommit repo in the parent stack, the same cyclic reference error gets thrown. |
…n sources that use CloudWatch Events (#20149) When using a newly-created, CDK-managed resource, such as an S3 Bucket or a CodeCommit Repository, as the source of a CodePipeline, when it's in a different Stack than the pipeline (but in the same environment as it), and we use CloudWatch Events to trigger the pipeline from the source, that would result in a cycle: 1. Because the Event Rule is created in the source Stack, that Stack needs the name of the pipeline to trigger from the Rule, and also the Role to use for the trigger, which is created in the target (in this case, the pipeline) Stack. So, the source Stack depends on the pipeline Stack. 2. The pipeline Stack needs the name of the source resource. So, the pipeline Stack depends on the source Stack. The only way to break this cycle is to move the Event Rule to the target Stack. This PR adds an API to the Events module to make it possible for event-creating constructs to make that choice, and uses that capability in the CodePipeline `CodeCommitSourceAction` and `S3SourceAction`. Fixes #3087 Fixes #8042 Fixes #10896 ---- ### All Submissions: * [x] Have you followed the guidelines in our [Contributing guide?](https://github.com/aws/aws-cdk/blob/master/CONTRIBUTING.md) ### Adding new Unconventional Dependencies: * [ ] This PR adds new unconventional dependencies following the process described [here](https://github.com/aws/aws-cdk/blob/master/CONTRIBUTING.md/#adding-new-unconventional-dependencies) ### New Features * [ ] Have you added the new feature to an [integration test](https://github.com/aws/aws-cdk/blob/master/INTEGRATION_TESTS.md)? * [ ] Did you use `yarn integ` to deploy the infrastructure and generate the snapshot (i.e. `yarn integ` without `--dry-run`)? *By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
|
…n sources that use CloudWatch Events (aws#20149) When using a newly-created, CDK-managed resource, such as an S3 Bucket or a CodeCommit Repository, as the source of a CodePipeline, when it's in a different Stack than the pipeline (but in the same environment as it), and we use CloudWatch Events to trigger the pipeline from the source, that would result in a cycle: 1. Because the Event Rule is created in the source Stack, that Stack needs the name of the pipeline to trigger from the Rule, and also the Role to use for the trigger, which is created in the target (in this case, the pipeline) Stack. So, the source Stack depends on the pipeline Stack. 2. The pipeline Stack needs the name of the source resource. So, the pipeline Stack depends on the source Stack. The only way to break this cycle is to move the Event Rule to the target Stack. This PR adds an API to the Events module to make it possible for event-creating constructs to make that choice, and uses that capability in the CodePipeline `CodeCommitSourceAction` and `S3SourceAction`. Fixes aws#3087 Fixes aws#8042 Fixes aws#10896 ---- ### All Submissions: * [x] Have you followed the guidelines in our [Contributing guide?](https://github.com/aws/aws-cdk/blob/master/CONTRIBUTING.md) ### Adding new Unconventional Dependencies: * [ ] This PR adds new unconventional dependencies following the process described [here](https://github.com/aws/aws-cdk/blob/master/CONTRIBUTING.md/#adding-new-unconventional-dependencies) ### New Features * [ ] Have you added the new feature to an [integration test](https://github.com/aws/aws-cdk/blob/master/INTEGRATION_TESTS.md)? * [ ] Did you use `yarn integ` to deploy the infrastructure and generate the snapshot (i.e. `yarn integ` without `--dry-run`)? *By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
❓ General Issue
I want to have a CodeCommit repository in one stack triggering a CodePipeline in another stack, both stacks being deployed to the same region.
When passing the repo as a prop to the stack containing the pipeline, I get a circular dependency error:
'RepoStack' depends on 'PipelineStack' (RepoStack -> PipelineStack/Pipeline/Resource.Ref, RepoStack -> PipelineStack/Pipeline/EventsRole/Resource.Arn). Adding this dependency (PipelineStack -> RepoStack/Repository/Resource.Name) would create a cyclic reference.
My code looks as follows:
The Question
How can I have a CodeCommit repository in one stack triggering a CodePipeline in another stack without introducing a circular dependency (both in the same account/region)?
When the
RepoStack
is deployed to another account, everything works fine. I suspect that, in this case, the cross-account support stacks do the magic.Still, it comes to a surprise that support for cross-references across accounts is better than support for cross-references within the same account.
I think this problem can be generalized to many scenarios with circular dependencies.
Environment
The text was updated successfully, but these errors were encountered: