Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support cross-account context queries from bootstrap stack #8905

Closed
rix0rrr opened this issue Jul 6, 2020 · 16 comments · Fixed by #14874
Closed

Support cross-account context queries from bootstrap stack #8905

rix0rrr opened this issue Jul 6, 2020 · 16 comments · Fixed by #14874
Assignees
Labels
@aws-cdk/core Related to core CDK functionality @aws-cdk/pipelines CDK Pipelines library effort/large Large work item – several weeks of effort feature-request A feature should be added or improved. p1

Comments

@rix0rrr
Copy link
Contributor

rix0rrr commented Jul 6, 2020

See title.


This is a 🚀 Feature Request

@rix0rrr rix0rrr added @aws-cdk/core Related to core CDK functionality feature-request A feature should be added or improved. needs-triage This issue or PR still needs to be triaged. @aws-cdk/pipelines CDK Pipelines library labels Jul 6, 2020
@SomayaB SomayaB removed the needs-triage This issue or PR still needs to be triaged. label Jul 7, 2020
@eladb eladb assigned rix0rrr and unassigned eladb Jul 7, 2020
@rix0rrr rix0rrr added the effort/large Large work item – several weeks of effort label Jul 8, 2020
@SomayaB SomayaB assigned ericzbeard and unassigned rix0rrr Jul 14, 2020
@ericzbeard ericzbeard assigned rix0rrr and unassigned ericzbeard Jul 14, 2020
@rix0rrr rix0rrr added this to the [CDK Pipelines] Soon milestone Aug 12, 2020
@rix0rrr rix0rrr added the p1 label Aug 12, 2020
@dnascimento
Copy link

@rix0rrr how would you address this issue? What's the best way to fix this? Happy to contribute

@ColeMurray
Copy link

Is there additional documentation on work needed to support this feature?

@blimmer
Copy link
Contributor

blimmer commented Nov 13, 2020

There's a good thread in the cdk.dev slack about this. It appears that a workaround is to populate the VPC IDs in a cdk.context.json file.

@agalazis
Copy link

so one fix is to have a script that populates cdk.context.json independently ?

@hoegertn
Copy link
Contributor

You can synth locally once and then commit the file.

@agalazis
Copy link

agalazis commented Dec 30, 2020

yep that's what I thought xD. Despite seeing many projects having context file in .gitignore , committing it is the recommended approach :

Don't forget to add the cdk.context.json file to your source control repository 
to ensure that subsequent synth commands will return the same result, 
and that your AWS account won't be needed when synthesizing from your build system.

from
https://docs.aws.amazon.com/cdk/latest/guide/context.html

still not only about running synth I will add a script that

  • switches off the flag
  • iterates though my environments and does synth
  • switches on the flag
  • deserves a script entry in package.json

@rix0rrr rix0rrr changed the title Use context queries with new bootstrap stack Support cross-account context queries from bootstrap stack Jan 7, 2021
@sean-kennedy-songtradr
Copy link

Hi, I have come across this problem while building a cdk pipeline, I think a solution/workaround may to to give the codebuild role or whatever role is performing the synth action permissions to use describe or list actions for whatever resource it needs such as vpc or listeners etc and then it can successfully synth with all the required context information.

Does this seem like a valid solution/workaround? I only discovered this today and will attempt to validate my findings tomorrow.

Sorry if this is the wrong place to mention it. Can give more information about my scenario if needed.

@skinny85
Copy link
Contributor

It's not enough @sean-kennedy-songtradr , because the cdk synth build may happen in a different account/region than the stacks that need the context values are in.

@skinny85
Copy link
Contributor

The best workaround is:

  1. Do the synth locally with the correct credentials (read-only should be fine).
  2. Commit and push the cdk.context.json file that gets generated this way.

@sean-kennedy-songtradr
Copy link

It's not enough @sean-kennedy-songtradr , because the cdk synth build may happen in a different account/region than the stacks that need the context values are in.

In my scenario (as far as I can tell) I am always synthesizing in the same account, that synth will require some ssm parameters from different accounts/regions and then use them in .fromLookup calls, I am using the CDK credential plugin so that cdk can assume the relevant role when it needs to get those parameters and make context queries.

If I can make sure the build always happens in the same account and region is this a valid method of supporting context queries right now or would you say it's a bit hacky for lack of a better term?

@skinny85
Copy link
Contributor

skinny85 commented Feb 18, 2021

I wouldn't use the word "hacky" necessarily, but I would say it's not a practice we want to encourage exactly. The reason is that we want to separate synthesizing your CDK application, which might mean running code from arbitrary third-party dependencies that you might not want to trust completely, from deploying it. If you give your build step permissions to read from your account, the third party dependencies will have those permissions as well.

Perhaps you consider that risk acceptable, in which case, go right ahead 🙂.

@greg-aws
Copy link
Contributor

@skinny85
This issue has kind of been a thorn in my side lately, so I wanted to make sure I understand it. It looks like context queries are currently done with whatever base credentials are present rather than assuming a role from Stack's account. Does it make sense to add an additional context lookup role to the bootstrap /w the appropriate read permissions and when an SDK is retrieved via sdk-provider.ts the assumeRoleArn is provided in CredentialOptions?

@skinny85
Copy link
Contributor

@rix0rrr should be able to answer that question better than me.

Rico, would you mind answering @greg-aws's question? Does new-style synthesis come into play here?

nayaran added a commit to nayaran/cdk-repro that referenced this issue May 22, 2021
Error message on running `cdk synth` - Error at /CdkReproPipeline] Could not find any VPCs matching {"account":DEV_ACCOUNT_ID","region":"REGION","filter":{"vpc-id":"PROD_VPC_ID"},"returnAsymmetricSubnets":true}

This seems to look for prod Vpc in my dev account (most likely because of line#61 where I am doing ec2.Vpc.from_lookup(self, 'VpcPrd', vpc_id=PROD_VPC_ID). Why? not clear to me but could be related to - aws/aws-cdk#8905 but will log a issue in CDK repo for the experts to look at.
@redbaron
Copy link
Contributor

@greg-aws, something like cdk-hnb659fds-lookup-role-{ACCOUNT_ID}-{REGION} with read only view access in the target account?

@greg-aws
Copy link
Contributor

@greg-aws, something like cdk-hnb659fds-lookup-role-{ACCOUNT_ID}-{REGION} with read only view access in the target account?

Exactly, in terms of the actual policy I would want some input on. We could be fairly fine grained to only allow access to the current actions required or it would be a more broad read policy which would leave room for future features.

@rix0rrr rix0rrr assigned otaviomacedo and unassigned rix0rrr Jun 1, 2021
@mergify mergify bot closed this as completed in #14874 Jun 3, 2021
mergify bot pushed a commit that referenced this issue Jun 3, 2021
Fixes #8905

----

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
@github-actions
Copy link

github-actions bot commented Jun 3, 2021

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

hollanddd pushed a commit to hollanddd/aws-cdk that referenced this issue Aug 26, 2021
Fixes aws#8905

----

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
@aws-cdk/core Related to core CDK functionality @aws-cdk/pipelines CDK Pipelines library effort/large Large work item – several weeks of effort feature-request A feature should be added or improved. p1
Projects
None yet
Development

Successfully merging a pull request may close this issue.