Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(core): environment-agnostic cloud assemblies #2922

Merged
merged 10 commits into from
Jun 19, 2019

Conversation

eladb
Copy link
Contributor

@eladb eladb commented Jun 19, 2019

Formalize the simple use case for synthesizing cloudformation templates that are not pre-associated with a specific AWS account/region.

When a CDK stack is defined without an explicit env configuration, or if env.account and/or env.region are set to Aws.accountId/Aws.region, the stack is said to be "environment-agnostic". This means that when a template is synthesized, we will use the CloudFormation intrinsics AWS::AccountId and AWS::Region instead of concrete account/region.

The cloud assembly manifest for such stacks will indicate aws://unknown-account/unknown region to represent that this stack is environment-agnostic, and tooling should rely on external configuration to determine the deployment environment.

Environment-agnostic stacks have limitations. For example, their resources cannot be referenced across accounts or regions, and context providers such as SSM, AZs, VPC and Route53 lookup cannot be used since they won't know which environment to query.

To faciliate the env-agnostic use case at the AWS Construct Library level, this change removes any dependency on concrete environment specification. Namely:

  • The AZ provider, which is now accessible through stack.availabilityZones will fall back to use [ Fn::GetAZs[0], Fn::GetAZs[1] ] in case the stack is env-agnostic. This is a safe fallback since all AWS regions have at least two AZs.
  • The use of the SSM context provider by the EC2 and ECS libraries to retrieve AMIs was replaced by deploy-time resolution of SSM parameters, so no fallback is required.

See list of breaking API changes below.

Added a few static methods to ssm.StringParameter to make it easier to reference values directly:

  • valueFromLookup will read a value during synthesis using the SSM context provider.
  • valueForStringParameter will return a deploy-time resolved value.
  • valueForSecureStringParameter will return a deploy-time resolved secure string value.

Fixes #2866

BREAKING CHANGE: ContextProvider is no longer designed to be extended. Use ContextProvider.getValue and ContextProvider.getKey as utilities.

  • core: Context.getSsmParameter has been removed. Use ssm.StringParameter.valueFromLookup
  • core: Context.getAvailabilityZones has been removed. Use stack.availabilityZones
  • core: Context.getDefaultAccount and getDefaultRegion have been removed an no longer available.
  • route52: HostedZoneProvider has been removed. Use HostedZone.fromLookup.
  • ec2: VpcNetworkProvider has been removed. Use Vpc.fromLookup.
  • ec2: ec2.MachineImage will now resolve AMIs from SSM during deployment.
  • ecs: ecs.EcsOptimizedAmi will now resolve AMis from SSM during deployment.

TODO:


Pull Request Checklist

  • Testing
    • Unit test added (prefer not to modify an existing test, otherwise, it's probably a breaking change)
    • CLI change?: coordinate update of integration tests with team
    • cdk-init template change?: coordinated update of integration tests with team
  • Docs
    • jsdocs: All public APIs documented
    • README: README and/or documentation topic updated
    • Design: For significant features, design document added to design folder
  • Title and Description
    • Change type: title prefixed with fix, feat and module name in parens, which will appear in changelog
    • Title: use lower-case and doesn't end with a period
    • Breaking?: last paragraph: "BREAKING CHANGE: <describe what changed + link for details>"
    • Issues: Indicate issues fixed via: "Fixes #xxx" or "Closes #xxx"
  • Sensitive Modules (requires 2 PR approvers)
    • IAM Policy Document (in @aws-cdk/aws-iam)
    • EC2 Security Groups and ACLs (in @aws-cdk/aws-ec2)
    • Grant APIs (only if not based on official documentation with a reference)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license.

Formalize the simple use case for synthesizing cloudformation templates that
are not pre-associated with a specific AWS account/region.

When a CDK stack is defined without an explicit `env` configuration, or if `env.account`
and/or `env.region` are set to `Aws.accountId`/`Aws.region`, the stack is said to be
"environment-agnostic". This means that when a template is synthesized, we will use
the CloudFormation intrinsics `AWS::AccountId` and `AWS::Region` instead of concrete
account/region.

The cloud assembly manifest for such stacks will indicate `aws://unknown-account/unknown-region`
to represent that this stack is environment-agnostic, and tooling should rely on external
configuration to determine the deployment environment.

Environment-agnostic stacks have limitations. For example, their resources cannot be referenced
across accounts or regions, and context providers such as SSM, AZs, VPC and Route53 lookup
cannot be used since they won't know which environment to query.

To faciliate the env-agnostic use case at the AWS Construct Library level,
this change removes any dependency on concrete environment specification. Namely:

- The AZ provider, which is now accessible through `stack.availabilityZones` will fall
  back to use `[ Fn::GetAZs[0], Fn::GetAZs[1] ]` in case the stack is env-agnostic. This is
  a safe fallback since all AWS regions have at least two AZs.
- The use of the SSM context provider by the EC2 and ECS libraries to retrieve AMIs was
  replaced by deploy-time resolution of SSM parameters, so no fallback is required.

See list of breaking API changes below.

Added a few static methods to `ssm.StringParameter` to make it easier to reference values directly:
* `valueFromLookup` will read a value during synthesis using the SSM context provider.
* `valueForStringParameter` will return a deploy-time resolved value.
* `valueForSecureStringParameter` will return a deploy-time resolved secure string value.

Fixes #2866

BREAKING CHANGE: `ContextProvider` is no longer designed to be extended. Use `ContextProvider.getValue` and `ContextProvider.getKey` as utilities.
* **core:** `Context.getSsmParameter` has been removed. Use `ssm.StringParameter.valueFromLookup`
* **core:** `Context.getAvailabilityZones` has been removed. Use `stack.availabilityZones`
* **core:** `Context.getDefaultAccount` and `getDefaultRegion` have been removed an no longer available.
* **route52:** `HostedZoneProvider` has been removed. Use `HostedZone.fromLookup`.
* **ec2:** `VpcNetworkProvider` has been removed. Use `Vpc.fromLookup`.
* **ec2:** `ec2.MachineImage` will now resolve AMIs from SSM during deployment.
* **ecs:** `ecs.EcsOptimizedAmi` will now resolve AMis from SSM during deployment.
@eladb eladb requested review from SoManyHs and a team as code owners June 19, 2019 11:59
packages/@aws-cdk/aws-ec2/lib/vpc-lookup.ts Show resolved Hide resolved
packages/@aws-cdk/cdk/lib/environment.ts Outdated Show resolved Hide resolved
packages/@aws-cdk/cdk/lib/stack.ts Show resolved Hide resolved
packages/@aws-cdk/cdk/lib/stack.ts Outdated Show resolved Hide resolved
@eladb eladb requested a review from skinny85 as a code owner June 19, 2019 17:48
@@ -58,6 +58,7 @@ class Test extends cdk.Stack {
name: 'Basic',
apiKey: key,
description: 'Free tier monthly usage plan',
throttle: { rateLimit: 5 },
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is actually required

@eladb eladb merged commit c75d245 into master Jun 19, 2019
@eladb eladb deleted the benisrae/environment-defaults-3 branch June 19, 2019 20:11
eladb pushed a commit that referenced this pull request Jun 20, 2019
following up on #2922, manifest may include "env" that indicates unknown-account/unknown-region,
which will cause old CLIs to fail, so we must bump the version to ensure compatibility.

BREAKING CHANGE: Previous versions of the CDK CLI will not be fully compatible with this version of the framework and vice versa.
eladb pushed a commit that referenced this pull request Jun 23, 2019
following up on #2922, manifest may include "env" that indicates unknown-account/unknown-region,
which will cause old CLIs to fail, so we must bump the version to ensure compatibility.

BREAKING CHANGE: Previous versions of the CDK CLI will not be fully compatible with this version of the framework and vice versa.
eladb pushed a commit that referenced this pull request Jun 30, 2019
…tacks

Following up on #2922: improve the error message emitted when trying to read context from an environment-agnostic stack by calling out CDK_DEFAULT_ACCOUNT and CDK_DEFAULT_REGION as a way to inherit environment information from the CLI.

Fixes #3078 
Fixes #3120 
Fixes #3130
@mhuebner mhuebner mentioned this pull request Jul 1, 2019
5 tasks
eladb pushed a commit that referenced this pull request Jul 1, 2019
…tacks (#3137)

Following up on #2922: improve the error message emitted when trying to read context from an environment-agnostic stack by calling out CDK_DEFAULT_ACCOUNT and CDK_DEFAULT_REGION as a way to inherit environment information from the CLI.

Fixes #3078
Fixes #3120
Fixes #3130
@mergify
Copy link
Contributor

mergify bot commented Sep 23, 2019

Thanks so much for taking the time to contribute to the AWS CDK ❤️

We will shortly assign someone to review this pull request and help get it
merged. In the meantime, please take a minute to make sure you follow this
checklist
:

  • PR title type(scope): text
    • type: fix, feat, refactor go into CHANGELOG, chore is hidden
    • scope: name of module without aws- or cdk- prefix or postfix (e.g. s3 instead of aws-s3-deployment)
    • text: use all lower-case, do not end with a period, do not include issue refs
  • PR Description
    • Rationale: describe rationale of change and approach taken
    • Issues: indicate issues fixed via: fixes #xxx or closes #xxx
    • Breaking?: last paragraph: BREAKING CHANGE: <describe what changed + link for details>
  • Testing
    • Unit test added. Prefer to add a new test rather than modify existing tests
    • CLI or init templates change? Re-run/add CLI integration tests
  • Documentation
    • README: update module README to describe new features
    • API docs: public APIs must be documented. Copy from official AWS docs when possible
    • Design: for significant features, follow design process

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
contribution/core This is a PR that came from AWS.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Environment defaults and support for relocatable templates
4 participants