Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Toolkit: run integration tests automatically #294

Closed
eladb opened this issue Jul 10, 2018 · 6 comments · Fixed by #1632
Closed

Toolkit: run integration tests automatically #294

eladb opened this issue Jul 10, 2018 · 6 comments · Fixed by #1632
Labels
ops-excellence Operational Excellence package/tools Related to AWS CDK Tools or CLI

Comments

@eladb
Copy link
Contributor

eladb commented Jul 10, 2018 •
edited

The toolkit has a bunch of integration tests that we currently execute manually. We need to run them per commit as part of the pipeline.
@eladb eladb added the ops-excellence Operational Excellence label Jul 10, 2018
@eladb eladb changed the title Toolkit: Missing unit test coverage Toolkit: Missing test coverage Oct 12, 2018
eladb pushed a commit that referenced this issue Oct 12, 2018
fix(toolkit): commands wouldn't work against the selected stacks (reg…
3427b72 
…ression)

Due to a recent cx protocol change (#868), some toolkit commands stopped
respecting the "selected" stacks (the ones specified in the command line).

"cdk synth" would always return the first stack, and "cdk deploy" would
always deploy all stacks.

Since we have test coverage gaps in the toolkit (#294), we did not
discover this before we released.

This change includes an initial set of integration tests for the 
toolkit. At the moment they should be manually executed when toolkit
changes are made, but we will execute them in a pipeline.

Fixes #910
@eladb eladb mentioned this issue Oct 12, 2018
Merged
RomainMuller pushed a commit that referenced this issue Oct 12, 2018
@RomainMuller
fix(toolkit): multi-stack apps cannot be synthesized or deployed (#911)
5511076 
Due to a recent cx protocol change (#868), some toolkit commands stopped
respecting the "selected" stacks (the ones specified in the command line).

"cdk synth" would always return the first stack, and "cdk deploy" would
always deploy all stacks.

Since we have test coverage gaps in the toolkit (#294), we did not
discover this before we released.

This change includes an initial set of integration tests for the 
toolkit. At the moment they should be manually executed when toolkit
changes are made, but we will execute them in a pipeline.

Fixes #910
RomainMuller added a commit that referenced this issue Oct 12, 2018
@RomainMuller
v0.12.0
09c85f3 
* **aws-codebuild:** allow passing oauth token to GitHubEnterpriseSource ([#908](#908)) ([c23da91](c23da91))
* **toolkit:** multi-stack apps cannot be synthesized or deployed ([#911](#911)) ([5511076](5511076)), closes [#868](#868) [#294](#294) [#910](#910)

* **aws-cloudformation:** add permission management to CreateUpdate and Delete Stack CodePipeline Actions. ([#880](#880)) ([8b3ae43](8b3ae43))
* **aws-codepipeline:** make input and output artifact names optional when creating Actions. ([#845](#845)) ([3d91c93](3d91c93))

* **aws-codepipeline:** this commit contains the following breaking changes:
* Rename 'artifactName' in Action construction properties to 'outputArtifactName'
* Rename the 'artifact' property of Actions to 'outputArtifact'
* No longer allow adding output artifacts to Actions by instantiating the Artifact class
* Rename Action#input/outputArtifacts properties to _input/_outputArtifacts

Previously, we always required customers to explicitly name the output artifacts the Actions used in the Pipeline,
and to explicitly "wire together" the outputs of one Action as inputs to another.
With this change, the CodePipeline Construct generates artifact names,
if the customer didn't provide one explicitly,
and tries to find the first available output artifact to use as input to a newly created Action that needs it,
thus turning both the input and output artifacts from required to optional properties.
@RomainMuller RomainMuller mentioned this issue Oct 12, 2018
Merged
RomainMuller added a commit that referenced this issue Oct 12, 2018
@RomainMuller
v0.12.0 (#912)
2dc324e 
* **aws-codebuild:** allow passing oauth token to GitHubEnterpriseSource ([#908](#908)) ([c23da91](c23da91))
* **toolkit:** multi-stack apps cannot be synthesized or deployed ([#911](#911)) ([5511076](5511076)), closes [#868](#868) [#294](#294) [#910](#910)

* **aws-cloudformation:** add permission management to CreateUpdate and Delete Stack CodePipeline Actions. ([#880](#880)) ([8b3ae43](8b3ae43))
* **aws-codepipeline:** make input and output artifact names optional when creating Actions. ([#845](#845)) ([3d91c93](3d91c93))

* **aws-codepipeline:** this commit contains the following breaking changes:
* Rename 'artifactName' in Action construction properties to 'outputArtifactName'
* Rename the 'artifact' property of Actions to 'outputArtifact'
* No longer allow adding output artifacts to Actions by instantiating the Artifact class
* Rename Action#input/outputArtifacts properties to _input/_outputArtifacts

Previously, we always required customers to explicitly name the output artifacts the Actions used in the Pipeline,
and to explicitly "wire together" the outputs of one Action as inputs to another.
With this change, the CodePipeline Construct generates artifact names,
if the customer didn't provide one explicitly,
and tries to find the first available output artifact to use as input to a newly created Action that needs it,
thus turning both the input and output artifacts from required to optional properties.
@debora-ito debora-ito added the package/tools Related to AWS CDK Tools or CLI label Nov 7, 2018
@eladb eladb changed the title Toolkit: Missing test coverage Toolkit: run integration tests automatically Jan 16, 2019
@eladb
Copy link
Contributor Author

eladb commented Jan 16, 2019

Okay, investigated the idea of creating an account for every test run 🏃‍♀️ and it seems like it is impossible to delete accounts using an API, only create accounts, so I think this a dead end at the moment. Anyone thinks otherwise? @rix0rrr @RomainMuller @sam-goodwin

@rix0rrr
Copy link
Contributor

rix0rrr commented Jan 16, 2019

I think it's fine to just have a single dedicated account to run the tests in. Yes, it's a bit of a bummer to have global state that can mess up your tests, but our tools deal with it pretty well I think.

@eladb
Copy link
Contributor Author

eladb commented Jan 16, 2019

Okay, discovered a couple of issues when trying to assume a role from a different account:

  1. The JavaScript SDK doesn't have support for the credential_source option, which is needed because credentials are available through the EcsContainer credentials provider (see Support credential_source for use with IAM roles aws-sdk-js#1916).
  2. If --profile is passed to the CDK toolkit and there are credentials in the environment, the profile should prevail and not the environment, but that's probably less of a problem because we don't have environment variables set on CodeBuild.

What I am going to try to do is extract credentials from the ECS provider on CodeBuild and see if I can construct an AWS config file that will work with the current implementations.

@rix0rrr
Copy link
Contributor

rix0rrr commented Jan 16, 2019 •
edited

Alternatively, go around the toolkit and use the CLI to assume role, then put the credentials in the environment? Not so nice, but I do this all the time:

#!/bin/bash
set -euo pipefail
arn=$1

name=$(whoami)-shell
creds=$(aws sts assume-role --role-arn "$arn" --role-session-name $name)

export AWS_ACCESS_KEY_ID=$(echo $creds | jq .Credentials.AccessKeyId | xargs)
export AWS_SECRET_ACCESS_KEY=$(echo $creds | jq .Credentials.SecretAccessKey | xargs)
export AWS_SESSION_TOKEN=$(echo $creds | jq .Credentials.SessionToken | xargs)

export PS1="\[\033[1;93m\][$arn] \W\$ \[\033[0m\]"
bash --norc -i

(Substitute Node magic for the jq call)

@eladb
Copy link
Contributor Author

eladb commented Jan 16, 2019

Yeah, why didn't I think of that!

@eladb
Copy link
Contributor Author

eladb commented Jan 16, 2019

Works! Hell yeah @rix0rrr Thanks!

eladb pushed a commit that referenced this issue Jan 29, 2019
chore(toolkit): integ tests moved to release pipeline
75b9838 
Integration tests are now executed automatically as part of our
release pipeline.

Fixes #294
@eladb eladb mentioned this issue Jan 29, 2019
Merged
4 tasks
eladb pushed a commit that referenced this issue Jan 29, 2019
chore(toolkit): integ tests moved to release pipeline (#1632)
5b2de2b 
Integration tests are now executed automatically as part of our
release pipeline.

Fixes #294
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
ops-excellence Operational Excellence package/tools Related to AWS CDK Tools or CLI
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

chore(toolkit): integ tests moved to release pipeline aws/aws-cdk
3 participants
@debora-ito @rix0rrr @eladb