Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(applets): integrate into toolkit #1039

Merged
merged 7 commits into from
Nov 2, 2018
Merged

feat(applets): integrate into toolkit #1039

merged 7 commits into from
Nov 2, 2018

Conversation

rix0rrr
Copy link
Contributor

@rix0rrr rix0rrr commented Oct 30, 2018

Integrate the Applet runner into the toolkit. When the --app argument
points to a .yml or .yaml file, the target will be assumed to be a
JavaScript applet, and run through the applet interpreter.

Enhancements to JS applet interpreter:

  • Allow using non-Stack constructs.
  • Multiple Stacks in one applet file by supplying an array.
  • Allow referencing packages directly from NPM.

Non-Stack constructs can now be used as applets. If a non-Stack applet
is selected, a Stack will be constructed around it. This makes it
possible to reuse Applet constructs in regular CDK applications (if they
meet the requirements).

Multiple applets can be define in one file by making the top-level
object in the YAML file a list. Each applet will get its own stack.

By starting an applet specifier with npm://, applet modules can
directly be referenced in NPM. You can include a version specifier
(@1.2.3) to reference specific versions.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license.

Integrate the Applet runner into the toolkit. When the --app argument
points to a .yml or .yaml file, the target will be assumed to be a
JavaScript applet, and run through the applet interpreter.

Enhancements to JS applet interpreter:

- Allow using non-Stack constructs.
- Multiple Stacks in one applet file by supplying an array.
- Allow referencing packages directly from NPM.

Non-Stack constructs can now be used as applets. If a non-Stack applet
is selected, a Stack will be constructed around it. This makes it
possible to reuse Applet constructs in regular CDK applications (if they
meet the requirements).

Multiple applets can be define in one file by making the top-level
object in the YAML file a list. Each applet will get its own stack.

By starting an applet specifier with npm://, applet modules can
directly be referenced in NPM. You can include a version specifier
(@1.2.3) to reference specific versions.
@rix0rrr rix0rrr self-assigned this Oct 30, 2018
Copy link
Contributor

@eladb eladb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am wondering about the file format for multiple applets (and also the fact that the name of the stack is not explicitly defined). How about we design this to look like a classic desired state configuration file (CFN template):

applets:
    applet1:
        type: <applet-type>
        properties:
            <properties>
    applet2:
        type: <applet2-type>
        properties:
            <properties>

The benefits:

  1. It's a familiar format, and maybe at somepoint, this will just be a CloudFormation template (there are some ideas circling around this).
  2. It will allow us to support references between applets (${applet1.output})
  3. The name of the applet can be used to determine the stack name.

import { debug } from './logging';

/** Invokes the cloud executable and returns JSON output */
export async function execProgram(aws: SDK, config: Settings): Promise<cxapi.SynthesizeResponse> {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The refactor makes it a bit hard to understand what had changed here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand, but I couldn't bear to keep on adding to the same file anymore.

The only functional change is the addition of the executable guessing.

/**
* Guess the executable from the command-line argument
*/
async function guessExecutable(commandLine: string[]) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was actually thinking we can design this a bit more generally and make people happy. For example, if app is a .js file, we can determine that it can be executed with node, if it's a .dll, we can use dotnet, etc. What do you think?

I was also thinking that we can introduce auto-detection (i.e. if there's a package.json file, or pom.xml, we can auto-detect where the CDK app is), but that's a later thing.

For now, I think adding support for .js could be nice and maybe implement this a bit more generically (extension to programming mapping).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that seems reasonable.

packages/@aws-cdk/applet-js/test/test.applets.ts Outdated Show resolved Hide resolved
packages/@aws-cdk/applet-js/bin/cdk-applet-js.ts Outdated Show resolved Hide resolved
#!/usr/bin/env cdk-applet-js
applet: ./test-applet:NoStackApplet
argument: this should be reflected in the template description
Applets:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any chance I can convince you to use lower case letters for the schema?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can, and I'm not liking this either.

But then let's not mimic the CloudFormation schema at all. Otherwise it's just going to be confusing, don't you think?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both Azure Resource Manager and Google Cloud Deployment Manager use very similar syntaxes. This is basically the standard-de-facto for desired state configuration.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alrighty

Copy link
Contributor

@eladb eladb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please Update applets topic in docs

@rix0rrr rix0rrr merged commit fdabe95 into master Nov 2, 2018
@rix0rrr rix0rrr deleted the huijbers/applets branch November 2, 2018 14:38
rix0rrr pushed a commit that referenced this pull request Nov 6, 2018
Bug Fixes
=========

* **aws-autoscaling:** allow minSize to be set to 0 ([#1015](#1015)) ([67f7fa1](67f7fa1))
* **aws-codebuild:** correctly pass the timeout property to CFN when creating a Project. ([#1071](#1071)) ([b1322bb](b1322bb))
* **aws-codebuild:** correctly set S3 path when using it as artifact. ([#1072](#1072)) ([f32cba9](f32cba9))
* **aws-kms:** add output value when exporting an encryption key ([#1036](#1036)) ([cb490be](cb490be))
* Switch from `js-yaml` to `yaml` ([#1092](#1092)) ([0b132b5](0b132b5))

Features
=========

* **applets:** integrate into toolkit ([#1039](#1039)) ([fdabe95](fdabe95)), closes [#849](#849) [#342](#342) [#291](#291)
* **aws-codecommit:** use CloudWatch Events instead of polling by default in the CodePipeline Action. ([#1026](#1026)) ([d09d30c](d09d30c))
* **aws-dynamodb:** allow specifying partition/sort keys in props ([#1054](#1054)) ([ec87331](ec87331)), closes [#1051](#1051)
* **aws-ec2:** AmazonLinuxImage supports AL2 ([#1081](#1081)) ([97b57a5](97b57a5)), closes [#1062](#1062)
* **aws-lambda:** high level API for event sources ([#1063](#1063)) ([1be3442](1be3442))
* **aws-sqs:** improvements to IAM grants API ([#1052](#1052)) ([6f2475e](6f2475e))
* don't upload the same asset multiple times ([#1011](#1011)) ([35937b6](35937b6)), closes [#989](#989)
* **codepipeline/cfn:** Use fewer statements for pipeline permissions ([#1009](#1009)) ([8f4c2ab](8f4c2ab))
* add a new construct library for ECS ([#1058](#1058)) ([ae03ddb](ae03ddb))
* **pkglint:** Make sure .snk files are ignored ([#1049](#1049)) ([53c8d76](53c8d76)), closes [#643](#643)
* **toolkit:** deployment ui improvements ([#1067](#1067)) ([c832eaf](c832eaf))

BREAKING CHANGES
=========

* The ec2.Connections object has been changed to be able to manage multiple
  security groups. The relevant property has been changed from `securityGroup`
  to `securityGroups` (an array of security group objects).
* **aws-codecommit:** This modifies the default behavior of the CodeCommit
  Action.  It also changes the internal API contract between the
  aws-codepipeline-api module and the CodePipeline Actions in the service
  packages.
* **applets:** The applet schema has changed to allow Multiple applets can be
  define in one file by structuring the files like this:
* **applets:** The applet schema has changed to allow definition of multiple
  applets in the same file.

The schema now looks like this:

    applets:
      MyApplet:
        type: ./my-applet-file
        properties:
          property1: value
          ...
By starting an applet specifier with npm://, applet modules can
directly be referenced in NPM. You can include a version specifier
(@1.2.3) to reference specific versions.
* **aws-sqs:** `queue.grantReceiveMessages` has been removed. It is unlikely
  that this would be sufficient to interact with a queue. Alternatively you can
  use `queue.grantConsumeMessages` or `queue.grant('sqs:ReceiveMessage')` if
  there's a need to only grant this action.
@rix0rrr rix0rrr mentioned this pull request Nov 6, 2018
rix0rrr pushed a commit that referenced this pull request Nov 6, 2018
Bug Fixes
========

* **aws-autoscaling:** allow minSize to be set to 0 ([#1015](#1015)) ([67f7fa1](67f7fa1))
* **aws-codebuild:** correctly pass the timeout property to CFN when creating a Project. ([#1071](#1071)) ([b1322bb](b1322bb))
* **aws-codebuild:** correctly set S3 path when using it as artifact. ([#1072](#1072)) ([f32cba9](f32cba9))
* **aws-kms:** add output value when exporting an encryption key ([#1036](#1036)) ([cb490be](cb490be))
* Switch from `js-yaml` to `yaml` ([#1092](#1092)) ([0b132b5](0b132b5))

Features
========

* don't upload the same asset multiple times ([#1011](#1011)) ([35937b6](35937b6)), closes [#989](#989)
* **app-delivery:** CI/CD for CDK Stacks ([#1022](#1022)) ([f2fe4e9](f2fe4e9))
* add a new construct library for ECS ([#1058](#1058)) ([ae03ddb](ae03ddb))
* **applets:** integrate into toolkit ([#1039](#1039)) ([fdabe95](fdabe95)), closes [#849](#849) [#342](#342) [#291](#291)
* **aws-codecommit:** use CloudWatch Events instead of polling by default in the CodePipeline Action. ([#1026](#1026)) ([d09d30c](d09d30c))
* **aws-dynamodb:** allow specifying partition/sort keys in props ([#1054](#1054)) ([ec87331](ec87331)), closes [#1051](#1051)
* **aws-ec2:** AmazonLinuxImage supports AL2 ([#1081](#1081)) ([97b57a5](97b57a5)), closes [#1062](#1062)
* **aws-lambda:** high level API for event sources ([#1063](#1063)) ([1be3442](1be3442))
* **aws-sqs:** improvements to IAM grants API ([#1052](#1052)) ([6f2475e](6f2475e))
* **codepipeline/cfn:** Use fewer statements for pipeline permissions ([#1009](#1009)) ([8f4c2ab](8f4c2ab))
* **pkglint:** Make sure .snk files are ignored ([#1049](#1049)) ([53c8d76](53c8d76)), closes [#643](#643)
* **toolkit:** deployment ui improvements ([#1067](#1067)) ([c832eaf](c832eaf))
* Update to CloudFormation resource specification v2.11.0

BREAKING CHANGES
========

* The ec2.Connections object has been changed to be able to manage multiple
  security groups. The relevant property has been changed from `securityGroup`
  to `securityGroups` (an array of security group objects).
* **aws-codecommit:** this modifies the default behavior of the CodeCommit
  Action.  It also changes the internal API contract between the
  aws-codepipeline-api module and the CodePipeline Actions in the service
  packages.
* **applets:** The applet schema has changed to allow Multiple applets can be
  define in one file by structuring the files like this:
* **applets:** The applet schema has changed to allow definition of multiple
  applets in the same file.

The schema now looks like this:

    applets:
      MyApplet:
        type: ./my-applet-file
        properties:
          property1: value
          ...
By starting an applet specifier with npm://, applet modules can directly be
referenced in NPM. You can include a version specifier (@1.2.3) to reference
specific versions.
* **aws-sqs:** `queue.grantReceiveMessages` has been removed. It is unlikely
  that this would be sufficient to interact with a queue. Alternatively you can
  use `queue.grantConsumeMessages` or `queue.grant('sqs:ReceiveMessage')` if
  there's a need to only grant this action.
rix0rrr added a commit that referenced this pull request Nov 6, 2018
Bug Fixes
========

* **aws-autoscaling:** allow minSize to be set to 0 ([#1015](#1015)) ([67f7fa1](67f7fa1))
* **aws-codebuild:** correctly pass the timeout property to CFN when creating a Project. ([#1071](#1071)) ([b1322bb](b1322bb))
* **aws-codebuild:** correctly set S3 path when using it as artifact. ([#1072](#1072)) ([f32cba9](f32cba9))
* **aws-kms:** add output value when exporting an encryption key ([#1036](#1036)) ([cb490be](cb490be))
* Switch from `js-yaml` to `yaml` ([#1092](#1092)) ([0b132b5](0b132b5))

Features
========

* don't upload the same asset multiple times ([#1011](#1011)) ([35937b6](35937b6)), closes [#989](#989)
* **app-delivery:** CI/CD for CDK Stacks ([#1022](#1022)) ([f2fe4e9](f2fe4e9))
* add a new construct library for ECS ([#1058](#1058)) ([ae03ddb](ae03ddb))
* **applets:** integrate into toolkit ([#1039](#1039)) ([fdabe95](fdabe95)), closes [#849](#849) [#342](#342) [#291](#291)
* **aws-codecommit:** use CloudWatch Events instead of polling by default in the CodePipeline Action. ([#1026](#1026)) ([d09d30c](d09d30c))
* **aws-dynamodb:** allow specifying partition/sort keys in props ([#1054](#1054)) ([ec87331](ec87331)), closes [#1051](#1051)
* **aws-ec2:** AmazonLinuxImage supports AL2 ([#1081](#1081)) ([97b57a5](97b57a5)), closes [#1062](#1062)
* **aws-lambda:** high level API for event sources ([#1063](#1063)) ([1be3442](1be3442))
* **aws-sqs:** improvements to IAM grants API ([#1052](#1052)) ([6f2475e](6f2475e))
* **codepipeline/cfn:** Use fewer statements for pipeline permissions ([#1009](#1009)) ([8f4c2ab](8f4c2ab))
* **pkglint:** Make sure .snk files are ignored ([#1049](#1049)) ([53c8d76](53c8d76)), closes [#643](#643)
* **toolkit:** deployment ui improvements ([#1067](#1067)) ([c832eaf](c832eaf))
* Update to CloudFormation resource specification v2.11.0

BREAKING CHANGES
========

* The ec2.Connections object has been changed to be able to manage multiple
  security groups. The relevant property has been changed from `securityGroup`
  to `securityGroups` (an array of security group objects).
* **aws-codecommit:** this modifies the default behavior of the CodeCommit
  Action.  It also changes the internal API contract between the
  aws-codepipeline-api module and the CodePipeline Actions in the service
  packages.
* **applets:** The applet schema has changed to allow Multiple applets can be
  define in one file by structuring the files like this:
* **applets:** The applet schema has changed to allow definition of multiple
  applets in the same file.

The schema now looks like this:

    applets:
      MyApplet:
        type: ./my-applet-file
        properties:
          property1: value
          ...
By starting an applet specifier with npm://, applet modules can directly be
referenced in NPM. You can include a version specifier (@1.2.3) to reference
specific versions.
* **aws-sqs:** `queue.grantReceiveMessages` has been removed. It is unlikely
  that this would be sufficient to interact with a queue. Alternatively you can
  use `queue.grantConsumeMessages` or `queue.grant('sqs:ReceiveMessage')` if
  there's a need to only grant this action.
@NGL321 NGL321 added the contribution/core This is a PR that came from AWS. label Sep 27, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
contribution/core This is a PR that came from AWS.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants