Skip to content

Commit

Permalink
chore: update cdk migrate command to allow for stack name input and a…
Browse files Browse the repository at this point in the history
…dditional languages (#27134)

This change updates use of noctilucent to cdk-from-cfn and installs it from npm. It also adds some additional language support and stack renaming.

Not included in this PR:
1. Automated testing - this PR has been manually tested and automated tests will be added in a subsequent PR.
2. Go support - there are still some bugs to work out with go (in cdk-from-cfn) so implementation for it will be added after that's been worked out.
3. The option to compress the app to a zip file
4. The other TODOs listed in the `migrate` file. 
----

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
  • Loading branch information
TheRealAmazonKendra authored Sep 14, 2023
1 parent dc87d28 commit 123ab70
Show file tree
Hide file tree
Showing 30 changed files with 172 additions and 137 deletions.
46 changes: 38 additions & 8 deletions packages/@aws-cdk/cli-lib-alpha/THIRD_PARTY_LICENSES
Original file line number Diff line number Diff line change
Expand Up @@ -138,6 +138,32 @@ OTHER DEALINGS IN THE SOFTWARE.

----------------

** archiver-utils@3.0.4 - https://www.npmjs.com/package/archiver-utils/v/3.0.4 | MIT
Copyright (c) 2015 Chris Talkington.

Permission is hereby granted, free of charge, to any person
obtaining a copy of this software and associated documentation
files (the "Software"), to deal in the Software without
restriction, including without limitation the rights to use,
copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the
Software is furnished to do so, subject to the following
conditions:

The above copyright notice and this permission notice shall be
included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
OTHER DEALINGS IN THE SOFTWARE.

----------------

** archiver@5.3.2 - https://www.npmjs.com/package/archiver/v/5.3.2 | MIT
Copyright (c) 2012-2014 Chris Talkington, contributors.

Expand Down Expand Up @@ -238,7 +264,7 @@ THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH RE

----------------

** aws-sdk@2.1447.0 - https://www.npmjs.com/package/aws-sdk/v/2.1447.0 | Apache-2.0
** aws-sdk@2.1451.0 - https://www.npmjs.com/package/aws-sdk/v/2.1451.0 | Apache-2.0
AWS SDK for JavaScript
Copyright 2012-2017 Amazon.com, Inc. or its affiliates. All Rights Reserved.

Expand Down Expand Up @@ -433,6 +459,10 @@ The above copyright notice and this permission notice shall be included in all c
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.


----------------

** cdk-from-cfn@0.11.0 - https://www.npmjs.com/package/cdk-from-cfn/v/0.11.0 | MIT OR Apache-2.0

----------------

** chalk@4.1.2 - https://www.npmjs.com/package/chalk/v/4.1.2 | MIT
Expand Down Expand Up @@ -532,7 +562,7 @@ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLI

----------------

** compress-commons@4.1.1 - https://www.npmjs.com/package/compress-commons/v/4.1.1 | MIT
** compress-commons@4.1.2 - https://www.npmjs.com/package/compress-commons/v/4.1.2 | MIT
Copyright (c) 2014 Chris Talkington, contributors.

Permission is hereby granted, free of charge, to any person
Expand Down Expand Up @@ -811,7 +841,7 @@ IN THE SOFTWARE.

----------------

** crc32-stream@4.0.2 - https://www.npmjs.com/package/crc32-stream/v/4.0.2 | MIT
** crc32-stream@4.0.3 - https://www.npmjs.com/package/crc32-stream/v/4.0.3 | MIT
Copyright (c) 2014 Chris Talkington, contributors.

Permission is hereby granted, free of charge, to any person
Expand Down Expand Up @@ -1367,7 +1397,7 @@ SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

----------------

** https-proxy-agent@7.0.1 - https://www.npmjs.com/package/https-proxy-agent/v/7.0.1 | MIT
** https-proxy-agent@7.0.2 - https://www.npmjs.com/package/https-proxy-agent/v/7.0.2 | MIT

----------------

Expand Down Expand Up @@ -2134,7 +2164,7 @@ IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.

----------------

** pac-proxy-agent@7.0.0 - https://www.npmjs.com/package/pac-proxy-agent/v/7.0.0 | MIT
** pac-proxy-agent@7.0.1 - https://www.npmjs.com/package/pac-proxy-agent/v/7.0.1 | MIT

----------------

Expand Down Expand Up @@ -2224,7 +2254,7 @@ THE SOFTWARE.

----------------

** proxy-agent@6.3.0 - https://www.npmjs.com/package/proxy-agent/v/6.3.0 | MIT
** proxy-agent@6.3.1 - https://www.npmjs.com/package/proxy-agent/v/6.3.1 | MIT

----------------

Expand Down Expand Up @@ -2793,7 +2823,7 @@ CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

----------------

** socks-proxy-agent@8.0.1 - https://www.npmjs.com/package/socks-proxy-agent/v/8.0.1 | MIT
** socks-proxy-agent@8.0.2 - https://www.npmjs.com/package/socks-proxy-agent/v/8.0.2 | MIT

----------------

Expand Down Expand Up @@ -3375,7 +3405,7 @@ THE SOFTWARE.

----------------

** zip-stream@4.1.0 - https://www.npmjs.com/package/zip-stream/v/4.1.0 | MIT
** zip-stream@4.1.1 - https://www.npmjs.com/package/zip-stream/v/4.1.1 | MIT
Copyright (c) 2014 Chris Talkington, contributors.

Permission is hereby granted, free of charge, to any person
Expand Down
16 changes: 0 additions & 16 deletions packages/@aws-cdk/cli-lib-alpha/generate.sh

This file was deleted.

8 changes: 4 additions & 4 deletions packages/@aws-cdk/cli-lib-alpha/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -50,9 +50,9 @@
"build+test": "yarn build && yarn test",
"build+test+extract": "yarn build+test && yarn rosetta:extract",
"build+test+package": "yarn build+test && yarn package",
"bundle": "esbuild --bundle lib/index.ts --target=node14 --platform=node --external:fsevents --minify-whitespace --outfile=lib/main.js",
"bundle": "esbuild --bundle lib/index.ts --target=node18 --platform=node --external:fsevents --minify-whitespace --outfile=lib/main.js",
"compat": "cdk-compat",
"gen": "./generate.sh",
"gen": "../../../packages/aws-cdk/generate.sh",
"lint": "cdk-lint",
"package": "cdk-package",
"pkglint": "pkglint -f",
Expand All @@ -68,8 +68,8 @@
"cdk-build": {
"post": [
"yarn attribute",
"cp ../../aws-cdk/lib/vendor/noctilucent/index_bg.wasm ./lib/",
"mkdir -p ./lib/api/bootstrap/ && cp ../../aws-cdk/lib/api/bootstrap/bootstrap-template.yaml ./lib/api/bootstrap/",
"cp ../../../node_modules/cdk-from-cfn/index_bg.wasm ./lib/",
"yarn bundle",
"node ./lib/main.js >/dev/null 2>/dev/null </dev/null"
],
Expand Down Expand Up @@ -126,4 +126,4 @@
},
"dependencies": {},
"peerDependencies": {}
}
}
1 change: 0 additions & 1 deletion packages/aws-cdk/.eslintrc.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
const baseConfig = require('@aws-cdk/cdk-build-tools/config/eslintrc');
baseConfig.ignorePatterns.push('lib/init-templates/**/typescript/**/*.ts');
baseConfig.ignorePatterns.push('test/integ/cli/sam_cdk_integ_app/**/*.ts');
baseConfig.ignorePatterns.push('vendor/noctilucent/**/*.ts');
baseConfig.parserOptions.project = __dirname + '/tsconfig.json';
module.exports = baseConfig;
6 changes: 2 additions & 4 deletions packages/aws-cdk/.gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -36,10 +36,8 @@ test/integ/cli/*.js
test/integ/cli/*.d.ts
!test/integ/cli-regression-patches/**/*

lib/vendor/noctilucent/*
!lib/vendor/noctilucent/.keepdir

.DS_Store

junit.xml
tmp/

lib/**/*.wasm
2 changes: 0 additions & 2 deletions packages/aws-cdk/.npmignore
Original file line number Diff line number Diff line change
Expand Up @@ -30,5 +30,3 @@ tsconfig.json
junit.xml

generate.sh
lib/vendor/noctilucent/Dockerfile
tmp/
4 changes: 4 additions & 0 deletions packages/aws-cdk/THIRD_PARTY_LICENSES
Original file line number Diff line number Diff line change
Expand Up @@ -459,6 +459,10 @@ The above copyright notice and this permission notice shall be included in all c
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.


----------------

** cdk-from-cfn@0.11.0 - https://www.npmjs.com/package/cdk-from-cfn/v/0.11.0 | MIT OR Apache-2.0

----------------

** chalk@4.1.2 - https://www.npmjs.com/package/chalk/v/4.1.2 | MIT
Expand Down
16 changes: 0 additions & 16 deletions packages/aws-cdk/generate.sh
Original file line number Diff line number Diff line change
Expand Up @@ -14,19 +14,3 @@ cat > build-info.json <<HERE
"commit": "${commit:0:7}"
}
HERE

# Download noctilucent wasm-pack build
NOCTILUCENT_VERSION=0.1.2
PACK_URL=https://github.com/iph/noctilucent/releases/download/v${NOCTILUCENT_VERSION}/wasm-pack.zip
zipfile=$PWD/tmp/noctilucent-wasm-${NOCTILUCENT_VERSION}.zip
outdir=lib/vendor/noctilucent

mkdir -p tmp
if [[ ! -f "$zipfile" ]]; then
curl -sSfLo "$zipfile" "$PACK_URL"
fi

(cd $outdir && unzip -qo $zipfile)

# Don't need these files
rm -f $outdir/{.gitignore,README.md,package.json}
16 changes: 10 additions & 6 deletions packages/aws-cdk/lib/cli.ts
Original file line number Diff line number Diff line change
Expand Up @@ -272,10 +272,10 @@ async function parseCommandLineArguments(args: string[]) {
.option('generate-only', { type: 'boolean', default: false, desc: 'If true, only generates project files, without executing additional operations such as setting up a git repo, installing dependencies or compiling the project' }),
)
.command('migrate', false /* hidden from "cdk --help" */, (yargs: Argv) => yargs
.option('language', { type: 'string', alias: 'l', desc: 'The language to be used for the new project (default can be configured in ~/.cdk.json)', choices: MIGRATE_SUPPORTED_LANGUAGES })
.option('input-path', { type: 'string', alias: 'inputpath', desc: 'The path to the CloudFormation template to migrate' })
.option('output-path', { type: 'string', alias: 'outputpath', desc: 'The output path for the migrated cdk code' })
.option('generate-only', { type: 'boolean', default: false, desc: 'If true, only generates project files, without executing additional operations such as setting up a git repo, installing dependencies or compiling the project' }),
.option('stack-name', { type: 'string', alias: 'n', desc: 'The name assigned to the stack created in the new project. The name of the app will be based off this name as well.', requiresArg: true })
.option('language', { type: 'string', default: 'typescript', alias: 'l', desc: 'The language to be used for the new project', choices: MIGRATE_SUPPORTED_LANGUAGES })
.option('from-path', { type: 'string', alias: 'p', desc: 'The path to the CloudFormation template to migrate. Use this for locally stored templates' })
.option('output-path', { type: 'string', alias: 'o', desc: 'The output path for the migrated cdk app' }),
)
.command('context', 'Manage cached context values', (yargs: Argv) => yargs
.option('reset', { alias: 'e', desc: 'The context key (or its index) to reset', type: 'string', requiresArg: true })
Expand Down Expand Up @@ -659,8 +659,12 @@ export async function exec(args: string[], synthesizer?: Synthesizer): Promise<n
return cliInit(args.TEMPLATE, language, undefined, args.generateOnly);
}
case 'migrate':
const migrateLanguage = configuration.settings.get(['language']);
return cliMigrate(args.inputpath, migrateLanguage, args.generateOnly, args.outputpath);
return cliMigrate({
stackName: args['stack-name'],
fromPath: args['from-path'],
language: args.language,
outputPath: args['output-path'],
});
case 'version':
return data(version.DISPLAY_VERSION);

Expand Down
115 changes: 61 additions & 54 deletions packages/aws-cdk/lib/commands/migrate.ts
Original file line number Diff line number Diff line change
@@ -1,71 +1,78 @@
import * as fs from 'fs';
import * as path from 'path';
import { initializeProject, availableInitTemplates } from '../../lib/init';
import * as cdk_from_cfn from 'cdk-from-cfn';
import { cliInit } from '../../lib/init';
import { warning } from '../logging';
import * as nocti from '../vendor/noctilucent';

/* eslint-disable @typescript-eslint/no-var-requires */ // Packages don't have @types module
// eslint-disable-next-line @typescript-eslint/no-require-imports
const camelCase = require('camelcase');
// eslint-disable-next-line @typescript-eslint/no-require-imports
const decamelize = require('decamelize');

/** The list of languages supported by the built-in noctilucent binary. */
export const MIGRATE_SUPPORTED_LANGUAGES: readonly string[] = nocti.supported_languages();
export const MIGRATE_SUPPORTED_LANGUAGES: readonly string[] = cdk_from_cfn.supported_languages();

export async function cliMigrate(
inputpath: string = process.cwd() + '/../template.txt',
language = MIGRATE_SUPPORTED_LANGUAGES[0],
generateOnly = false,
outputpath = process.cwd(),
) {
warning('This is an experimental feature. We make no guarantees about the outcome or stability of the functionality.');
const type = 'default'; // "default" is the default type (and maps to 'app')
const template = (await availableInitTemplates()).find(t => t.hasName(type!));
if (!template) {
throw new Error(`couldn't find template for ${type} app type, this should never happen`);
}
export interface CliMigrateOptions {
readonly stackName: string;
readonly language?: string;
readonly fromPath?: string;
readonly outputPath?: string;
}

if (!MIGRATE_SUPPORTED_LANGUAGES.includes(language)) {
throw new Error(`Unsupported language for cdk migrate: ${language}. Supported languages are: ${MIGRATE_SUPPORTED_LANGUAGES.join(', ')}`);
}
export async function cliMigrate(options: CliMigrateOptions) {
warning('This is an experimental feature. We make no guarantees about the outcome or stability of the functionality.');

await initializeProject(template, language, true, generateOnly, outputpath);
const template_file = fs.readFileSync(inputpath, 'utf8');
const generated_app = nocti.transmute(template_file, language);
// TODO: Validate stack name

// clear out the init'd bin/lib files to replace with our own
delete_files(outputpath + '/lib/');
const language = options.language ?? 'typescript';
const outputPath = path.join(options.outputPath ?? process.cwd(), options.stackName);

// we hardcode everything to be called noctstack still so this works for now.
// Will change this to be much smarter once we can change stack name in noct
const bin_app = `#!/usr/bin/env node
import 'source-map-support/register';
import * as cdk from 'aws-cdk-lib';
import { NoctStack } from '../lib/generated_stack';
const generatedStack = generateStack(options, language);
const stackName = decamelize(options.stackName);

const app = new cdk.App();
new NoctStack(app, 'NoctStack', {
/* If you don't specify 'env', this stack will be environment-agnostic.
* Account/Region-dependent features and context lookups will not work,
* but a single synthesized template can be deployed anywhere. */
try {
fs.rmSync(outputPath, { recursive: true, force: true });
fs.mkdirSync(outputPath, { recursive: true });
await cliInit('app', language, true, false, outputPath, options.stackName);

/* Uncomment the next line to specialize this stack for the AWS Account
* and Region that are implied by the current CLI configuration. */
// env: { account: process.env.CDK_DEFAULT_ACCOUNT, region: process.env.CDK_DEFAULT_REGION },
let stackFileName: string;
switch (language) {
case 'typescript':
stackFileName = `${outputPath}/lib/${stackName}-stack.ts`;
break;
case 'java':
stackFileName = `${outputPath}/src/main/java/com/myorg/${camelCase(stackName, { pascalCase: true })}Stack.java`;
break;
case 'python':
stackFileName = `${outputPath}/${stackName.replace(/-/g, '_')}/${stackName.replace(/-/g, '_')}_stack.py`;
break;
case 'csharp':
stackFileName = `${outputPath}/src/${camelCase(stackName, { pascalCase: true })}/${camelCase(stackName, { pascalCase: true })}Stack.cs`;
break;
// TODO: Add Go support
default:
throw new Error(`${language} is not supported by CDK Migrate. Please choose from: ${MIGRATE_SUPPORTED_LANGUAGES.join(', ')}`);
}
fs.writeFileSync(stackFileName!, generatedStack);
} catch (error) {
fs.rmSync(outputPath, { recursive: true, force: true });
throw error;
}

/* Uncomment the next line if you know exactly what Account and Region you
* want to deploy the stack to. */
// env: { account: '123456789012', region: 'us-east-1' },
}

/* For more information, see https://docs.aws.amazon.com/cdk/latest/guide/environments.html */
});`;
const myname = path.basename(outputpath);
fs.writeFileSync(outputpath + '/lib/' + 'generated_stack.ts', generated_app);
fs.writeFileSync(outputpath + '/bin/' + `${myname}.ts`, bin_app);
function generateStack(options: CliMigrateOptions, language: string) {
const stackName = `${camelCase(decamelize(options.stackName), { pascalCase: true })}Stack`;
// We will add other options here in a future change.
if (options.fromPath) {
return fromPath(stackName, options.fromPath, language);
}
// TODO: replace with actual output for other options.
return '';
}

function delete_files(filepath: string) {
fs.readdir(filepath, (err, files) => {
if (err) throw err;
for (const file of files) {
fs.unlink(filepath + file, (cause) => {
if (cause) throw cause;
});
}
});
function fromPath(stackName: string, inputPath: string, language: string): string {
const templateFile = fs.readFileSync(inputPath, 'utf8');
return cdk_from_cfn.transmute(templateFile, language, stackName);
}
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ sealed class Program
public static void Main(string[] args)
{
var app = new App();
new %name.PascalCased%Stack(app, "%name.PascalCased%Stack", new StackProps
new %name.PascalCased%Stack(app, "%stackname%", new StackProps
{
// If you don't specify 'env', this stack will be environment-agnostic.
// Account/Region-dependent features and context lookups will not work,
Expand Down
Loading

0 comments on commit 123ab70

Please sign in to comment.