add serverless.ts support #7755

bryan-hunter · 2020-05-20T18:26:53Z

Closes: #6335

Very simple PR to handle serverless.ts files.

Try to find ts-node module when .ts file is detected and throw an error that you need to provide this dependency to use serverless.ts files.

bryan-hunter · 2020-05-20T18:30:42Z

yesterday I added AWS types for serverless.yml files DefinitelyTyped/DefinitelyTyped#44894

the combination of these 2 things could make it very easy to use Serverless with AWS and get type safety/auto completion.

codecov-commenter · 2020-05-20T18:32:52Z

Codecov Report

Merging #7755 into master will increase coverage by 0.17%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master    #7755      +/-   ##
==========================================
+ Coverage   88.00%   88.18%   +0.17%     
==========================================
  Files         245      245              
  Lines        9254     9293      +39     
==========================================
+ Hits         8144     8195      +51     
+ Misses       1110     1098      -12

Impacted Files	Coverage Δ
lib/classes/Utils.js	`95.33% <100.00%> (+0.06%)`	⬆️
lib/utils/getServerlessConfigFile.js	`100.00% <100.00%> (ø)`
lib/utils/log/log.js	`0.00% <0.00%> (ø)`
lib/classes/Variables.js	`99.72% <0.00%> (ø)`
lib/classes/YamlParser.js	`100.00% <0.00%> (ø)`
lib/plugins/create/create.js	`91.11% <0.00%> (ø)`
...ws/package/compile/events/cognitoUserPool/index.js	`100.00% <0.00%> (ø)`
lib/plugins/interactiveCli/inquirer.js
lib/utils/logDeprecation.js	`100.00% <0.00%> (ø)`
lib/classes/Service.js	`97.47% <0.00%> (+0.04%)`	⬆️
... and 3 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3e446f0...6ab43c2. Read the comment docs.

medikoo

@bryan-hunter great thanks for giving that a spin! It's definitely a worthwhile improvement for all TypeScript developers.

We'll be happy to take this in, still please see my suggestions

medikoo · 2020-05-21T07:33:15Z

lib/utils/getServerlessConfigFile.js

  BbPromise.try(() => {
-    // use require to load serverless.js
+    if (isTs) {


Let's not introduce isTs argument, but simply detect that by configFile.endsWith('.ts')

medikoo · 2020-05-21T07:33:38Z

lib/utils/getServerlessConfigFile.js

    // In case of a promise result, first resolve it.
    return configExport;
  }).then(config => {
    if (_.isPlainObject(config)) {
      return config;
    }
-    throw new Error('serverless.js must export plain object');
+    throw new Error(`serverless.${isTs ? 'ts' : 'js'} must export plain object`);


Let's resolve config filename via path.basename instead

medikoo · 2020-05-21T07:36:21Z

lib/utils/getServerlessConfigFile.js

  BbPromise.try(() => {
-    // use require to load serverless.js
+    if (isTs) {
+      // eslint-disable-next-line global-require


Let's remove this comment (we do not have this rule turned on)

medikoo · 2020-05-21T08:03:52Z

lib/utils/getServerlessConfigFile.js

-    // use require to load serverless.js
+    if (isTs) {
+      // eslint-disable-next-line global-require
+      require('ts-node').register();


I would prefer not add it as dependency. I think TS users should just ensure that both typescript and ts-node are installed in their setup.

Also we need to ensure it'll work with standalone build, which makes it a bit more tricky (as it doesn't have natural access to service or global npm dependencies)

I imagine reliable logic of acquisition of ts-node as follows:

const resolveModulePath = require('ncjsm/resolve'); const spawn = require('child-process-ext/spawn') ... const tsNode = () => { // 1. Resolve as serverless peer dependency return resolveModulePath(__dirname, "ts-node") .catch((error) => { if (error.code !== "MODULE_NOT_FOUND") throw error; // 2. Resolve as service dependency return resolveModulePath(path.dirname(configFile), "ts-node"); }) .catch((error) => { if (error.code !== "MODULE_NOT_FOUND") throw error; // 2. Resolve as global installation return spawn("npm", ["root", "-g"]).then( ({ stdoutBuffer }) => { return resolveModulePath("/", `${String(stdoutBuffer).trim()}/ts-node`).catch( (error) => { if (error.code !== "MODULE_NOT_FOUND") throw error; return null; } ); }, (error) => null ); }) .then((tsNodePath) => { if (!tsNodePath) { throw new ServerlessError( "Ensure 'ts-node' dependency for working with TypeScript configuration files" ); } return require(tsNodePath); }); };

I updated this to remove ts-node and resolve it here - some weird stuff I had to do to get it to resolve symlinks (because it returns an object instead of filepath), but works for all 3 cases. Let me know if you like this approach, or if you want to include ts-node as a dep.

…stall

bryan-hunter · 2020-05-26T15:49:08Z

package.json

@@ -171,6 +171,7 @@
    "sinon-chai": "^3.5.0",
    "standard-version": "^8.0.0",
    "strip-ansi": "^5.2.0",
+    "ts-node": "^8.10.1",


added as devDependency for test case

Let's not add it.

In previous comment I proposed to use fixture and when testing in fixture we may create a dummy node_modules/ts-node.js in a fixture folder, which should be a module that exposes a register method. Through that we may confirm it doesn't crash and that register is invoked

bryan-hunter · 2020-05-26T15:50:29Z

@medikoo I have removed ts-node as a dependency (made it a devDependency for testing).

I have updated code and added tests. Let me know what you think :)

medikoo

Thanks @bryan-hunter for update! Please see my remarks

medikoo · 2020-05-27T16:49:08Z

lib/utils/getServerlessConfigFile.js

+        throw error;
+      }
+      // if there is a symlink, we need to get 'realPath' from the return object
+      const pathString = typeof result === 'string' ? result : result.realPath;


In all cases object with realPath property would be returned (sorry I forgot to reflect that in my proposal).

So technically we should just return result.realPath

medikoo · 2020-05-27T16:51:36Z

lib/utils/getServerlessConfigFile.js

+      if (!result) {
+        const error = new Error(`Cannot find module '${args[1]}'`);
+        error.code = 'MODULE_NOT_FOUND';
+        throw error;
+      }


resolveModulePath when module is not found will naturally crash with such error, so this seems as dead code

medikoo · 2020-05-27T16:55:26Z

lib/utils/getServerlessConfigFile.js

+      }
+      // if there is a symlink, we need to get 'realPath' from the return object
+      const pathString = typeof result === 'string' ? result : result.realPath;
+      return require(pathString);


I don't think we should require here.

At this point we know that ts-node module exists at path, and that should be a final path to be used. If for some reason require of existing ts-node crashes, it should be exposed, and not that we should try other locations.

It'll also reflect how resolution works in Node.js (other locations are not inspected if existing module crashes)

moved it to after the chain of catches once module is resolved, so won't error on require and be caught erroneously

medikoo · 2020-05-27T16:56:44Z

lib/utils/getServerlessConfigFile.js

+  const resolveAsServiceDependency = () => requireOrThrow(serviceDir, 'ts-node');
+  const resolveAsGlobalInstallation = () =>
+    spawn('npm', ['root', '-g']).then(({ stdoutBuffer }) => {
+      return requireOrThrow('/', `${String(stdoutBuffer).trim()}/ts-node`);


Here as we have an abosolute path, it'll be good to just use require.resolve() (in previous example I used ncjsm resolve, but in this case it's not justified)

modified to use require.resolve

medikoo · 2020-05-27T16:57:37Z

lib/utils/getServerlessConfigFile.js

  });

 const getServerlessConfigFile = _.memoize(
  serverless =>
    getServerlessConfigFilePath(serverless).then(configFilePath => {
      if (configFilePath !== null) {
-        const isJSConfigFile = _.last(_.split(configFilePath, '.')) === 'js';
+        const fileExtension = _.last(_.split(configFilePath, '.'));


I know it's in old code like that, but maybe refactor it to use path.ext. This lodash usage is highly unjustified

modified to use path.extname and changed the if check to include the . on .ts and .js

medikoo · 2020-05-27T17:05:21Z

lib/utils/getServerlessConfigFile.test.js

@@ -146,6 +148,81 @@ describe('#getServerlessConfigFile()', () => {
    ).to.be.rejectedWith('serverless.js must export plain object');
  });

+  it('should return the file content if a serverless.ts file found', () => {


For tests we should use runServerless util, as documented here: https://github.com/serverless/serverless/blob/master/tests/README.md#unit-tests

It's the only reliable way to confirm that it works. We may prepare some basic fixture that involves serverless.ts config (but with no real TS internals, to not impose TS dependency on test engine)

I added a fixture and added a test case. Let me know if you think there's more that I should be testing

medikoo · 2020-05-27T17:07:05Z

package.json

@@ -171,6 +171,7 @@
    "sinon-chai": "^3.5.0",
    "standard-version": "^8.0.0",
    "strip-ansi": "^5.2.0",
+    "ts-node": "^8.10.1",


Let's not add it.

In previous comment I proposed to use fixture and when testing in fixture we may create a dummy node_modules/ts-node.js in a fixture folder, which should be a module that exposes a register method. Through that we may confirm it doesn't crash and that register is invoked

medikoo

@bryan-hunter looks great! I've posted just few last minor suggestions and we should be good to go

medikoo · 2020-05-28T07:48:29Z

lib/utils/getServerlessConfigFile.js

+  const resolveModuleRealPath = (...args) =>
+    resolveModulePath(...args).then(({ realPath }) => realPath);
+
+  const createOnError = cb => error => {


Let's rename it to ifNotFoundContinueWith (at least createOnError doesn't reflect what this util does)

medikoo · 2020-05-28T07:51:28Z

lib/utils/getServerlessConfigFile.js

+  const throwTsNodeError = () => {
+    throw new ServerlessError(
+      'Ensure "ts-node" dependency when working with TypeScript configuration files'
+    );


For better processing (e.g. in tests), it'll be nice to add code to this error (it's accepted as second argument), e.g. 'TS_NODE_NOT_FOUND'

done and used in test :)

medikoo · 2020-05-28T07:52:31Z

tests/fixtures/serverlessTs/node_modules/ts-node.js

@@ -0,0 +1,3 @@
+'use strict';
+
+module.exports.register = () => null;


I was thinking of adding this conditionally.

So first test without it (and confirm that 'TS_NODE_NOT_FOUND' error was thrown), then create this file in test, and confirm it resolves.

Note that fixures.cleanup will remove created file from fixture, so you don't need to worry about cleanup

done. I did have to pass in "extraPaths" option to fixture.cleanup because it only tries to cleanup .serverless, but was easy enough and seems to work nicely.

Also had to pass in shouldStubSpawn: true so wouldn't resolve my global npm ts-node... was nice to already have that option available - nice job on making everything easy to use though :)

medikoo

Looks great, thank you @bryan-hunter ! Thanks for good words on test utils, it's nice to hear you quickly managed to solve approached issues.

bryan-hunter · 2020-06-01T15:41:54Z

@medikoo do you know when it can be released?

medikoo · 2020-06-01T19:05:58Z

@bryan-hunter tomorrow morning CET time

DavidEspinosa42 · 2020-06-18T20:40:01Z

This would allow to replace a current serverless.yml for a serverless.ts?
Can you provide a simple example of the .ts file format?

bryan-hunter · 2020-06-18T21:11:39Z

This would allow to replace a current serverless.yml for a serverless.ts?
Can you provide a simple example of the .ts file format?

@DavidEspinosa42 this would be one example:

serverless.ts

import type Aws from 'serverless/aws';

const myService: Aws.Serverless = {
    plugins: ['serverless-webpack', 'serverless-offline'],
    frameworkVersion: '>=1.72.0',
    service: {
        name: 'YOUR_SERVICE',
    },
    provider: {
        name: 'aws',
        runtime: 'nodejs10.x',
        timeout: 30,
        memorySize: 1024,
        environment: {
            SERVICE_NAME: 'YOUR_SERVICE',
        },
        apiGateway: {
            minimumCompressionSize: 0,
        },
        tracing: {
            lambda: true,
            apiGateway: true,
        },
        logs: {
            frameworkLambda: true,
        },
    },
    package: {
        individually: true,
    },
    custom: {
        webpack: {
            includeModules: {
                forceExclude: ['aws-sdk'],
            },
            packager: 'yarn',
        },
        'serverless-offline': {
            host: '0.0.0.0',
            port: 3000,
            location: '.webpack/service',
            apiKey: 'fakeKey',
        },
    },
    resources: {
        Resources: {
            GatewayResponseDefault4XX: {
                Type: 'AWS::ApiGateway::GatewayResponse',
                Properties: {
                    ResponseParameters: {
                        'gatewayresponse.header.Access-Control-Allow-Origin': "'*'",
                        'gatewayresponse.header.Access-Control-Allow-Headers': "'*'",
                    },
                    ResponseType: 'DEFAULT_4XX',
                    RestApiId: {
                        Ref: 'ApiGatewayRestApi',
                    },
                    ResponseTemplates: {
                        'application/json':
                            '{"error": {"code": "custom-4XX-generic", "message": $context.error.messageString, "type": "$context.error.responseType"}, "requestId": "$context.requestId"}',
                    },
                },
            },
            GatewayResponseDefault5XX: {
                Type: 'AWS::ApiGateway::GatewayResponse',
                Properties: {
                    ResponseParameters: {
                        'gatewayresponse.header.Access-Control-Allow-Origin': "'*'",
                        'gatewayresponse.header.Access-Control-Allow-Headers': "'*'",
                    },
                    ResponseType: 'DEFAULT_5XX',
                    RestApiId: {
                        Ref: 'ApiGatewayRestApi',
                    },
                    ResponseTemplates: {
                        'application/json':
                            '{"error": {"code": "custom-5XX-generic", "message": $context.error.messageString, "type": "$context.error.responseType"}, "requestId": "$context.requestId"}',
                    },
                },
            },
            DynamoDbTable: {
                Type: 'AWS::DynamoDB::Table',
                DeletionPolicy: 'Retain',
                Properties: {
                    TableName: 'YOUR_TABLE_NAME',
                    BillingMode: 'PAY_PER_REQUEST',
                    AttributeDefinitions: [
                        {
                            AttributeName: 'id',
                            AttributeType: 'S',
                        },
                    ],
                    KeySchema: [
                        {
                            AttributeName: 'id',
                            KeyType: 'HASH',
                        },
                    ],
                    TimeToLiveSpecification: {
                        AttributeName: 'expiresAt',
                        Enabled: true,
                    },
                },
            },
        },
    },
    functions: {
        get: {
            handler: 'src/handler/YourHandler.get',
            environment: {},
            events: [
                {
                    http: {
                        path: 'data/{id}',
                        method: 'get',
                        private: true
                    },
                },
            ],
            tags: {},
        },
    },
};

module.exports = myService;

be sure to install @types/serverless (some of the types may be slightly incorrect there, but can easily make a PR to fix them) if you want to use Serverless types out of the box - otherwise you can use your own typings :)

DavidEspinosa42 · 2020-06-18T22:06:59Z

Thanks Bryan!!
It worked perfectly 😄
Awesome work!

bharat-tiwari · 2020-08-08T18:01:59Z

how to use the custom variables in the ts version

like, how would the below yml would convert to ts:

…
…
 custom:
    dbTable: 'myDBTable-${opt:stage}'
…
…
  environment:
    DB_TABLE: ${self:custom.dbTable}

DavidEspinosa42 · 2020-08-10T00:42:54Z

@bharat-tiwari You could use something like yargs:

import * as yargs from 'yargs';

custom: {
'dbTable': 'myDBTable-' + yargs.argv.stage
}

mospina · 2020-10-06T06:21:38Z

I still can figure it out how to do this:

environment:
    DB_TABLE: ${self:custom.dbTable}

DavidEspinosa42 · 2020-10-06T13:06:01Z

@mospina have you tried with quotes?
DB_TABLE: '${self:custom.dbTable}'

add serverless.ts support

6f03b31

bryan-hunter mentioned this pull request May 20, 2020

Extend serverless.js config files to support TypeScript serverless.ts #6335

Closed

mfulton26 previously approved these changes May 20, 2020

View reviewed changes

jonnyg5309 previously approved these changes May 20, 2020

View reviewed changes

medikoo added the enhancement label May 21, 2020

medikoo suggested changes May 21, 2020

View reviewed changes

medikoo assigned bryan-hunter May 21, 2020

remove ts-node, resolve by peerDep, service dependency, and global in…

8d797e5

…stall

bryan-hunter dismissed stale reviews from jonnyg5309 and mfulton26 via 8d797e5 May 21, 2020 23:32

bryan-hunter added 5 commits May 21, 2020 19:04

run prettier

fd7ec3b

add ts-node devDependency for testing, add tests

a6c1845

prettify

6268023

add another test for MODULE_NOT_FOUND

7831aee

wording

59e26cd

bryan-hunter commented May 26, 2020

View reviewed changes

medikoo suggested changes May 27, 2020

View reviewed changes

bryan-hunter added 2 commits May 27, 2020 15:10

add fixture for testing with runServerless

a81a340

run prettify

1c9628c

medikoo suggested changes May 28, 2020

View reviewed changes

bryan-hunter added 2 commits May 28, 2020 11:53

updates

9dc8101

update tests

6ab43c2

medikoo approved these changes Jun 1, 2020

View reviewed changes

medikoo merged commit 4db8b63 into serverless:master Jun 1, 2020

bryan-hunter mentioned this pull request Jun 2, 2020

feat(ts-support): adds ts support serverless-operations/serverless-step-functions#342

Merged

fredericbarthelet mentioned this pull request Jul 12, 2020

Use serverless.ts configuration file format for aws-nodejs-typescript example #7933

Closed

robburger mentioned this pull request Jul 26, 2020

Support for serverless.ts configuration serverless/dashboard-plugin#455

Closed

natac13 mentioned this pull request Sep 19, 2020

AWS Serverless !Ref? DefinitelyTyped/DefinitelyTyped#47791

Closed

4 tasks

		@@ -0,0 +1,3 @@
		'use strict';

		module.exports.register = () => null;

add serverless.ts support #7755

add serverless.ts support #7755

Conversation

bryan-hunter commented May 20, 2020 • edited

bryan-hunter commented May 20, 2020

codecov-commenter commented May 20, 2020 • edited

Codecov Report

medikoo left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bryan-hunter commented May 26, 2020

medikoo left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

medikoo left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

medikoo left a comment

Choose a reason for hiding this comment

bryan-hunter commented Jun 1, 2020

medikoo commented Jun 1, 2020

DavidEspinosa42 commented Jun 18, 2020

bryan-hunter commented Jun 18, 2020

DavidEspinosa42 commented Jun 18, 2020

bharat-tiwari commented Aug 8, 2020

DavidEspinosa42 commented Aug 10, 2020

mospina commented Oct 6, 2020

DavidEspinosa42 commented Oct 6, 2020

bryan-hunter commented May 20, 2020 •

edited

codecov-commenter commented May 20, 2020 •

edited