Skip to content

feat: Esbuild Implementation Changes #322

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 11 commits into from
Feb 4, 2022
Merged

feat: Esbuild Implementation Changes #322

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
0e8d3f5
Use scratch dir. Read props from options
mildaniel Feb 2, 2022
2688521
Docstrings and black reformat
mildaniel Feb 2, 2022
2412760
Add tests
mildaniel Feb 2, 2022
bf80252
Add logic for accepting implicit entry points
mildaniel Feb 3, 2022
a7b405e
Uncomment feature flag
mildaniel Feb 3, 2022
35bbf78
Merge branch 'develop' into esbuild-in-scratch
mildaniel Feb 3, 2022
68394e0
Black reformat
mildaniel Feb 3, 2022
e3982cf
Merge branch 'develop' into esbuild-in-scratch
mndeveci Feb 3, 2022
66ef7c9
Address comments
mildaniel Feb 3, 2022
c86eed7
Merge branch 'esbuild-in-scratch' of github.com:mildaniel/aws-lambda-…
mildaniel Feb 3, 2022
f20dac3
Fix integration test
mildaniel Feb 4, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
161 changes: 0 additions & 161 deletions aws_lambda_builders/workflows/nodejs_npm/DESIGN.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -119,8 +119,6 @@ always downloading all the dependencies.
Packaging without a bundler does not require additional tools installed on the
development environment or CI systems, as it can just work with NPM.

Packaging with a bundler requires installing additional tools (eg `esbuild`).

#### handling local dependencies

Packaging without a bundler requires complex
Expand All @@ -130,9 +128,6 @@ initial version of the `npm_nodejs` builder, but due to issues with container
environments and how `aws-lambda-builders` mounts the working directory, it was
not added for several years, and likely will not be implemented soon.

Packaging with a bundler can handle local dependencies out of the box, since
it just traverses relative file liks.

#### including non-javascript files

Packaging without a bundler zips up entire contents of NPM packages.
Expand Down Expand Up @@ -165,7 +160,6 @@ required including a separate NPM package, or additional tools. Since Node 14,
stack trace translation can be [activated using an environment
variable](https://serverless.pub/aws-lambda-node-sourcemaps/)


### Implementation without a bundler

The general algorithm for preparing a node package for use on AWS Lambda
Expand Down Expand Up @@ -226,158 +220,3 @@ _(out of scope for the current version)_
To fully support dependencies that download or compile binaries for a target platform, this step
needs to be executed inside a Docker image compatible with AWS Lambda.
_(out of scope for the current version)_

### Implementation with a bundler

The general algorithm for preparing a node package for use on AWS Lambda
with a bundler (`esbuild` or `webpack`) is as follows.

#### Step 1: ensure production dependencies are installed

If the directory contains `package-lock.json` or `npm-shrinkwrap.json`,
execute [`npm ci`](https://docs.npmjs.com/cli/v7/commands/npm-ci). This
operation is designed to be faster than installing dependencies using `npm install`
in automated CI environments.

If the directory does not contain lockfiles, but contains `package.json`,
execute [`npm install --production`] to download production dependencies.

#### Step 2: bundle the main Lambda file

Execute `esbuild` to produce a single JavaScript file by recursively resolving
included dependencies, and optionally a source map.

Ensure that the target file name is the same as the entry point of the Lambda
function, so that there is no impact on the CloudFormation template.


### Activating the bundler workflow

Because there are advantages and disadvantages to both approaches (with and
without a bundler), the user should be able to choose between them. The default
is not to use a bundler (both because it's universally applicable and for
backwards compatibility). Node.js pakage manifests (`package.json`) allow for
custom properties, so a user can activate the bundler process by providing an
`aws_sam` configuration property in the package manifest. If this property is
present in the package manifest, and the sub-property `bundler` equals
`esbuild`, the Node.js NPM Lambda builder activates the bundler process.

Because the Lambda builder workflow is not aware of the main lambda function
definition, (the file containing the Lambda handler function) the user must
also specify the main entry point for bundling . This is a bit of an
unfortunate duplication with SAM Cloudformation template, but with the current
workflow design there is no way around it.

In addition, as a single JavaScript source package can contain multiple functions,
and can be included multiple times in a single CloudFormation template, it's possible
that there may be multiple entry points for bundling. SAM build executes the build
only once for the function in this case, so all entry points have to be bundled
at once.

The following example is a minimal `package.json` to activate the `esbuild` bundler
on a javascript file, starting from `lambda.js`. It will produce a bundled `lambda.js`
in the artifacts folder.

```json
{
"name": "nodeps-esbuild",
"version": "1.0.0",
"license": "APACHE2.0",
"aws_sam": {
"bundler": "esbuild",
"entry_points": ["lambda.js"]
}
}
```

#### Locating the esbuild binary

`esbuild` supports platform-independent binary distribution using NPM, by
including the `esbuild` package as a dependency. The Lambda builder should
first try to locate the binary in the Lambda code repository (allowing the
user to include a specific version). Failing that, the Lambda builder should
try to locate the `esbuild` binary in the `executable_search_paths` configured
for the workflow, then the operating system `PATH` environment variable.

The Lambda builder **should not** bring its own `esbuild` binary, but it should
clearly point to the error when one is not found, to allow users to configure the
build correctly.

In the previous example, the esbuild binary is not included in the package dependencies,
so the Lambda builder will use the system executable paths to search for it. In the
example below, `esbuild` is included in the package, so the Lambda builder should use it
directly.

```json
{
"name": "with-deps-esbuild",
"version": "1.0.0",
"license": "APACHE2.0",
"aws_sam": {
"bundler": "esbuild",
"entry_points": ["lambda.js"]
},
"devDependencies": {
"esbuild": "^0.11.23"
}
}
```

For a full example, see the [`with-deps-esbuild`](../../../tests/integration/workflows/nodejs_npm_esbuild/testdata/with-deps-esbuild/) test project.

#### Building typescript

`esbuild` supports bundling typescript out of the box and transpiling it to plain
javascript. The user just needs to point to a typescript file as the main entry point,
as in the example below. There is no transpiling process needed upfront.


```js
{
"name": "with-deps-esbuild-typescript",
"version": "1.0.0",
"license": "APACHE2.0",
"aws_sam": {
"bundler": "esbuild",
"entry_points": ["included.ts"]
},
"dependencies": {
"@types/aws-lambda": "^8.10.76"
},
"devDependencies": {
"esbuild": "^0.11.23"
}
}
```

For a full example, see the [`with-deps-esbuild-typescript`](../../../tests/integration/workflows/nodejs_npm_esbuild/testdata/with-deps-esbuild-typescript/) test project.

**important note:** esbuild does not perform type checking, so users wanting to ensure type-checks need to run the `tsc` process as part of their
testing flow before invoking `sam build`. For additional typescript caveats with esbuild, check out <https://esbuild.github.io/content-types/#typescript>.

#### Configuring the bundler

The Lambda builder invokes `esbuild` with sensible defaults that will work for the majority of cases. Importantly, the following three parameters are set by default

* `--minify`, as it [produces a smaller runtime package](https://esbuild.github.io/api/#minify)
* `--sourcemap`, as it generates a [source map that allows for correct stack trace reporting](https://esbuild.github.io/api/#sourcemap) in case of errors (see the [Error reporting](#error-reporting) section above)
* `--target es2020`, as it allows for javascript features present in Node 14

Users might want to tweak some of these runtime arguments for a specific project, for example not including the source map to further reduce the package size, or restricting javascript features to an older version. The Lambda builder allows this with optional sub-properties of the `aws_sam` configuration property.

* `target`: string, corresponding to a supported [esbuild target](https://esbuild.github.io/api/#target) property
* `minify`: boolean, defaulting to `true`
* `sourcemap`: boolean, defaulting to `true`

Here is an example that deactivates minification and source maps, and supports JavaScript features compatible with Node.js version 10.

```json
{
"aws_sam": {
"bundler": "esbuild",
"entry_points": ["included.ts"],
"target": "node10",
"minify": false,
"sourcemap": false
}
}
140 changes: 140 additions & 0 deletions aws_lambda_builders/workflows/nodejs_npm_esbuild/DESIGN.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,140 @@
## NodeJS - NPM Lambda Builder Using `esbuild`

### Scope

The scope for this builder is to take an existing
directory containing customer code, including a valid `package.json` manifest
specifying third-party dependencies. The builder will use NPM to include
production dependencies and exclude test resources in a way that makes them
deployable to AWS Lambda. It will then bundle the code using `esbuild` with the properties
passed in through the builder options field.

### Additional Tools

Packaging with a bundler requires installing additional tools (eg `esbuild`).

### Implementation with a bundler

The general algorithm for preparing a node package for use on AWS Lambda
with a bundler (`esbuild` or `webpack`) is as follows.

#### Step 1: ensure production dependencies are installed

If the directory contains `package-lock.json` or `npm-shrinkwrap.json`,
execute [`npm ci`](https://docs.npmjs.com/cli/v7/commands/npm-ci). This
operation is designed to be faster than installing dependencies using `npm install`
in automated CI environments.

If the directory does not contain lockfiles, but contains `package.json`,
execute [`npm install --production`] to download production dependencies.

#### Step 2: bundle the main Lambda file

Execute `esbuild` to produce a single JavaScript file by recursively resolving
included dependencies, and optionally a source map.

Ensure that the target file name is the same as the entry point of the Lambda
function.

### Activating the bundler workflow

The workflow can be activated by using the `("nodejs", "npm-esbuild", None)` Capability.
The distinguishing factor being the `npm-esbuild` dependency-manager property of the builder.

An entrypoint or entrypoints array must be included in the options passed
to Lambda Builders for this workflow to succeed.

The following example is a minimal options object that can be passed to
the esbuild workflow, starting from `lambda.js`. It will produce a bundled `lambda.js`
in the artifacts folder.

```json
{
"options": {
"entry_points": ["lambda.js"]
}
}
```

#### Locating the esbuild binary

`esbuild` supports platform-independent binary distribution using NPM, by
including the `esbuild` package as a dependency. The Lambda builder should
first try to locate the binary in the Lambda code repository (allowing the
user to include a specific version). Failing that, the Lambda builder should
try to locate the `esbuild` binary in the `executable_search_paths` configured
for the workflow, then the operating system `PATH` environment variable.

The Lambda builder **should not** bring its own `esbuild` binary, but it should
clearly point to the error when one is not found, to allow users to configure the
build correctly.

In the previous example, the esbuild binary is not included in the package dependencies,
so the Lambda builder will use the system executable paths to search for it. In the
example below, `esbuild` is included in the package, so the Lambda builder should use it
directly.

```json
{
"name": "with-deps-esbuild",
"version": "1.0.0",
"license": "APACHE2.0",
"devDependencies": {
"esbuild": "^0.11.23"
}
}
```

For a full example, see the [`with-deps-esbuild`](../../../tests/integration/workflows/nodejs_npm_esbuild/testdata/with-deps-esbuild/) test project.

#### Building typescript

`esbuild` supports bundling typescript out of the box and transpiling it to plain
javascript. The user just needs to point to a typescript file as the main entry point,
as in the example below. There is no transpiling process needed upfront.

If no file type is provided for the entrypoint, esbuild will first look for a
TypeScript file, and the a JavaScript file with the given filename.

```js
{
"name": "with-deps-esbuild-typescript",
"version": "1.0.0",
"license": "APACHE2.0",
"dependencies": {
"@types/aws-lambda": "^8.10.76"
},
"devDependencies": {
"esbuild": "^0.11.23"
}
}
```

For a full example, see the [`with-deps-esbuild-typescript`](../../../tests/integration/workflows/nodejs_npm_esbuild/testdata/with-deps-esbuild-typescript/) test project.

**important note:** esbuild does not perform type checking, so users wanting to ensure type-checks need to run the `tsc` process as part of their
testing flow before invoking `sam build`. For additional typescript caveats with esbuild, check out <https://esbuild.github.io/content-types/#typescript>.

#### Configuring the bundler

The Lambda builder invokes `esbuild` with sensible defaults that will work for the majority of cases. Importantly, the following three parameters are set by default

* `--minify`, as it [produces a smaller runtime package](https://esbuild.github.io/api/#minify)
* `--sourcemap`, as it generates a [source map that allows for correct stack trace reporting](https://esbuild.github.io/api/#sourcemap) in case of errors (see the [Error reporting](#error-reporting) section above)
* `--target es2020`, as it allows for javascript features present in Node 14

Users might want to tweak some of these runtime arguments for a specific project, for example not including the source map to further reduce the package size, or restricting javascript features to an older version. The Lambda builder allows this with optional sub-properties of the `aws_sam` configuration property.

* `target`: string, corresponding to a supported [esbuild target](https://esbuild.github.io/api/#target) property
* `minify`: boolean, defaulting to `true`
* `sourcemap`: boolean, defaulting to `true`

Here is an example that deactivates minification and source maps, and supports JavaScript features compatible with Node.js version 10.

```json
{
"entry_points": ["included.ts"],
"target": "node10",
"minify": false,
"sourcemap": false
}
Loading