Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
cf4e089
feat: Migrate to TypeScript (#882)
seb-cr Jan 3, 2023
036e86a
ci: Run publish job on `beta` branch (#1017)
seb-cr Jan 4, 2023
3845b0f
fix: Make `@types/aws-lambda` a dependency (#1029)
seb-cr Jan 11, 2023
2cec559
docs: Fix typo in readme (#1041)
seb-cr Jan 23, 2023
b13a449
docs: Show how to install the beta version (#1040)
seb-cr Jan 23, 2023
d8857d0
feat: Export `RESPONSE_HEADERS` and fix its type (#1039)
seb-cr Jan 23, 2023
b5294a0
Merge branch 'master' into beta
seb-cr Jan 23, 2023
cb4863e
docs: Add note about disabling minification
seb-cr Aug 9, 2023
b73313b
feat: Propagate config type to dependencies (#1112)
seb-cr Sep 14, 2023
82a5aa8
fix: Add missing space in DI error message (#1137)
seb-cr Sep 18, 2023
d6a3479
fix: Dependency updates (#1138)
seb-cr Sep 18, 2023
df570b0
docs: Clarify some things in v2 migration guide
seb-cr Sep 19, 2023
fa71459
docs: Consistently coerce queue names to string
seb-cr Sep 19, 2023
15742f4
fix: Allow synchronous handlers (#1159)
seb-cr Sep 28, 2023
fcdff3e
feat: Replace Epsagon with Lumigo (#1168)
seb-cr Oct 19, 2023
beea2ea
feat: Throw if there are conflicting dependency names (#1155)
seb-cr Nov 3, 2023
481d430
feat: Replace `StatusModel` with types (#1172)
seb-cr Nov 3, 2023
141898e
feat: Return lowercase header keys from `getAllHeaders` (#1187)
seb-cr Dec 11, 2023
6a14aca
feat: `SQSMessageModel` validates required fields of message (#1188)
seb-cr Dec 11, 2023
baa4966
fix: Improve `SQSMessageModel` types (#1189)
seb-cr Dec 11, 2023
4581d61
refactor: Rename SQS message model class (#1191)
seb-cr Dec 15, 2023
7bac68d
docs: Add instructions to `SQSMessageModel` (#1190)
seb-cr Dec 18, 2023
dca7620
feat: Throw by default from failed SQS publish (#1194)
seb-cr Jan 19, 2024
fdda2d5
Merge branch 'master' into beta
seb-cr Feb 29, 2024
2047e2d
fix: More dependency updates
seb-cr Feb 29, 2024
d4f275b
chore: Revert ts-jest to v29.1.1
seb-cr Feb 29, 2024
b42e74f
test: Add type tests covering `di.get` and `SQSService` (#1198)
seb-cr Mar 25, 2024
87f765f
docs: Remove remaining references to Epsagon (#1197)
seb-cr Mar 28, 2024
0d55ff1
feat: Runtime config validation (#1199)
seb-cr Mar 28, 2024
59b2450
docs: Remove refs to beta ahead of 2.0.0 release (#1200)
seb-cr Apr 2, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 3 additions & 4 deletions .eslintrc.yml
Original file line number Diff line number Diff line change
@@ -1,10 +1,8 @@
root: true

extends:
- '@comicrelief/eslint-config'
- '@comicrelief/eslint-config/mixins/jsdoc'

parser: '@babel/eslint-parser'
- '@comicrelief/eslint-config/mixins/base'
- '@comicrelief/eslint-config/mixins/ts'

ignorePatterns:
- node_modules
Expand All @@ -14,3 +12,4 @@ ignorePatterns:

rules:
unicorn/prefer-node-protocol: off
'@typescript-eslint/no-explicit-any': off
23 changes: 22 additions & 1 deletion .github/workflows/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -49,11 +49,32 @@ jobs:
env:
CI: true

type-test:
name: Type test
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v2

- uses: actions/setup-node@v1
with:
node-version: 14

- name: Restore cache
uses: actions/cache@v2
with:
path: '**/node_modules'
key: ${{ runner.os }}-modules-${{ hashFiles('**/yarn.lock') }}

- run: yarn install

- run: yarn test:types

publish:
name: Publish
runs-on: ubuntu-latest
needs: unit-test
if: github.ref == 'refs/heads/master'
if: github.ref == 'refs/heads/master' || github.ref == 'refs/heads/beta'
steps:
- name: Checkout
uses: actions/checkout@v2
Expand Down
6 changes: 6 additions & 0 deletions .nycrc.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
extends: '@istanbuljs/nyc-config-typescript'

all: true
check-coverage: true
include:
- src/**
255 changes: 206 additions & 49 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,97 +4,254 @@
[![semantic-release](https://img.shields.io/badge/%20%20%F0%9F%93%A6%F0%9F%9A%80-semantic--release-e10079.svg)](https://github.com/semantic-release/semantic-release)
[![semantic-release](https://badge.fury.io/js/%40comicrelief%2Flambda-wrapper.svg)](https://www.npmjs.com/package/@comicrelief/lambda-wrapper)

When writing Serverless endpoints, we have found ourselves replicating a lot of boiler plate code to do basic actions, such as reading request variables or writing to SQS. The aim of this package is to provide a wrapper for our Lambda functions, to provide some level of dependency and configuration injection and to reduce time spent on project setup.
When writing Serverless applications, we have found ourselves replicating a lot of boilerplate code to do basic actions, such as reading request data or sending messages to SQS. The aim of this package is to provide a wrapper for our Lambda functions, to provide some level of dependency and configuration injection and to reduce time spent on project setup.

> 🚀 [Lambda Wrapper v2 is now available in beta!](https://github.com/comicrelief/lambda-wrapper/tree/beta) This major release includes TypeScript support and some significant design changes.
If you're coming from v1 and updating to v2, check out the [v2 migration guide](docs/migration/v2.md).

## Installation & usage
## Getting started

Install via npm:

```bash
npm install --save @comicrelief/lambda-wrapper
```

Or via yarn:
Install via npm or Yarn:

```bash
npm i @comicrelief/lambda-wrapper
# or
yarn add @comicrelief/lambda-wrapper
```

You can then wrap your lambdas as follows.
You can then wrap your Lambda handler functions like this:

```js
import {
LambdaWrapper,
```ts
// src/action/Hello.ts
import lambdaWrapper, {
ResponseModel,
RequestService,
} from '@comicrelief/lambda-wrapper';

export default LambdaWrapper({}, (di, request, done) => {
const response = new ResponseModel({}, 200, `hello ${request.get('name', 'nobody')}`);
done(null, response.generate());
export default lambdaWrapper.wrap(async (di) => {
const request = di.get(RequestService);
return ResponseModel.generate(
{},
200,
`hello ${request.get('name', 'nobody')}`,
);
});
```

Here we've used the default export `lambdaWrapper` which is a preconfigured instance that can be used out of the box. You'll likely want to add your own dependencies and service config using the `configure` method:

```ts
// src/config/LambdaWrapper.ts
import lambdaWrapper from '@comicrelief/lambda-wrapper';

export default lambdaWrapper.configure({
// your config goes here
});
```

`configure` returns a new Lambda Wrapper instance with the given configuration. You'll want to export it and then use this when wrapping your handler functions.

Read the next section to see what goes inside the config object!

If you want to start from scratch without the built-in dependencies, you can use the `LambdaWrapper` constructor directly.

```ts
// src/config/LambdaWrapper.ts
import { LambdaWrapper } from '@comicrelief/lambda-wrapper';

export default new LambdaWrapper({
// your config goes here
});
```

## Dependencies

Lambda Wrapper comes with some commonly used dependencies built in:

- [HTTPService](docs/services/HTTPService.md)
- [LoggerService](docs/services/LoggerService.md)
- [RequestService](docs/services/RequestService.md)
- [SQSService](docs/services/SQSService.md)
- [TimerService](docs/services/TimerService.md)

Access these via dependency injection. You've already seen an example of this where we got `RequestService`. Pass the dependency class to `di.get()` to get its instance:

```ts
export default lambdaWrapper.wrap(async (di) => {
const request = di.get(RequestService);
const sqs = di.get(SQSService);
// ...
});
```

To add your own dependencies, first extend `DependencyAwareClass`.

```ts
// src/services/MyService.ts
import { DependencyAwareClass } from '@comicrelief/lambda-wrapper';

export default class MyService extends DependencyAwareClass {
doSomething() {
// ...
}
}
```

If you need to override the constructor, it must take a `DependencyInjection` instance and pass it to `super`.

```ts
export default class MyService extends DependencyAwareClass {
constructor(di: DependencyInjection) {
super(di);
// now do your other constructor stuff
}
}
```

Then add it to your Lambda Wrapper configuration in the `dependencies` key.

```ts
// src/config/LambdaWrapper.ts
import lambdaWrapper from '@comicrelief/lambda-wrapper';

import MyService from '@/src/services/MyService';

export default lambdaWrapper.configure({
dependencies: {
MyService,
},
});
```

Now you can use it inside your handler functions and other dependencies!

```ts
// src/action/DoSomething.ts
import lambdaWrapper from '@/src/config/LambdaWrapper';
import MyService from '@/src/services/MyService';

export default lambdaWrapper.wrap(async (di) => {
di.get(MyService).doSomething();
});
```

## Serverless Offline & SQS Emulation
## Service config

Serverless Offline only emulates API Gateway and Lambda, so publishing an SQS message would use the real SQS queue and trigger the consumer function (if any) in AWS. When working with offline code, you often want the local functions to be invoked instead.
Some dependencies need their own config. This goes in per-service keys within your Lambda Wrapper config. For an example, see [SQSService](docs/services/SQSService.md) which uses the `sqs` key.

Offline SQS behaviour can be configured by setting the `LAMBDA_WRAPPER_OFFLINE_SQS_MODE` environment variable. Available modes are:
```ts
export default lambdaWrapper.configure({
dependencies: {
// your dependencies
},
sqs: {
// your SQSService config
},
// ... other configs ...
});
```

To use config with your own dependencies, you need to do three things:

1. Define the key and type of your config object.

Using `SQSService` as an example, we have the `sqs` key which has the `SQSServiceConfig` type:

```ts
export interface SQSServiceConfig {
queues?: Record<string, string>;
queueConsumers?: Record<string, string>;
}
```

2. Define a type that can be applied to a Lambda Wrapper config.

- `direct` (the default): invokes the consumer function directly via an offline Lambda endpoint
- `local`: send messages to an offline SQS endpoint, such as Localstack
- `aws`: no special handling of SQS offline; messages will be sent to AWS
This simply combines the key and type defined in step 1. Conventionally we name these `With...` types.

Details of each mode are documented in the sections below. When you send a message using `SQSService.prototype.publish`, it will check which mode to use and dispatch the message appropriately. These modes take effect only when running offline (as defined by `DependencyInjection.prototype.isOffline`). In a deployed environment, SQS messages will always be sent to AWS SQS.
```ts
export interface WithSQSServiceConfig {
sqs?: SQSServiceConfig;
}
```

### Direct Lambda mode
In the case of `SQSService`, the `sqs` key is optional because this dependency is included by default and not all applications need it. If your dependency requires config in order to work, you can make this a required key.

This is the default mode if `LAMBDA_WRAPPER_OFFLINE_SQS_MODE` is not set. A Lambda client will be created and the message will be delivered to the offline Lambda endpoint, effectively running the consumer function _immediately_ as part of the original Lambda invocation. This works very well in the offline environment because invoking a Lambda function will trigger its whole (local) execution tree.
3. In your dependency constructor, cast the config to this type.

To take advantage of SQS emulation, you will need to define the following in the implementing service:
```ts
export default class SQSService extends DependencyAwareClass {
constructor(di: DependencyInjection) {
super(di);

**QUEUE_CONSUMERS**
const config = (this.di.config as WithSQSServiceConfig).sqs;
// Bear in mind that because the `sqs` key is optional, the type of
// `config` will be `SQSServiceConfig | undefined`. Take care when
// accessing its properties! You can use optional chaining:
const queues = config?.queues || {};
// ...
}
}
```

In your `src/Config/Configuration` define a `QUEUE_CONSUMERS` object. `QUEUE_CONSUMERS` will map the queue name to the fully qualified `FunctionName` that we want to trigger when messages are published to that queue.
When you go to configure your Lambda Wrapper, you can now include your dependency's config type in the generic for `configure` to get IntelliSense completions and type checking for your config keys.

You will need to export `QUEUE_CONSUMERS` as part of your default export, alongside `DEFINITIONS`, `DEPENDENCIES`, `QUEUES`, `QUEUE_DEFINITIONS`, etc.
```ts
lambdaWrapper.configure<WithSQSServiceConfig>({
sqs: {
queues: 42 // Oops! This will be flagged as a type error by TypeScript
},
});
```

A `Configuration` example can be found in the `serverless-prize-platform` repository [here](https://github.com/comicrelief/serverless-prize-platform/blob/master/src/Config/Configuration.js).
You can combine types for multiple dependencies if needed using `&`:

**process.env.SERVICE_LAMBDA_URL**
```ts
lambdaWrapper.configure<WithSQSServiceConfig & WithOtherServiceConfig>({
sqs: {
// SQSService config
},
other: {
// OtherService config
},
});
```

While creating the Lambda client, we need to point it to our offline environment. LambdaWrapper will take care of the specifics, but it will need to know the Lambda endpoint URL. This _can_ and _must_ be specified via the `SERVICE_LAMBDA_URL` environment variable.
## Monitoring

The URL is likely to be your localhost URL and the next available port from the offline API Gateway. So, if you are running Serverless Offline on `http://localhost:3001`, the Lambda URL is likely to be `http://localhost:3002`. You can check the port in the output during Serverless Offline startup by looking for the following line:
At Comic Relief we use [Lumigo](https://lumigo.io/) for monitoring and observability of our deployed services. Lambda Wrapper includes the Lumigo tracer to allow us to tag traces with custom labels and metrics ([execution tags](https://docs.lumigo.io/docs/execution-tags)).

offline: Offline [http for lambda] listening on http://localhost:3002
Lumigo integration works out-of-the-box with Lumigo's [auto-trace feature](https://docs.lumigo.io/docs/serverless-applications#automatic-instrumentation). If you prefer manual tracing, enable it by setting `LUMIGO_TRACER_TOKEN` in your Lambda environment variables.

#### Caveats
And if you don't use Lumigo, don't worry, their tracer will not be instantiated in your functions and no calls will be made to their servers unless `LUMIGO_TRACER_TOKEN` is set.

1. You will be running the SQS-triggered lambdas in the same Serverless Offline context as your triggering lambda. Expect logs from both lambdas in the Serverless Offline output.
## Notes

2. If you await `sqs.publish` you will effectively wait until all SQS-triggered lambdas (and possibly their own SQS-triggered lambdas) have all completed. This is necessary to avoid any pending execution (i.e. the lambda terminating before its async processes are completed).
Lambda Wrapper's dependency injection relies on class names being preserved. If your build process includes minifying or uglifying your code, you'll need to disable these transformations.

3. If the triggered lambda incurs an exception, this will be propagated upstream, effectively killing the execution of the calling lambda.
In many of our projects we use `serverless-webpack` to bundle service code prior to deployment. To disable name mangling, set `optimization.minimize` to `false` in your webpack config:

```js
// webpack.config.js
module.exports = {
// ...
optimization: {
minimize: false,
},
```

### Local SQS mode
## Development

Use this mode by setting `LAMBDA_WRAPPER_OFFLINE_SQS_MODE=local`. Messages will still be sent to an SQS queue, but using a locally simulated version instead of AWS. This allows you to test your service using a tool like Localstack.
### Testing

By default, messages will be sent to a SQS service running on `localhost:4576`. If you need to change the hostname, you can set `process.env.LAMBDA_WRAPPER_OFFLINE_SQS_HOST`.
Also, if you need to change the port, you can set `process.env.LAMBDA_WRAPPER_OFFLINE_SQS_PORT`.
Run `yarn test` to run the unit tests, and `yarn test:types` to run the type tests.

### AWS SQS mode
When writing a bugfix, start by writing a test that reproduces the problem. It should fail with the current version of Lambda Wrapper, and pass once you've implemented the fix.

Use this mode by setting `LAMBDA_WRAPPER_OFFLINE_SQS_MODE=aws`. Messages will be sent to the real queue in AWS. This mode is useful when a queue is consumed by an external service, rather than another function in the service under test.
When adding a feature, ensure it's covered by tests that adequately define its behaviour.

In order for queue URLs to be correctly constructed, you must either:
### Linting

- set `AWS_ACCOUNT_ID` to the account ID that hosts your queue; or
- invoke offline functions via the Lambda API, passing a context that contains a realistic `invokedFunctionArn` including the account ID.
Run `yarn lint` to check code style complies to our standard. Many problems can be auto-fixed using `yarn lint --fix`.

## Semantic release
### Releases

Release management is automated using [semantic-release](https://www.npmjs.com/package/semantic-release).
13 changes: 0 additions & 13 deletions babel.config.js

This file was deleted.

Loading