Serverless Compose?

[mnapoli]

Hi, Serverless Framework maintainer here. I stumbled upon this repository and I'm wondering if Serverless Framework Compose could be useful here?

Furthermore we're exploring possible integrations with Learn/Nx/Turborepo, looking to bounce ideas about that. So I thought I'd open an issue ^^

[NetanelBasal]

Hi!

Sure. I'm not very familiar with the new compose capabilities, but Nx (with this setup) already has everything included there and more.

Currently, it's annoying that we need to serve the different services over different ports. I could handle this by creating a proxy server, but it would be great if a solution were integrated into the serverless-offline plugin.

Feel free to explore the repository, share your ideas, and make suggestions. Thank you for your excellent work creating the serverless framework.

[mnapoli]

Indeed Nx takes care of the deployments right now, which is great. What Compose could bring to the table here is the ability to share outputs between services easily.

Instead of hardcoding a resources name (e.g. the DynamoDB table names/ARNs) or using CloudFormation exports/imports (which lock the stacks completely together, which is super painful when you want to change them), you can do:

# serverless-compose.yml
services:

  core:
    path: services/core

  users:
    path: services/users
    params:
      restApiId: ${core.restApiId}
      tableArn: ${core.tableArn}
  ...

This is YAML instead of TS, and very simple, but you get the idea. Compose doesn't use CloudFormation Imports, instead it fetches the CloudFormation output value directly, which is fast and doesn't lock the stacks together.

Also, once you have set up variables between each service, Compose will deploy them in order automatically.

We are looking into improvements that may also interest you:

1. defining parameters in serverless-compose.yml that apply to all services ([RFC] Common parameters serverless/compose#38):

# serverless-compose.yml

params:
  restApiId: ${core.restApiId}
  tableArn: ${core.tableArn}

services:

  core:
    path: services/core

  users:
    path: services/users
  ...

2. integrating with Nx/Lerna/etc. to deploy only "affected" services (i.e. services with changes)

Oh, also one final advantage to consider with Compose: you can run logs across all functions of all services: https://youtu.be/lPevgIk11Rg?t=783

[NetanelBasal]

I'm not sure what the added benefit is. Right now, I can share single source variables within any service. I have the power of Javascript.

Regarding the dependencies, I can tell Nx which service is dependent on which. (https://github.com/ngneat/nx-serverless/blob/main/services/todos/project.json#L60)

[mnapoli]

Sorry if I wasn't more clear, let me rephrase the benefit here: Compose would avoid:

• either hardcoding resource names (e.g. DynamoDB table name/ARN, particularly useful for bucket names because they need to be unique/random)

  nx-serverless/environments/environment.prod.ts

  Line 8 in 8185766

  tableName: `prod-AppTable`,

• or using CF Export/Import, which locks stacks together and is very painful to deal with

  nx-serverless/serverless.base.ts

  Lines 42 to 49 in e8afbd1

  	// @ts-ignore
  	restApiId: {
  	'Fn::ImportValue': `${env.name}-AppApiGW-restApiId`,
  	},
  	// @ts-ignore
  	restApiRootResourceId: {
  	'Fn::ImportValue': `${env.name}-AppApiGW-rootResourceId`,
  	},

Also my apologies, I forgot to mention that serverless-compose.yml can also be written in TS as serverless-compose.ts, so it doesn't mean giving up the power of Javascript.

[NetanelBasal]

It'll be interesting to see the same project using serverless-compose. Maybe we could do it in a new branch and compare the solutions.