Recommended packaging setup for SAM stacks

[basvandriel]

I'm currently working on an project that uses multiple AWS SAM stacks.
Some include one or more lambda's and various resources using Python 3.12.

We want the code to be easily testable and decoupled from eachother. Also, we want to keep our deployed lambda functions small, we don't want to deploy the whole project.
Per stack, I was thinking the underneath structure.

# lambda one code
~/project_sam_stack_one/template.yaml
~/project_sam_stack_one/src/lambda_one/__init__.py
~/project_sam_stack_one/src/lambda_one/module_one.py
~/project_sam_stack_one/src/lambda_one/module_one.py
~/project_sam_stack_one/src/lambda_one/somedir/__init__.py
~/project_sam_stack_one/src/lambda_one/somedir/module_nested.py

# lambda two code
~/project_sam_stack_one/src/lambda_two/__init__.py
~/project_sam_stack_one/src/lambda_two/module_one.py
~/project_sam_stack_one/src/lambda_two/module_one.py
~/project_sam_stack_one/src/lambda_two/somedir/__init__.py
~/project_sam_stack_one/src/lambda_two/somedir/module_nested.py

# shared library
~/project_sam_stack_one/src/shared_code/__init__.py
~/project_sam_stack_one/src/shared_code/postgres.py

# testing 
~/project_sam_stack_one/tests/...

# packaging to make the project installable and easy to test
~/project_sam_stack_one/pyprojerct.toml

When then deploying the lamdbas, to ensure we don'at deploy the full project, we set the CodeUri to each of the lambdas:

~/project_sam_stack_one/src/lambda_two/

This means, from within the runtime of the lambda, the PYTHONPATH will be set to the CodeUri. This breaks.

The option I can think of: doubling the directory structure to simulate the package being installedModify the PYTHONPATH and double your directory structure to
For this approach, the lambda packages are doubled. This is inspired by this article, doubling the directory structure.

# lambda one code
~/project_sam_stack_one/template.yaml
~/project_sam_stack_one/src/lambda_one/__init__.py
~/project_sam_stack_one/src/lambda_one/module_one.py
~/project_sam_stack_one/src/lambda_one/module_one.py
~/project_sam_stack_one/src/lambda_one/somedir/__init__.py
~/project_sam_stack_one/src/lambda_one/somedir/module_nested.py

# lambda two code
~/project_sam_stack_one/src/lambda_two/__init__.py
~/project_sam_stack_one/src/lambda_two/module_one.py
~/project_sam_stack_one/src/lambda_two/module_one.py
~/project_sam_stack_one/src/lambda_two/somedir/__init__.py
~/project_sam_stack_one/src/lambda_two/somedir/module_nested.py

# shared library
~/project_sam_stack_one/src/shared_code/__init__.py
~/project_sam_stack_one/src/shared_code/postgres.py

# testing 
~/project_sam_stack_one/tests/...

# packaging to make the project installable and easy to test
~/project_sam_stack_one/pyprojerct.toml

For local development, this requires the PYTHONPATH to be modified, but feels weird with importing.

Questions

• Aside from the list above, are there any othere recommended approaches?
• Can we implement something in the application to ease this process?

[vicheey]

Hi @basvandriel,

Thanks for raising this question about packaging SAM stacks with shared code! This is a common challenge when building multi-function serverless applications, and there are several good approaches to solve it.

I'd recommend using Lambda Layers for your shared code - this is the most AWS-native solution and addresses all your requirements.

Here is some wiki about Lambda Layer:

• Managing Lambda dependencies with layers - AWS LambdaManaging Lambda dependencies with layers - AWS Lambda
• Using AWS SAM with layers - AWS LambdaUsing AWS SAM with layers - AWS Lambda

Lambda layer is a resource in SAM template and managed independently from you functions. Here is a basic example on how to specify a layer in template.yaml:

SAM Template:
yaml
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31

Globals:
  Function:
    Runtime: python3.12
    Layers:
      - !Ref SharedCodeLayer

Resources:
  SharedCodeLayer:
    Type: AWS::Serverless::LayerVersion
    Properties:
      LayerName: !Sub "${AWS::StackName}-shared-code"
      ContentUri: src/shared_code/
      CompatibleRuntimes:
        - python3.12
    Metadata:
      BuildMethod: python3.12

  LambdaOne:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: src/lambda_one/
      Handler: app.lambda_handler

  LambdaTwo:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: src/lambda_two/
      Handler: app.lambda_handler