SAM local Lambda run/debug with Lambda Layer reference in template.yaml

[wywarren]

Describe the bug
When invoking the hello world function that contains a layer by

sam local invoke --no-event

with cmd, it works fine, but when using the toolkit's Run Locally/Debug Locally, it cannot find the libraries inside the layers (or rather they're non existent)

To Reproduce

1. Create a template.yaml that looks something like this (modified from the boilerplate):

Transform: AWS::Serverless-2016-10-31
Description: >
  Test

  Sample SAM Template for Test

# More info about Globals: https://github.com/awslabs/serverless-application-model/blob/master/docs/globals.rst
Globals:
  Function:
    Timeout: 3

Resources:
  HelloWorldFunction:
    Type: AWS::Serverless::Function # More info about Function Resource: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#awsserverlessfunction
    Properties:
      CodeUri: hello_world/
      Handler: app.lambda_handler
      Runtime: python3.6
      Layers:
        - !Ref HelloWorldLayerFunction
      Events:
        HelloWorld:
          Type: Api # More info about API Event Source: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#api
          Properties:
            Path: /hello
            Method: get
  HelloWorldLayerFunction:
    Type: AWS::Serverless::LayerVersion
    Properties:
      LayerName: sam-app-dependencies
      ContentUri: hello_world_layer/
      CompatibleRuntimes:
        - python3.6

Outputs:
  # ServerlessRestApi is an implicit API created out of Events key under Serverless::Function
  # Find out more about other implicit resources you can reference within SAM
  # https://github.com/awslabs/serverless-application-model/blob/master/docs/internals/generated_resources.rst#api
  HelloWorldApi:
    Description: "API Gateway endpoint URL for Prod stage for Hello World function"
    Value: !Sub "https://${ServerlessRestApi}.execute-api.${AWS::Region}.amazonaws.com/Prod/hello/"
  HelloWorldFunction:
    Description: "Hello World Lambda Function ARN"
    Value: !GetAtt HelloWorldFunction.Arn
  HelloWorldFunctionIamRole:
    Description: "Implicit IAM Role created for Hello World function"
    Value: !GetAtt HelloWorldFunctionRole.Arn

Note that the layer is referenced using !Ref

2. Create a folder hello_world_layer in the project folder and put some of your own custom libraries or other pre-existing built pip libraries exported using the -t argument into the folder

3. In your app.py, try to import one of these custom libraries in your layer folder.

4. Run with
   sam local invoke --no-event

or try pressing Run Locally or Debug Locally

Expected behavior
Running sam local invoke --no-event, docker creates two images:

REPOSITORY          TAG                                   IMAGE ID            CREATED             SIZE
samcli/lambda       python3.6-e1dbf2fa602f528e69cf9be2f   a7ded16b7d3d        19 minutes ago      870MB
lambci/lambda       python3.6                             c62438d19947        10 days ago         856MB

which executes as expected and reaches the end of the hello world function spitting out the correct response json.

However, when pressing Run Locally (expected behaviour as per source code), it executes something like this:
sam local invoke awsToolkitSamLocalResource --template C:\Users\MyUser\AppData\Local\Temp\aws-toolkit-vscode\vsctksytYxA\output\template.yaml --event C:\Users\MyUser\AppData\Local\Temp\aws-toolkit-vscode\vsctksytYxA\event.json --env-vars C:\Users\MyUser\AppData\Local\Temp\aws-toolkit-vscode\vsctksytYxA\env-vars.json

Which creates a unique temp folder every time you press it and creates a single docker image (as opposed to two) with the repository lambci/lambda.

REPOSITORY          TAG                                   IMAGE ID            CREATED             SIZE
lambci/lambda       python3.6                             c62438d19947        10 days ago         856MB

Running it would result in an error stating that the library could not be found.

Workaround (...ish)

If I retain my original invoke string
sam local invoke awsToolkitSamLocalResource --template C:\Users\MyUser\AppData\Local\Temp\aws-toolkit-vscode\vsctksytYxA\output\template.yaml --event C:\Users\MyUser\AppData\Local\Temp\aws-toolkit-vscode\vsctksytYxA\event.json --env-vars C:\Users\MyUser\AppData\Local\Temp\aws-toolkit-vscode\vsctksytYxA\env-vars.json

and I navigate to the temp folder, I proceed to modify the two yaml files in there, namely

1. C:\Users\MyUser\AppData\Local\Temp\aws-toolkit-vscode\vsctksytYxA\input\input-template.yaml
2. C:\Users\MyUser\AppData\Local\Temp\aws-toolkit-vscode\vsctksytYxA\output\template.yaml

They're modified as follows:

template.yaml (before):

Globals:
  Function:
    Timeout: 3
Resources:
  awsToolkitSamLocalResource:
    Properties:
      CodeUri: awsToolkitSamLocalResource
      Handler: app.lambda_handler
      Runtime: python3.6
    Type: AWS::Serverless::Function

template.yaml (after):

Globals:
  Function:
    Timeout: 3
Resources:
  awsToolkitSamLocalResource:
    Properties:
      CodeUri: awsToolkitSamLocalResource
      Handler: app.lambda_handler
      Runtime: python3.6
      Layers:
        - !Ref HelloWorldLayerFunction
    Type: AWS::Serverless::Function
  HelloWorldLayerFunction:
    Type: AWS::Serverless::LayerVersion
    Properties:
      LayerName: sam-app-dependencies
      ContentUri: >-
        c:\Users\MyUser\Desktop\Git\lambda-vscode-test\hello_world_layer
      CompatibleRuntimes: 
        - python3.6

input-template.yaml (before):

Resources:
  awsToolkitSamLocalResource:
    Type: 'AWS::Serverless::Function'
    Properties:
      Handler: app.lambda_handler
      CodeUri: >-
        c:\Users\MyUser\Desktop\Git\lambda-vscode-test\hello_world
      Runtime: python3.6
Globals:
  Function:
    Timeout: 3

input-template.yaml (after):

Resources:
  awsToolkitSamLocalResource:
    Type: 'AWS::Serverless::Function'
    Properties:
      Handler: app.lambda_handler
      CodeUri: >-
        c:\Users\MyUser\Desktop\Git\lambda-vscode-test\hello_world
      Runtime: python3.6
      Layers:
        - !Ref HelloWorldLayerFunction
  HelloWorldLayerFunction:
    Type: AWS::Serverless::LayerVersion
    Properties:
      LayerName: sam-app-dependencies
      ContentUri: >-
        c:\Users\MyUser\Desktop\Git\lambda-vscode-test\hello_world_layer
      CompatibleRuntimes: 
        - python3.6
Globals:
  Function:
    Timeout: 3

With these two dependencies in place, executing the previous invoke works as expected.

Desktop:

• OS: Windows 10 Pro
• Visual Studio Code Version: 1.37.1
• AWS Toolkit for Visual Studio Code Version: 1.0.0

Additional context
I've narrowed it down to a few things but have yet to find a way to modify the source to fix this but it seems the following is currently true:

1. The layers in my project do not get copied into the build/mount folder for running locally.
2. The layers in my project do not get referenced anywhere in the files in my temp folder.

[bryceitoc9]

Labeling this as a feature request. This is a case we didn't account for: the temporary templates specifically only copy the resource of which the handler is a part; unfortunately, this means that the layer also won't get copied over.

[joshpeng]

I experience the same end result too if I use a pre-packaged .zip in the layer's ContentUri.

SomeLayer:
  Type: AWS::Serverless::LayerVersion
  Properties:
    ...
    ContentUri: somelayer-Python37.zip

Another situation (not sure if related) but if my Lambda's Layers is set to an ARN on my cloud. It doesn't pull it down for local caching.

SomeFunction:
  Type: AWS::Serverless::Function
  Properties:
    Runtime: python3.7
    ...
    Layers:
      - !Sub arn:aws:lambda:us-east-1:1111111111:layer:someLayerName:1

[dtelaroli]

+1

[justinmk3]

AWS Toolkit 1.12 (released today) includes a new SAM local Lambda debugging experience which leverages VS Code launch configurations. Please try it out, it may help with this issue.

Documentation: https://docs.aws.amazon.com/toolkit-for-vscode/latest/userguide/run-debug-sam-app.html

[bryceitoc9]

Looking into layers: looks like this still doesn't work with the newer experience if you're following the demo app here (which I did): https://aws.amazon.com/blogs/compute/working-with-aws-lambda-and-lambda-layers-in-aws-sam/

Specifically: our toolkit executes a sam build + a sam local invoke. The sam build fails when you follow these instructions as the application itself has no package.json, which means it can't be built. The main workaround I've found so far is to add a package.json which has no dependencies.