Build fails using --use-container when a Java lambda function has a dependency from a Java layer in the same project.

[bogdan-romaniv]

Description:
When trying to build a lambda function that reference a layer in the same project using

sam build --use-container

SAM will fail to include the transitive dependencies between the lambda function and the layer

Steps to reproduce:
clone this repo "https://github.com/bogdan-romaniv/sam-java-layer-issue.git"

With this commands

sam build
sam local invoke HelloWorldFunction
sam local start-api
sam deploy

Observed result:
Everything OK

With this command

sam build --use-container

Observed result:
PS C:\XXX\XXX\lambda-layer-issues> sam build --use-container
Starting Build inside a container
Your template contains a resource with logical ID "ServerlessRestApi", which is a reserved logical ID in AWS SAM. It could result in unexpected behaviors and is not recommended.
Building layer 'LambdaLayer'

Fetching public.ecr.aws/sam/build-java11:latest-x86_64 Docker container image...........................................................................................................................................................
........................................................................................................................................................................................................................................
........................................................................................................................................................................................................................................
...................................................................................................................................................................................................................
Mounting C:\XXX\XXX\lambda-layer-issues\lambda-layer as /tmp/samcli/source:ro,delegated inside runtime container
Running JavaMavenWorkflow:CopySource
Running JavaMavenWorkflow:MavenBuild
Running JavaMavenWorkflow:MavenCopyDependency
Running JavaMavenWorkflow:MavenCopyLayerArtifacts
Building codeuri: C:\Cursos\Issues\lambda-layer-issues\HelloWorldFunction runtime: java11 metadata: {} architecture: x86_64 functions: HelloWorldFunction

Fetching public.ecr.aws/sam/build-java11:latest-x86_64 Docker container image......
Mounting C:\XXX\XXX\lambda-layer-issues\HelloWorldFunction as /tmp/samcli/source:ro,delegated inside runtime container

Build Failed
Error: JavaMavenWorkflow:MavenBuild - Maven Failed: [INFO] Scanning for projects...
[INFO]
[INFO] -------------------< org.example.lambda:HelloWorld >--------------------
[INFO] Building A sample Hello World created for SAM CLI. 1.0
[INFO] --------------------------------[ jar ]---------------------------------
Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-clean-plugin/2.5/maven-clean-plugin-2.5.pom
Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-clean-plugin/2.5/maven-clean-plugin-2.5.pom (3.9 kB at 21 kB/s)
Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-plugins/22/maven-plugins-22.pom
Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-plugins/22/maven-plugins-22.pom (13 kB at 543 kB/s)
Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/21/maven-parent-21.pom
Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/21/maven-parent-21.pom (26 kB at 1.0 MB/s)
Downloading from central: https://repo.maven.apache.org/maven2/org/apache/apache/10/apache-10.pom
Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/apache/10/apache-10.pom (15 kB at 231 kB/s)
Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-clean-plugin/2.5/maven-clean-plugin-2.5.jar
Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-clean-plugin/2.5/maven-clean-plugin-2.5.jar (25 kB at 1.1 MB/s)
Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-resources-plugin/2.6/maven-resources-plugin-2.6.pom
Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-resources-plugin/2.6/maven-resources-plugin-2.6.pom (8.1 kB at 507 kB/s)
Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-plugins/23/maven-plugins-23.pom
Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-plugins/23/maven-plugins-23.pom (9.2 kB at 484 kB/s)
Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/22/maven-parent-22.pom
Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/22/maven-parent-22.pom (30 kB at 1.6 MB/s)
Downloading from central: https://repo.maven.apache.org/maven2/org/apache/apache/11/apache-11.pom
Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/apache/11/apache-11.pom (15 kB at 411 kB/s)
Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-resources-plugin/2.6/maven-resources-plugin-2.6.jar
Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-resources-plugin/2.6/maven-resources-plugin-2.6.jar (30 kB at 1.4 MB/s)
Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-compiler-plugin/3.1/maven-compiler-plugin-3.1.pom
Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-compiler-plugin/3.1/maven-compiler-plugin-3.1.pom (10 kB at 601 kB/s)
Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-plugins/24/maven-plugins-24.pom
Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-plugins/24/maven-plugins-24.pom (11 kB at 600 kB/s)
Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/23/maven-parent-23.pom
Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/23/maven-parent-23.pom (33 kB at 1.1 MB/s)
Downloading from central: https://repo.maven.apache.org/maven2/org/apache/apache/13/apache-13.pom
Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/apache/13/apache-13.pom (14 kB at 873 kB/s)
Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-compiler-plugin/3.1/maven-compiler-plugin-3.1.jar
Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-compiler-plugin/3.1/maven-compiler-plugin-3.1.jar (43 kB at 2.5 MB/s)
Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-surefire-plugin/2.12.4/maven-surefire-plugin-2.12.4.pom
Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-surefire-plugin/2.12.4/maven-surefire-plugin-2.12.4.pom (10 kB at 655 kB/s)
Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire/2.12.4/surefire-2.12.4.pom
Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/surefire/surefire/2.12.4/surefire-2.12.4.pom (14 kB at 985 kB/s)
Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-surefire-plugin/2.12.4/maven-surefire-plugin-2.12.4.jar
Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-surefire-plugin/2.12.4/maven-surefire-plugin-2.12.4.jar (30 kB at 1.6 MB/s)
Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-jar-plugin/2.4/maven-jar-plugin-2.4.pom
Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-jar-plugin/2.4/maven-jar-plugin-2.4.pom (5.8 kB at 417 kB/s)
Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-jar-plugin/2.4/maven-jar-plugin-2.4.jar
Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-jar-plugin/2.4/maven-jar-plugin-2.4.jar (34 kB at 2.1 MB/s)
Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-shade-plugin/3.2.4/maven-shade-plugin-3.2.4.pom
Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-shade-plugin/3.2.4/maven-shade-plugin-3.2.4.pom (11 kB at 359 kB/s)
Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-plugins/34/maven-plugins-34.pom
Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-plugins/34/maven-plugins-34.pom (11 kB at 534 kB/s)
Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/34/maven-parent-34.pom
Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/34/maven-parent-34.pom (43 kB at 1.9 MB/s)
Downloading from central: https://repo.maven.apache.org/maven2/org/apache/apache/23/apache-23.pom
Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/apache/23/apache-23.pom (18 kB at 1.0 MB/s)
Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-shade-plugin/3.2.4/maven-shade-plugin-3.2.4.jar
Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-shade-plugin/3.2.4/maven-shade-plugin-3.2.4.jar (134 kB at 4.1 MB/s)
Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-install-plugin/2.4/maven-install-plugin-2.4.pom
Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-install-plugin/2.4/maven-install-plugin-2.4.pom (6.4 kB at 425 kB/s)
Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-install-plugin/2.4/maven-install-plugin-2.4.jar
Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-install-plugin/2.4/maven-install-plugin-2.4.jar (27 kB at 1.7 MB/s)
[WARNING] The POM for org.example.layer:lambda-layer:jar:1.0-SNAPSHOT is missing, no dependency information available
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1.081 s
[INFO] Finished at: 2022-12-06T15:47:50Z
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal on project HelloWorld: Could not resolve dependencies for project org.example.lambda:HelloWorld:jar:1.0: Could not find artifact org.example.layer:lambda-layer:jar:1.0-SNAPSHOT -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException
Running JavaMavenWorkflow:CopySource
Running JavaMavenWorkflow:MavenBuild

Expected result:
Being able to build in the same way as sam build

Additional environment details (Ex: Windows, Mac, Amazon Linux etc)
OS: Windows
sam --version: SAM CLI, version 1.63.0
AWS region: eu-west-1

[jfuss]

@bogdan-romaniv When running a build in a container, we only mount the CodeUri directory and anything maven might find locally will not work in the container.

Unfortunately, this is expected behavior today.

I am not familiar with maven/Java but is there a way you can define the dependency that would allow build to work? You shouldn't need the layer code as a dependency until runtime correct?

[bogdan-romaniv]

Hi @jfuss, thanks for the answer.

Then in a project where a layer is defined, if the build inside the container works differently than it works outside, you can never deploy a project with many lambdas using a Layer (all in the same project) on a CI/CD with GitHub actions or Jenkins.

What would the life cycle of a project be like at the enterprise level using sam if I can't use layers? Should layers be deployed in their separate projects and use the layer ARN in the template?

Not using SAM and using CloudFormation templates instead? I know this may be a bit particular to Java only, but I'd like to know how to implement an enterprise-level project with its CI/CD pipelines correctly.

[jfuss]

@bogdan-romaniv This issue isn't related to Layers and Lambda Functions used in the same template but when the build happens, one of your dependencies isn't resolved, specifically "Could not find artifact org.example.layer:lambda-layer:jar:1.0-SNAPSHOT".

Even is the layers were defined separately and by an ARN, your build will still fail for the same reason. This is the design of the builder: https://github.com/aws/aws-lambda-builders/blob/develop/aws_lambda_builders/workflows/java_maven/DESIGN.md.

Do you need to build in a container for this application? If it's not strictly required, I would just not build in a container. Container builds are great for when you need the exact (or close enough) environment to lambda, which is typically when you link things on the Lambda Execution Environment. Otherwise, you need to define your dependencies in a way that maven will be able to build without that dependency being available (which I assume is possible).

[bogdan-romaniv]

Hi @jfuss.

When creating a pipeline with the sam pipeline init --bootstrap command the pipeline.yaml file internally uses the sam build --use-container.

I thought it was recommended to do it like this since the default yaml file does it like this.

From what you say, I understand that the use of --use-container is limited to certain specific cases. In that case, why does the default pipeline generation use it?

Modifying the file pipeline.yaml and removing the --use-container, there are no problems. Thanks

[jfuss]

@bogdan-romaniv I wasn't apart of the direct decision on to include --use-container. My guess is in most cases it is fine and handles the cases in which dependencies need the linking. So as a place to start, --use-container seems reasonable, even though it doesn't handle everything.

Another thing to consider is these init's have to target multiple different languages and customers. Which makes it really hard to cover everything with one thing. Maybe the right thing to do is allow for different variations but that becomes harder to communicate in a simple manner.

Does the removing --use-container solve this for you? Want to ensure you got to a resolution before closing.

[bogdan-romaniv]

Yes, by removing --use-container in the pipeline.yaml file, everything works fine with the build.

It could work for the rest of the languages, but in the case of Java and .Net they are very similar. I'll try to do some tests about it.

For now, it works. Thank you

[jfuss]

@bogdan-romaniv Sounds good. I am going to close this as there is a resolution. Feel free to re-open or create a new Issue if you run into something else. Happy to support further if needed.

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.