Support Lambda Layers

[popey456963]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Description

AWS has recently released a new feature, "Lambda Layers", a way of sharing code between Lambda functions. It would be useful for Terraform to support this.

New or Affected Resource(s)

• aws_lambda

Potential Terraform Configuration

# An implementation would likely have two aspects to it, an addition lambda parameter:

resource "aws_lambda_function" "example_function" {
  layers = [
    // reference to layer
    // another reference
  ]
}

# And the layer type itself

resource "aws_lambda_layer" "example_layer" {
  # layer metadata
  layer_name = ""
  tags = []
  description = ""

  # location of layer
  s3_bucket = []
  s3_key = []
  s3_object_version = []
}

[robertpeteuil]

Might also need a data resource to be able to retrieve the latest version of a layer.

It looks like ListLayers and GetLayerVersion API's cloud support this.

[acburdine]

I'll take a look at this one 😄

[ispulkit]

Waiting for this to come to life...

[popey456963]

@acburdine Wondered if you'd had a chance to look into this yet?

[acburdine]

@popey456963 yup - just finished up the initial work on it this morning. Just need to add some docs and a data source for lambda_layer and I think it should be good.

[ampopp04]

If I wanted to deploy a simple AWSLambda-Python37-SciPy1x layer to a python lambda, is there a way to do it within terraform until your enhancement gets pushed? I can look through the docs to see if there is some sort of generic mechanism I can use to call a layer create. Great work by the way!

[namliz]

@acburdine Thank you for doing this, your PR (#6782) looks very straightforward, any idea if it will make it into 0.12.0?

[alixaxel]

When can we expect this to be included in the stable Terraform release?

[bflad]

Just to clear up any confusion, Terraform core and the Terraform AWS provider are on independently versioned and released. The provider aims for a weekly release cadence, but there will be a gap in that schedule with the end of December holiday season, resuming first week of January.

As for when "this" will be released, depends on what you're looking for:

• A new aws_lambda_layer_version resource for managing a version of a Lambda layer is in progress with r/lambda_layer_version: Add lambda_layer_version resource #6782. If review items are addressed (may require additional rounds) and documentation is added before mid Thursday, that can get released this week. If not, very early January.
• Once the above is merged, that opens the doors to creating other surrounding Lambda Layer resources (e.g. Lambda Layer Version Permissions), data source(s), and updating the aws_lambda_function resource itself. Unless those are made extremely quickly and without review items, they probably won't land until early January given the current timing.

[jonathan-kosgei]

#6782 Seems complete? @bflad

[bflad]

The review comments in #6782 are not fully addressed.

[nvdbleek]

@bflad sorry to bother you again with this issue. But is there an update about when we could use layers in terraform? It looks like @acburdine addressed the remarks on the PR. If possible I want to get a rough estimate so I know I can wait on it, or try to work around this using a Null provider.

[bflad]

The aws_lambda_layer_version resource has been merged in and will release with version 1.56.0 of the Terraform AWS provider in the next day or two.

[bflad]

Support for a layers argument in the aws_lambda_function resource and a layers attribute in the aws_lambda_function data source has also been merged for release with version 1.56.0 of the Terraform AWS provider.

While basic management use cases should now be covered, two pieces of functionality do still remain:

• A new aws_lambda_layer_version data source to lookup Lambda Layers
• A new aws_lambda_layer_version_permission resource (similar to aws_lambda_permission) for allowing cross-account access

[acburdine]

@bflad I was actually looking at implementing the data source (i've got some code locally), but I wasn't sure what the parameters would look like. I tried finding data sources with similar setups, but didn't find one quite like what would be needed for the lambda layer. My idea was this:

To fetch a specific version:

data "aws_lambda_layer_version" "example" {
  layer_name = "my-layer"
  version = "3"
}

To get the most recent version:

data "aws_lambda_layer_verson" "example" {
  layer_name = "my-layer"
  most_recent = true
}

thoughts?

[acburdine]

also, should have a PR for the layer version permission up shortly

[bflad]

@acburdine I think we can omit the most_recent argument and just default to that behavior if version is not provided. 😄

[acburdine]

sounds good - will work on that as well 👍

[bflad]

The new aws_lambda_layer_version resource and layers support in aws_lambda_function resource/data source has been released in version 1.56.0 of the AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

[oxlade39]

@bflad "aws_lambda_layer_version" doesn't seem to support data.

I need to lookup a layer in my lambda deployment. How would I do that?

[acburdine]

@oxlade39 the data source hasn't been implemented yet. I've got some initial work on it locally that I've been meaning to get back to, will see if I can get it finished sometime this week or next.

[jhulfikarali]

@acburdine sorry to bother again with the same question. Is the work for data support is done? If so can you point me to the documentation to it?

[kpedrok]

How do I add "AWSLambda-Python36-SciPy1x" to my deploy? That is a layer that already exists in AWS...

[oxlade39]

@kkpedrok Until the data support is added you will have to manually look up the arn for your region from the aws console and then use directly in your lambda terrraform. You probably want to have a property for the arn to make it simpler to change/upgrade

[acburdine]

Apologies for the delay, finally got the lambda_layer datasource submitted in #8577 😄

[bflad]

Hi everyone! 👋 Great news, a new aws_lambda_layer_version data source has been merged and will release later today with version 2.10.0 of the Terraform AWS Provider. 🎉 Thanks to @acburdine and others who made this functionality possible!

Since it appears we have covered functionality requested in the initial feature request here and these general "support X" type feature requests do not have a great definition of "done", I'm going to opt to close this issue now so we can focus on specific asks going forward.

For any additional feature requests or bug reports with the Lambda Layer functionality in the Terraform AWS Provider, please create new GitHub issues so we can triage them. Thanks!

[acburdine]

@bflad the lambda_layer_permission resource hasn't been implemented yet, which was I believe in this issue as well. Can open a new issue though if needed

[bflad]

Let's treat it separately now 👍

[bflad]

The aws_lambda_layer_version data source has been released in version 2.10.0 of the Terraform AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

[svlaskovic]

If I wanted to deploy a simple AWSLambda-Python37-SciPy1x layer to a python lambda, is there a way to do it within terraform until your enhancement gets pushed? I can look through the docs to see if there is some sort of generic mechanism I can use to call a layer create. Great work by the way!

Hi,

I am trying to achieve the same?

May you please let me know if you done the same and how?

Thank you,
S

[stasovon]

@svlaskovic @kpedrok the following worked for me:

resource "aws_lambda_function" "my_lambda" {
  layers = ["arn:aws:lambda:eu-west-1:399891621064:layer:AWSLambda-Python37-SciPy1x:2"]
  ...
}

I didn't create a aws_lambda_layer_version resource, the above was enough.

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!