New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for Lambda@Edge #3944

Open
horike37 opened this Issue Jul 18, 2017 · 35 comments

Comments

Projects
None yet
@horike37
Copy link
Member

horike37 commented Jul 18, 2017

This is a (Feature Proposal)

Description

For feature proposals:

  • What is the use case that should be solved. The more detail you describe this in the easier it is to understand for us.

https://aws.amazon.com/blogs/aws/lambdaedge-intelligent-processing-of-http-requests-at-the-edge/
Today, Lambda@Edge has been GA released. When CloudFormation will support for this, would be great that implement this as a new event source.

  • If there is additional config how would it look
    So I think it would be something like this:
functions:
  index:
    handler: handler.hello
    events:
      - cloudFront: originRequest
@pmuens

This comment has been minimized.

Copy link
Member

pmuens commented Jul 18, 2017

Great idea! Thanks for keeping an eye on the feature and opening this @horike37 👍

Hopefully CloudFormation will support it soon!

@jthomerson

This comment has been minimized.

Copy link
Contributor

jthomerson commented Aug 24, 2017

I posted something on the CloudFormation forum asking if they could give us a clue when they'd be supporting it: https://forums.aws.amazon.com/thread.jspa?threadID=262327

@pmuens

This comment has been minimized.

Copy link
Member

pmuens commented Aug 24, 2017

I posted something on the CloudFormation forum asking if they could give us a clue when they'd be supporting it: https://forums.aws.amazon.com/thread.jspa?threadID=262327

Great! Thanks for reaching out to the AWS Forums @jthomerson 👍

@jthomerson

This comment has been minimized.

Copy link
Contributor

jthomerson commented Aug 25, 2017

FYI all: I wrote a plugin to support Lambda@Edge for now: https://github.com/silvermine/serverless-plugin-cloudfront-lambda-edge.

@horike37

This comment has been minimized.

Copy link
Member

horike37 commented Aug 25, 2017

@jthomerson
AWSOME!!! great work!!!!

@pmuens

This comment has been minimized.

Copy link
Member

pmuens commented Aug 26, 2017

Great stuff 🎉! Thanks for sharing @jthomerson 👍

@grancalavera

This comment has been minimized.

Copy link

grancalavera commented Dec 21, 2017

I believe this should be already supported in CloudFormation, unless something else besides LambdaFunctionAssociation is required... Is that the case?

@katspaugh

This comment has been minimized.

Copy link

katspaugh commented Dec 21, 2017

@grancalavera LambdaFunctionAssociation requires an ARN with a specific version of the lambda.
I guess the proposal is to somehow automate the version update for a given CloudFront distribution when deploying a new lambda version.

@grancalavera

This comment has been minimized.

Copy link

grancalavera commented Dec 22, 2017

@katspaugh right... would that mean that the CloudFormation template specifying the CloudFront distribution would have to be changed on the fly whenever a new version of the function is deployed?

I don't really know Serverless internals, but I've seen generated CloudFormation templates...

@Birowsky

This comment has been minimized.

Copy link

Birowsky commented Jan 7, 2018

How do you guys do this with CloudFormation? You can help me out here: https://stackoverflow.com/q/48137499/592641

@gdi2290

This comment has been minimized.

Copy link
Contributor

gdi2290 commented Jan 7, 2018

+1
Lambda@Edge is a game changer for Web Apps

github-tipe-logo

@horike37

This comment has been minimized.

Copy link
Member

horike37 commented Jan 9, 2018

Does anyone want to implement this functionality?
We are always welcome your contribution anytime 😄

@laardee

This comment has been minimized.

Copy link
Member

laardee commented Jan 12, 2018

@horike37 Do you have more specific thoughts about the implementation? Should the event source create a new or update an existing CloudFront distribution? Or both?

If event source is updating existing distribution, that should be done with AWS SDK getDistribution & updateDistribution as the LambdaFunctionAssociation is a part of AWS::CloudFront::Distribution and not a separate CloudFormation resource. A pretty much same way that the serverless-plugin-cloudfront-lambda-edge is implemented.

The event source should contain the distribution id and path pattern along with the event type.

functions:
  index:
    handler: handler.hello
    events:
      - cloudFront: 
          distribution: distribution-id
          eventType: event-type
          pathPattern: path-pattern # for other than default cache behaviours

But if event source creates a new distribution, all the other distribution parameters (origins, certificates, aliases etc.) should be defined somewhere. I would personally prefer resources block when creating a new CloudFront distribution.

I can start working on this if you still want this to be implemented to the framework.

@horike37

This comment has been minimized.

Copy link
Member

horike37 commented Jan 13, 2018

@laardee
Thank you for jumping in 😄

We would adopt a way that event source creates a new distribution. In general, the framework core completely relies on CloudFormation behind the scene, so we can't update an existing distribution.(we want that as well if CloudFormation supports for updating an existing one though)

But if event source creates a new distribution, all the other distribution parameters (origins, >certificates, aliases etc.) should be defined somewhere. I would personally prefer resources block >when creating a new CloudFront distribution.

Fully agreed! it would be good to specify an additional settings for cloudfront on resources section.

I can start working on this if you still want this to be implemented to the framework.

AWESOME 👍 🎉 People who is in the serverless community still hope that!!
Let @serverless/framework-maintainers know if you need any helps!

@laardee

This comment has been minimized.

Copy link
Member

laardee commented Jan 14, 2018

@serverless/framework-maintainers I have a question about the stack deployment region. As the Lambda functions triggered with CloudFront has to be deployed into us-east-1, should the stack to be restricted so that it cannot be deployed to other regions or should lambda@edge functions have a separate CloudFormation stack that is deployed to us-east-1?

@horike37

This comment has been minimized.

Copy link
Member

horike37 commented Jan 15, 2018

@laardee

should the stack to be restricted so that it cannot be deployed to other regions

I think it would be good to this idea. Additionally, would be good to add the attention to documentation of this feature as Current Gottcha.

In general, the framework core launch one CF stack for each one deployment.
It may introduce various any side effects if we adopt a separating stack way.

@horike37 horike37 added pr/in-progress and removed help wanted labels Jan 15, 2018

@laardee

This comment has been minimized.

Copy link
Member

laardee commented Jan 16, 2018

@horike37 FYI, currently deleting the CloudFormation stack that has lambda@edge functions seem to be impossible. The master version of Lambda function cannot be removed at all, and that prevents stack removal -> https://forums.aws.amazon.com/thread.jspa?threadID=260242&tstart=0#jive-message-824818. Anyways, I'll continue with the implementation. I have high hopes that AWS fixes this issue any day now. 😁

@horike37

This comment has been minimized.

Copy link
Member

horike37 commented Jan 16, 2018

Oh... well 😅
So that means we should wait for fix the issue on AWS end to go ahead with this feature.

Anyways, I'll continue with the implementation. I have high hopes that AWS fixes this issue any day now.

Great thanks! Yes, I hope that too.

@debanjanbasu

This comment has been minimized.

Copy link

debanjanbasu commented Jan 29, 2018

The delete functionality seems to be working now. Is there any chance of updates now?

@laardee

This comment has been minimized.

Copy link
Member

laardee commented Jan 29, 2018

@debanjanbasu Nice, I'll try to open the PR today.

@laardee

This comment has been minimized.

Copy link
Member

laardee commented Jan 30, 2018

I managed to delete the lambda after I manually remove the lambda association from the CloudFront distribution and wait for a while (maybe 15 minutes). CloudFormation still fails to remove the function when the stack is deleted.

Lambda was unable to delete arn:aws:lambda:us-east-1:XXXXXXXXX:function:serverless-hello-world-dev-helloWorld3:1 because it is a replicated function.

The stack remains to the "delete failed" state. After 15 minutes I'm was able to delete the rest of the stack. 😒

@debanjanbasu is there more info about the fix somewhere?

@horike37 because the manual removal seems to work, we could first remove the associations and lambdas with the aws-sdk and then remove the stack. As a temporary solution before CloudFormation gets fixed...

@horike37

This comment has been minimized.

Copy link
Member

horike37 commented Jan 31, 2018

Thank you for testing @laardee 👍
Imo, we should wait for the implementation until a lifecycle from sls deploy to sls remove will work fine via CloudFormation.

We consistently have been implementing without operating deployed resources with aws-sdk(i.e operating deployed resources are done completely via CloudFormation), otherwise, consistency of deployment would be broken.

@laardee

This comment has been minimized.

Copy link
Member

laardee commented Jan 31, 2018

@horike37 ok, I'll check if I get more info about the removal from AWS.

@mfrankwork

This comment has been minimized.

Copy link

mfrankwork commented Feb 14, 2018

AWS responded to an open support ticket today and said they have enabled the deletion of Lambda@Edge functions. I was able to successfully delete a master Lambda@Edge function that previously could not be removed.

@laardee

This comment has been minimized.

Copy link
Member

laardee commented Feb 14, 2018

I still get an error when I try to delete replicated function. Maybe they make it available gradually. I'll do more testing.

@yvele

This comment has been minimized.

Copy link

yvele commented Feb 20, 2018

Simple question.. How do you manage the fact that CloudFront distribution and Lambda Edge association are tightly coupled.

Like if you manage the CloudFront distribution in a separate CloudFormation stack, then the Lambda Edge association might be removed.

🤔

See also the question on AWS Forum or on SO.

@laardee

This comment has been minimized.

Copy link
Member

laardee commented Feb 26, 2018

That could be an option, possibly a bit complex to handle, but if the CloudFront and invoke permissions were in a separate stack, removal process could stop to wait for deletion of replicated functions. I check if it is possible to check when the replicated functions are deleted.

I think the issue is that the CloudFormation doesn't wait long enough for replications to be deleted. When I delete the CF stack, first round always fails because it cannot delete the replicated functions, but if I wait ~15 minutes and then try to delete the stack again, it will delete the functions that were left from the previous round. Removing the stack takes least 30+ minutes total... Debugging this is really time-consuming 😁

@laardee laardee referenced a pull request that will close this issue Mar 3, 2018

Open

[WIP] Lambda@Edge #4796

2 of 7 tasks complete
@jthomerson

This comment has been minimized.

Copy link
Contributor

jthomerson commented Apr 10, 2018

@laardee @mfrankwork @horike37 I described a bit more about why the delete doesn't work in a similar issue opened on our plugin that supports Lambda@Edge via Serverless: silvermine/serverless-plugin-cloudfront-lambda-edge#6

@jthomerson

This comment has been minimized.

Copy link
Contributor

jthomerson commented Apr 10, 2018

All: please note that I'm updating our serverless-plugin-cloudfront-lambda-edge plugin to use the new CloudFormation support for Lambda@Edge. The principle is that the plugin will merely make the required modifications to the CloudFormation stack after Serverless has initially created the stack template, but before it has deployed. Those modifications include:

  • remove environment variables from functions that have a Lambda@Edge event trigger
  • modify the permissions granted to the function's role since it will need to be able to create log groups that AWS controls the name of (which typically include the region name)
  • modify the CloudFront distribution resource (which you define in your serverless.yml file) to add the LambdaFunctionAssociations parameter to it.

The changes planned for v2.0 can be found in the issues listed below. Please let us know if you have other requirements as we hope to ship 2.0 in the next week or so.

@horike37

This comment has been minimized.

Copy link
Member

horike37 commented Apr 10, 2018

@jthomerson
Awesome 🎉 👍 😄 Thank you for the infomation!

@claytopiccinin

This comment has been minimized.

Copy link

claytopiccinin commented Jun 27, 2018

Awesome work @jthomerson!!! Is there some way we can automate Cloud Front Distribution to set up a new Lambda@Edge version?

Currently, I'm using a proxy with multiple behaviors, and I'd like to update a specific cache behavior with a new version of Lambda@Edge.

Additionally, can someone explain me why AWS just permit to add triggers for a numbered version, not for $LATEST or for aliases?

@jthomerson

This comment has been minimized.

Copy link
Contributor

jthomerson commented Jun 27, 2018

@claytopiccinin thanks. I'm not sure I understand your question. Serverless automatically creates new versions of your function when they change. Combine that with serverless-plugin-cloudfront-lambda-edge and you can deploy Lambda@Edge functions to your CloudFront distributions easily with Serverless. Check out the plugin's documentation and see if it answers your question.

@joseoscargarciajr

This comment has been minimized.

Copy link

joseoscargarciajr commented Aug 7, 2018

Is there a way for the plugin to use the Arn of the cloudfront instead of the name, I have functions and api in the ap-northeast-1 and lambdaegde is only available in us-east-1, I want to create the function and just reference the Arn of the cloudfront via Fn::ImportValue using Arn, is this possible?

@ejaylegaspi

This comment has been minimized.

Copy link

ejaylegaspi commented Dec 27, 2018

@jthomerson I'm using your plugin and it works great! However, is there a way to rollback a lambda deployment using Serverless and your plugin? Like to revert back to the old version of Lambda@Edge if there's a problem? I'm trying to find ways how to do it. My ideal deployment would be to release it to Production but with a flag to go and use the latest version of Lambda and see if everything looks good. If there's a problem, i can easily rollback to the previous version without doing another whole new deployment just to rollback.

@jthomerson

This comment has been minimized.

Copy link
Contributor

jthomerson commented Dec 27, 2018

@ejaylegaspi unfortunately with Lambda@Edge, any change to your function (including rollback) is constrained by the amount of time it takes the CloudFront distribution to roll out a change ... which for me averages 30-45 minutes. So, there's really no quick method. They don't support pointing to a Lambda alias (presumably because of the function replication "magic") (see [1]), so you're a bit stuck there. There's not much we could do at a plugin level for you to help with this.

[1] https://forums.aws.amazon.com/thread.jspa?messageID=816124

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment