Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for Lambda@Edge #3944

Closed
horike37 opened this issue Jul 18, 2017 · 43 comments
Closed

Support for Lambda@Edge #3944

horike37 opened this issue Jul 18, 2017 · 43 comments

Comments

@horike37
Copy link
Member

@horike37 horike37 commented Jul 18, 2017

This is a (Feature Proposal)

Description

For feature proposals:

  • What is the use case that should be solved. The more detail you describe this in the easier it is to understand for us.

https://aws.amazon.com/blogs/aws/lambdaedge-intelligent-processing-of-http-requests-at-the-edge/
Today, Lambda@Edge has been GA released. When CloudFormation will support for this, would be great that implement this as a new event source.

  • If there is additional config how would it look
    So I think it would be something like this:
functions:
  index:
    handler: handler.hello
    events:
      - cloudFront: originRequest
@pmuens
Copy link
Contributor

@pmuens pmuens commented Jul 18, 2017

Great idea! Thanks for keeping an eye on the feature and opening this @horike37 👍

Hopefully CloudFormation will support it soon!

@jthomerson
Copy link
Contributor

@jthomerson jthomerson commented Aug 24, 2017

I posted something on the CloudFormation forum asking if they could give us a clue when they'd be supporting it: https://forums.aws.amazon.com/thread.jspa?threadID=262327

@pmuens
Copy link
Contributor

@pmuens pmuens commented Aug 24, 2017

I posted something on the CloudFormation forum asking if they could give us a clue when they'd be supporting it: https://forums.aws.amazon.com/thread.jspa?threadID=262327

Great! Thanks for reaching out to the AWS Forums @jthomerson 👍

@jthomerson
Copy link
Contributor

@jthomerson jthomerson commented Aug 25, 2017

FYI all: I wrote a plugin to support Lambda@Edge for now: https://github.com/silvermine/serverless-plugin-cloudfront-lambda-edge.

@horike37
Copy link
Member Author

@horike37 horike37 commented Aug 25, 2017

@jthomerson
AWSOME!!! great work!!!!

@pmuens
Copy link
Contributor

@pmuens pmuens commented Aug 26, 2017

Great stuff 🎉! Thanks for sharing @jthomerson 👍

@grancalavera
Copy link

@grancalavera grancalavera commented Dec 21, 2017

I believe this should be already supported in CloudFormation, unless something else besides LambdaFunctionAssociation is required... Is that the case?

@katspaugh
Copy link

@katspaugh katspaugh commented Dec 21, 2017

@grancalavera LambdaFunctionAssociation requires an ARN with a specific version of the lambda.
I guess the proposal is to somehow automate the version update for a given CloudFront distribution when deploying a new lambda version.

@grancalavera
Copy link

@grancalavera grancalavera commented Dec 22, 2017

@katspaugh right... would that mean that the CloudFormation template specifying the CloudFront distribution would have to be changed on the fly whenever a new version of the function is deployed?

I don't really know Serverless internals, but I've seen generated CloudFormation templates...

@Birowsky
Copy link

@Birowsky Birowsky commented Jan 7, 2018

How do you guys do this with CloudFormation? You can help me out here: https://stackoverflow.com/q/48137499/592641

@PatrickJS
Copy link
Contributor

@PatrickJS PatrickJS commented Jan 7, 2018

+1
Lambda@Edge is a game changer for Web Apps

@horike37
Copy link
Member Author

@horike37 horike37 commented Jan 9, 2018

Does anyone want to implement this functionality?
We are always welcome your contribution anytime 😄

@laardee
Copy link
Member

@laardee laardee commented Jan 12, 2018

@horike37 Do you have more specific thoughts about the implementation? Should the event source create a new or update an existing CloudFront distribution? Or both?

If event source is updating existing distribution, that should be done with AWS SDK getDistribution & updateDistribution as the LambdaFunctionAssociation is a part of AWS::CloudFront::Distribution and not a separate CloudFormation resource. A pretty much same way that the serverless-plugin-cloudfront-lambda-edge is implemented.

The event source should contain the distribution id and path pattern along with the event type.

functions:
  index:
    handler: handler.hello
    events:
      - cloudFront: 
          distribution: distribution-id
          eventType: event-type
          pathPattern: path-pattern # for other than default cache behaviours

But if event source creates a new distribution, all the other distribution parameters (origins, certificates, aliases etc.) should be defined somewhere. I would personally prefer resources block when creating a new CloudFront distribution.

I can start working on this if you still want this to be implemented to the framework.

@horike37
Copy link
Member Author

@horike37 horike37 commented Jan 13, 2018

@laardee
Thank you for jumping in 😄

We would adopt a way that event source creates a new distribution. In general, the framework core completely relies on CloudFormation behind the scene, so we can't update an existing distribution.(we want that as well if CloudFormation supports for updating an existing one though)

But if event source creates a new distribution, all the other distribution parameters (origins, >certificates, aliases etc.) should be defined somewhere. I would personally prefer resources block >when creating a new CloudFront distribution.

Fully agreed! it would be good to specify an additional settings for cloudfront on resources section.

I can start working on this if you still want this to be implemented to the framework.

AWESOME 👍 🎉 People who is in the serverless community still hope that!!
Let @serverless/framework-maintainers know if you need any helps!

@laardee
Copy link
Member

@laardee laardee commented Jan 14, 2018

@serverless/framework-maintainers I have a question about the stack deployment region. As the Lambda functions triggered with CloudFront has to be deployed into us-east-1, should the stack to be restricted so that it cannot be deployed to other regions or should lambda@edge functions have a separate CloudFormation stack that is deployed to us-east-1?

@horike37
Copy link
Member Author

@horike37 horike37 commented Jan 15, 2018

@laardee

should the stack to be restricted so that it cannot be deployed to other regions

I think it would be good to this idea. Additionally, would be good to add the attention to documentation of this feature as Current Gottcha.

In general, the framework core launch one CF stack for each one deployment.
It may introduce various any side effects if we adopt a separating stack way.

@laardee
Copy link
Member

@laardee laardee commented Jan 16, 2018

@horike37 FYI, currently deleting the CloudFormation stack that has lambda@edge functions seem to be impossible. The master version of Lambda function cannot be removed at all, and that prevents stack removal -> https://forums.aws.amazon.com/thread.jspa?threadID=260242&tstart=0#jive-message-824818. Anyways, I'll continue with the implementation. I have high hopes that AWS fixes this issue any day now. 😁

@horike37
Copy link
Member Author

@horike37 horike37 commented Jan 16, 2018

Oh... well 😅
So that means we should wait for fix the issue on AWS end to go ahead with this feature.

Anyways, I'll continue with the implementation. I have high hopes that AWS fixes this issue any day now.

Great thanks! Yes, I hope that too.

@debanjanbasu
Copy link

@debanjanbasu debanjanbasu commented Jan 29, 2018

The delete functionality seems to be working now. Is there any chance of updates now?

@laardee
Copy link
Member

@laardee laardee commented Jan 29, 2018

@debanjanbasu Nice, I'll try to open the PR today.

@laardee
Copy link
Member

@laardee laardee commented Jan 30, 2018

I managed to delete the lambda after I manually remove the lambda association from the CloudFront distribution and wait for a while (maybe 15 minutes). CloudFormation still fails to remove the function when the stack is deleted.

Lambda was unable to delete arn:aws:lambda:us-east-1:XXXXXXXXX:function:serverless-hello-world-dev-helloWorld3:1 because it is a replicated function.

The stack remains to the "delete failed" state. After 15 minutes I'm was able to delete the rest of the stack. 😒

@debanjanbasu is there more info about the fix somewhere?

@horike37 because the manual removal seems to work, we could first remove the associations and lambdas with the aws-sdk and then remove the stack. As a temporary solution before CloudFormation gets fixed...

@horike37
Copy link
Member Author

@horike37 horike37 commented Jan 31, 2018

Thank you for testing @laardee 👍
Imo, we should wait for the implementation until a lifecycle from sls deploy to sls remove will work fine via CloudFormation.

We consistently have been implementing without operating deployed resources with aws-sdk(i.e operating deployed resources are done completely via CloudFormation), otherwise, consistency of deployment would be broken.

@laardee
Copy link
Member

@laardee laardee commented Jan 31, 2018

@horike37 ok, I'll check if I get more info about the removal from AWS.

@mfrankwork
Copy link

@mfrankwork mfrankwork commented Feb 14, 2018

AWS responded to an open support ticket today and said they have enabled the deletion of Lambda@Edge functions. I was able to successfully delete a master Lambda@Edge function that previously could not be removed.

@laardee
Copy link
Member

@laardee laardee commented Feb 14, 2018

I still get an error when I try to delete replicated function. Maybe they make it available gradually. I'll do more testing.

@jthomerson
Copy link
Contributor

@jthomerson jthomerson commented Jun 27, 2018

@claytopiccinin thanks. I'm not sure I understand your question. Serverless automatically creates new versions of your function when they change. Combine that with serverless-plugin-cloudfront-lambda-edge and you can deploy Lambda@Edge functions to your CloudFront distributions easily with Serverless. Check out the plugin's documentation and see if it answers your question.

@joseoscargarciajr
Copy link

@joseoscargarciajr joseoscargarciajr commented Aug 7, 2018

Is there a way for the plugin to use the Arn of the cloudfront instead of the name, I have functions and api in the ap-northeast-1 and lambdaegde is only available in us-east-1, I want to create the function and just reference the Arn of the cloudfront via Fn::ImportValue using Arn, is this possible?

@ejaylegaspi
Copy link

@ejaylegaspi ejaylegaspi commented Dec 27, 2018

@jthomerson I'm using your plugin and it works great! However, is there a way to rollback a lambda deployment using Serverless and your plugin? Like to revert back to the old version of Lambda@Edge if there's a problem? I'm trying to find ways how to do it. My ideal deployment would be to release it to Production but with a flag to go and use the latest version of Lambda and see if everything looks good. If there's a problem, i can easily rollback to the previous version without doing another whole new deployment just to rollback.

@jthomerson
Copy link
Contributor

@jthomerson jthomerson commented Dec 27, 2018

@ejaylegaspi unfortunately with Lambda@Edge, any change to your function (including rollback) is constrained by the amount of time it takes the CloudFront distribution to roll out a change ... which for me averages 30-45 minutes. So, there's really no quick method. They don't support pointing to a Lambda alias (presumably because of the function replication "magic") (see [1]), so you're a bit stuck there. There's not much we could do at a plugin level for you to help with this.

[1] https://forums.aws.amazon.com/thread.jspa?messageID=816124

@kmjennison
Copy link

@kmjennison kmjennison commented Feb 8, 2019

In case it's helpful, I wanted to share the somewhat convoluted setup I used. Our constraints were: we want to use Serverless to manage the Lambda@Edge functions, our existing services are not in us-east-1, and and our CloudFront distribution is defined in a separate Serverless service.

Unfortunately, serverless-plugin-cloudfront-lambda-edge does not support cross-service references, so I chose to define the Lambda@Edge associations manually via LambdaFunctionAssociations in the CloudFront resource.

I created a separate Serverless config file for the Lambda@Edge functions, because they must be in us-east-1 (unlike the rest of our stack) and Serverless does not support multi-region deployments. Serverless also does not support customizing the yml filename, so we simply rename each yml file to "serverless.yml" before deploying each one.

When we update the Lambda@Edge functions, we manually increment the version number in the CloudFront resource definition.

I'd love any suggestions if someone found a simpler approach, especially if I overlooked functionality in Serverless or serverless-plugin-cloudfront-lambda-edge. A new plugin could probably address this issue.

Hopefully Serverless core can eventually handle this multi-service, multi-region use case.

@garethmcc
Copy link
Contributor

@garethmcc garethmcc commented Feb 8, 2019

One thing to bear in mind throughout all of this is that practically it doesn't matter in which "region" you define your CloudFront distribution. It only matter what price option you pick to support the edges you want. Cloudfront is essentially global otherwise which is why there is the restriction on us-east-1. What may have made it easier is to move the CloudFront definition from the other service to the same service as your lambda.

@kmjennison
Copy link

@kmjennison kmjennison commented Feb 8, 2019

@garethmcc True. I didn't do that because we have other resources/variables defined in the CloudFront service, so it seemed like a lot of trouble—but, at the end of the day, maybe less trouble than the approach I outlined :)

@pmuens pmuens added this to the 1.39.0 milestone Feb 20, 2019
@Maxwell2022
Copy link
Contributor

@Maxwell2022 Maxwell2022 commented Feb 25, 2019

@kmjennison

[...] I created a separate Serverless config file for the Lambda@Edge functions, because they must be in us-east-1 [...]

Is that because of your architecture or is it an AWS requirement?

EDIT
https://aws.amazon.com/about-aws/whats-new/2017/07/lambda-at-edge-now-generally-available/

"Lambda@Edge functions can now be authored in US East (N. Virginia), and will be replicated globally for invocation in response to CloudFront events."

I understand that if you want replication, then you need to deploy to us-east-1. It's not really clear here. If you don't care about replication, could you deploy it in other regions? I guess yes. However, if you enabled CloudFront with A-Z replication, then you probably need to deploy this Edge functions to us-east-1.

@kmjennison
Copy link

@kmjennison kmjennison commented Feb 26, 2019

@Maxwell2022 To my knowledge, Lambda@Edge functions must be in us-east-1.

AWS docs tutorial for Lambda@Edge:

Make sure that you’re in the US-East-1 (N. Virginia) Region. You must be in this Region to create Lambda@Edge functions.

AWS docs on Lambda limitations:

You can add triggers only for functions in the US East (N. Virginia) Region.

@eahefnawy eahefnawy removed this from the 1.39.0 milestone Mar 12, 2019
@pmuens pmuens modified the milestones: 1.49.0, 1.50.0 Jul 16, 2019
@pmuens pmuens mentioned this issue Aug 7, 2019
10 of 10 tasks complete
@medikoo medikoo modified the milestones: 1.50.0, 1.51.0 Aug 14, 2019
@pmuens pmuens modified the milestones: 1.51.0, 1.52.0 Aug 27, 2019
@pmuens pmuens closed this in #6512 Sep 5, 2019
@medikoo medikoo removed this from the 1.52.0 milestone Sep 11, 2019
@c-ameron
Copy link

@c-ameron c-ameron commented Apr 30, 2020

Hi @horike37 , thanks for your work with this. I can see that you've done https://www.npmjs.com/package/serverless-lambda-edge-pre-existing-cloudfront which fixes it - however I can't see it on Github anywhere? Could you please upload it to Github if you don't mind? Thanks!

@horike37
Copy link
Member Author

@horike37 horike37 commented May 5, 2020

@c-ameron
Hi
Thank you for seeing my plugin! Already uploaded it here.
https://github.com/serverless-operations/serverless-lambda-edge-pre-existing-cloudfront

Please check it 👍

@c-ameron
Copy link

@c-ameron c-ameron commented May 7, 2020

ah great - thank you! :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.