Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Lambda as ALB target in Cloudformation #721

Open
ssoulier opened this issue Dec 14, 2018 · 34 comments
Open

Lambda as ALB target in Cloudformation #721

ssoulier opened this issue Dec 14, 2018 · 34 comments

Comments

@ssoulier
Copy link

@ssoulier ssoulier commented Dec 14, 2018

Hi,

Could you please add the cloudformation and SAM the ability to specify an events for lambda which is an Application Load Balancer ?

Thanks.

@rsram312
Copy link

@rsram312 rsram312 commented Dec 14, 2018

Went through the documentations. Not sure if it is updated either. Can you please add the SAM template ability to specify ALB feature for Lambda

Loading

@lorddelicious
Copy link

@lorddelicious lorddelicious commented Dec 17, 2018

Yes please SAM ALB is needed. Discontinuing use of API Gateway due to high cost, transitioning to ALB.

Loading

@ssoulier
Copy link
Author

@ssoulier ssoulier commented Jan 30, 2019

Any update on this feature request ?

Loading

@kauphylover
Copy link

@kauphylover kauphylover commented Jan 30, 2019

Another request to the AWS team to prioritize this!

Loading

@dbettin
Copy link

@dbettin dbettin commented Feb 1, 2019

Another request. Thanks!

Loading

@brettstack
Copy link
Contributor

@brettstack brettstack commented Feb 1, 2019

Absolutely. At a minimum we could add a new Event for ALB which create the permission for you and you could specify your existing ALB. However, it's probably more likely that you want SAM to create the ALB target group for you, or maybe even the entire ALB (similar to how we create the API).

Would love to get some more feedback from everyone on this, including some SAM syntax.

Loading

@rsram312
Copy link

@rsram312 rsram312 commented Feb 2, 2019

@brettstack Ideally, what I was hoping is if SAM can create the entire ALB and register Lambda as target. Probably following might be capabilities required:

  1. Creating an ALB
  2. Creating a target group with target type as Lambda with options to enable health check
  3. Adding listeners to the ALB
  4. Adding listener rules to the ALB
  5. Adding permissions to the Lambda to access the ALB
  6. Register the targets with the target group

I am not exactly sure which of the above features are already available in SAM. So, please correct me if anything above is redundant.

Currently, I was using a workaround by creating a Lambda function which does all these operations and was invoking that lambda wherever required. However, the feature being available in SAM would be really handy in directly leveraging it within the SAM template.

Loading

@brettstack
Copy link
Contributor

@brettstack brettstack commented Feb 2, 2019

@rsram312 that's very useful. Do you have an existing minimal template that includes all of those required resources?

Loading

@rsram312
Copy link

@rsram312 rsram312 commented Feb 2, 2019

@brettstack The sample template I was using as workaround leverages boto3 to create those resources. If it might be of any use, I could probably send it over.

Loading

@deleugpn
Copy link

@deleugpn deleugpn commented Feb 6, 2019

For me the preference is in specifying an existing ALB and getting the target group and permissions created by SAM.

Loading

@kauphylover
Copy link

@kauphylover kauphylover commented Feb 7, 2019

I like what @deleugpn suggests - seems like the most natural way to go about it.

Loading

@brettstack
Copy link
Contributor

@brettstack brettstack commented Feb 7, 2019

We will try to accommodate all scenarios.

  1. Create everything for you (ALB, Group, Permissions)
  2. Create just Group and Permission
  3. Create just Permission

Loading

@deleugpn
Copy link

@deleugpn deleugpn commented Feb 7, 2019

I'm not sure if it would be possible, but I would guess that the following would be quite an amazing syntax:

      Events: 
        MyEventName:
          Type: ALB
          Properties:
            LoadBalancerArn: !ImportValue LoadBalancer
            ListenerArn: !ImportValue Listener
            CertificateArn: !ImportValue Certificate
            Condition: [...]

I don't know if I would need to specify anything else other than this.

The expected result would be something similar to this:

  HttpsListenerRule:
    Type: AWS::ElasticLoadBalancingV2::ListenerRule
    Properties:
      Actions:
      - Type: forward
        TargetGroupArn: !Ref TargetGroup
      Conditions: [Available on the Event Property]
      ListenerArn: [Available on the Event Property]
      Priority: [This is a tricky one. I think we have no other option than to delegate this to the user to define, unfortunately]

  TargetGroup:
    Type: AWS::ElasticLoadBalancingV2::TargetGroup
    Properties:
      TargetType: lambda

Loading

@luketn
Copy link

@luketn luketn commented Mar 7, 2019

  Priority: [This is a tricky one. I think we have no other option than to delegate this to the user to define, unfortunately]

The valid range of priorities is from 1 - 50,000.

One approach we could take to this is to default the value to a random number 40,000 to 50,000 catch the exception of an already used priority and choose again.

As long as the path / domain is unique, the priority should not matter.

Loading

@deleugpn
Copy link

@deleugpn deleugpn commented Mar 7, 2019

SAM would be limited to deploy up to 10,000 lambdas on a single Listener, I guess that's fine.

Loading

@ravibarkhani
Copy link

@ravibarkhani ravibarkhani commented Mar 20, 2019

Hello, is SAM supports event from ALB now ?
I tried to create a TargetGroup using cloudformation it fails in two way:

  ALBTargetGroup: 
    Type: AWS::ElasticLoadBalancingV2::TargetGroup
    Properties:
      Name: !Ref LoadBalancerName
      VpcId: !ImportValue "VPCCreate-VpcId"
      Port : 80
      Protocol: HTTP
      TargetType: lambda
  1. With below template it fails with error : "Port cannot be specified for target groups with target type 'lambda' (Service: AmazonElasticLoadBalancingV2; Status Code: 400; Error Code: ValidationError; Request ID: 3eaa3569-4b36-11e9-a442-1f94e3803149)"

  2. When I do not provide port in above template it fails with message "Property Port cannot be empty."

Loading

@kylegordon
Copy link

@kylegordon kylegordon commented Mar 20, 2019

This is still an issue. I get the same errors as @ravibarkhani

It's been 4 months since Lambda targets for ALBs was announced, and Cloudformation is lagging behind.

Loading

@keetonian
Copy link
Contributor

@keetonian keetonian commented Mar 25, 2019

You can use any cloudformation resource in your SAM templates; SAM does not alter these resources. You should be able to use any new feature from CFN in SAM without a SAM update unless an update is required to one of the AWS::Serverless::* resources.

If this doesn't work, it's an issue with either the configuration or CFN support, not SAM.

Loading

@s0enke
Copy link

@s0enke s0enke commented Apr 19, 2019

Loading

@brettstack
Copy link
Contributor

@brettstack brettstack commented Apr 23, 2019

Thanks @deleugpn for the proposed syntax #721 (comment) and @s0enke for the example. How might the input and output look for defining multiple paths/methods like we do for Api event? That is, we should allow defining the Rule for advanced configuration, but for common use-cases (e.g map this Path) we should provide simpler syntax which creates the necessary resources for you.

Loading

@petehanssens
Copy link

@petehanssens petehanssens commented May 25, 2019

Thanks @s0enke for putting that together, I've gone one step further to add custom subdomain and auth0 authentication to the template, enjoy:
https://www.peterhanssens.com.au/blog/2019-05/alb-to-lambda-with-auth0-authentication/

Loading

@metaskills
Copy link

@metaskills metaskills commented Jun 29, 2019

@s0enke Thanks for that sample code. For Rails & Lambda using SAM, we were able to side step official support for this by just using CloudFormation. For the Lamby/Rails community, we had no need to for the sam local because Rails development is already easy and we use SAM as a means to define what we need in staging, production, etc.

customink/lamby#37

So maybe this helps shape what is needed for SAM to support this? How much should SAM build? I feel these are two distinct questions below after going thru this exercise with Lamby.

  • Syntactic sugar for CloudFormation?
  • CLI sam local server mocking an Application Load Balancer?

Loading

@raaone7
Copy link

@raaone7 raaone7 commented Jul 17, 2019

+1

Loading

@mneil
Copy link

@mneil mneil commented Aug 9, 2019

I'm concerned that the example, along with any examples I've found, for how to use Lambda behind an ALB leaves the lambda open to execution from any loadbalancer on AWS. The examples in this rails app do not show how to use SourceAccount or SourceArn to restrict access. I have been unable to launch a lambda using either of those restrictions behind an ALB successfully. Could someone update the example, or provide an example, of how to use lambda behind an ALB that does not allow access to the world the entire account?

Edited: The lambda is access to the entire account, not the world

Loading

@raaone7
Copy link

@raaone7 raaone7 commented Aug 10, 2019

Loading

@praneetap
Copy link
Contributor

@praneetap praneetap commented Aug 16, 2019

Thanks for the feedback @mneil! Could you send us some links to the examples you are referring to?

Loading

@luketn
Copy link

@luketn luketn commented Aug 22, 2019

BTW due to the long timeframe on this I’m using Serverless for ALB/lambda projects:
https://serverless.com/framework/docs/providers/aws/events/alb/

I’d suggest taking inspiration from their syntax - it’s pretty nice!

Loading

@deleugpn
Copy link

@deleugpn deleugpn commented Aug 22, 2019

I also stopped recommending SAM as the go-to tool for serverless deployment. AWS is developing and marketing serverless A LOT but lagging behind on CloudFormation / SAM A LOT as well. 3rd party tools are doing a much better job at supporting AWS-provided features than AWS themselves.

Loading

@brysontyrrell
Copy link
Contributor

@brysontyrrell brysontyrrell commented Oct 18, 2019

Plus one from me on this.

I'm breaking up a stack currently and taking a set of my endpoints off an API Gateway and moving them behind an ALB. Based on another stack where I did an ALB->Lambda stack, I need to build the following resources into my template to make this work:

AWS::EC2::VPC (and all the stuff that goes with that like subnets, IGW, EIP, etc.)
AWS::EC2::SecurityGroup
AWS::Lambda::Permission
AWS::ElasticLoadBalancingV2::LoadBalancer
AWS::ElasticLoadBalancingV2::TargetGroup (per Lambda)
AWS::ElasticLoadBalancingV2::ListenerRule (per Lambda)

That's... a lot of stuff. Really, AWS needs to make ALBs a true serverless service and remove the requirements for an underlying VPC. This all becomes much simpler then. Until such time, SAM has an opportunity to abstract this boilerplate madness away in a simplified syntax.

Loading

@andrewdmay
Copy link

@andrewdmay andrewdmay commented May 8, 2020

I'm concerned that the example, along with any examples I've found, for how to use Lambda behind an ALB leaves the lambda open to execution from any loadbalancer on AWS.

@mneil The only way I've found to restrict access to the Lambda to a specific target group is to name the target group and use a wildcard in the SourceArn on the Permission:

SourceArn: !Sub arn:aws:elasticloadbalancing:${AWS::Region}:${AWS::AccountId}:targetgroup/MyTargetGroupName/*

This allows the permission to be created before the Target Group (which otherwise fails because of the missing permission) and prevents a circular reference. Unfortunately it does mean naming the target group, which isn't idea, and you can't match on the full ARN.

You can modify the permission to add the SourceARN after the fact (and that could be done within a custom resource).

I took a look at what the Serverless Framework was doing for this because at first it appeared they had somehow worked around this problem - but actually what they are doing is first creating a permission without a SourceARN that the Target Group depends upon, and then they add a second permission that uses the SourceARN after the Target Group has been created. This doesn't increase security, but does make the ALB show up as the event source for the Lambda in the Console.

(which if you use the wildcard SourceARN, you get an error about it not matching a Target group, but it does appear to work, and I assume the restriction works, but I haven't verified that).

Loading

@andrewdmay
Copy link

@andrewdmay andrewdmay commented May 8, 2020

For supporting ALBs as a source in SAM, I've been experimenting with a CloudFormation macro that accepts events like this:

      Events:
        AlbApi:
          Type: ALB
          Properties:
            Host: abc.def.com
            ListenerArn: !Ref ListenerArn
            Priority: 123

I'm following the basic pattern of how the different events work, and then using the Serverless Framework as inspiration for the property names. I'm assuming that the Listener already exists and is shared for multiple lambdas, so I'm just generating the permission, target group and listener rule.

I think automatically generating the ALB and listeners with an appropriate security group etc. just from the events would be difficult.

Loading

@aidansteele
Copy link

@aidansteele aidansteele commented May 9, 2020

@andrewdmay I did a similar thing (and so far have made the same assumption re: pre-existing listener) and it’s been working very well for us.

https://github.com/glassechidna/sam-alb

Loading

@samirhsnv
Copy link

@samirhsnv samirhsnv commented May 28, 2021

Any updates here? We need this feature because API Gateway has a hard timeout limit of 30 seconds

Loading

@deleugpn
Copy link

@deleugpn deleugpn commented May 28, 2021

@samirhsnv I believe this ticket is about making it easier, however it's already possible to deploy your Lambda behind ALB using CFN. You can see an example here: https://blog.deleu.dev/running-lambda-behind-application-load-balancer/

Loading

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet