Support for Application Load Balancers events

[owliveira]

This is a Feature Proposal

Description

As announced in Re:Invent, Lambda functions are now able to be set as targets for Application Load Balancer events, which is such like another fashion over setting up an API Gateway that triggers a Lambda.

For more detailed information about the announcement: AWS Blog
Using the feature with AWS CLI: AWS Docs

This feature request is all about:

• Allow to configure ALB events for triggering functions

Example of usage (under events section):

events:
    - alb:
        name: load-balancer-name
        listener: HTTPS:443
        host: www.example.com
        path: /lambda/my-handler-function

Similar or dependent issues:

Let me now if you need some more detailed information.

[alecventura]

This would be awesome!

[dschep]

Would you expect this to create the ALB for you or require an existing ALB?

[owliveira]

AWS gives us two ways of using this new feature, the first is by creating an ALB and providing the Lambda's ARN. The second, you can point up an existing ALB in the Lambda configuration page. The second one seems to be more likely to the framework, so, requiring an existing ALB would make more sense, I guess.

[novemberde]

@owliveira Additionally, host and listener on ALB are able to be an array.
So there're two options.

Here's second example of usage.

events:
    - alb:
        name: load-balancer-name
        listeners: 
            - HTTPS: 443
              HOST: www.example.com
              PATH: /lambda/my-handler-function
            - HTTPS: 443
              HOST: www.example2.com
              PATH: /*
            - HTTPS: 80
              HOST: www.example.com
              PATH: /lambda/my-handler-function

I'm looking forward to applying this feature on my production!

[dschep]

@owliveira, the first would be more similar how APIG integration is handled in the framework tho.

[rdbatch02]

I'm pretty interested in having this functionality, I could take a stab at adding it.

[dschep]

Feel free to go for it @c1phr!

[psyhomb]

Is this feature available in CloudFormation, given that Serverless is using CF under the hood?

According to documentation it seems like it's not available yet:
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-elasticloadbalancingv2-targetgroup.html#cfn-elasticloadbalancingv2-targetgroup-targettype

[dschep]

Good catch @psyhomb. that's a problem 😞

[rdbatch02]

Yep, I found this a few days ago and opened a thread with AWS to determine when CloudFormation will get support, but still waiting to hear back.

[psyhomb]

Yep, I found this a few days ago and opened a thread with AWS to determine when CloudFormation will get support, but still waiting to hear back.

@c1phr thanks for opening a thread and please let us know when/if you get the response back.

[jthomerson]

I agree that the framework should require an existing ALB. Creating an ALB automatically would be messy, and it's totally possible to (and very likely that you will - see note below) share one ALB for multiple APIs.

CloudFormation does not yet support TargetGroup with Lambda behind it. I've implemented the required resource(s) here: https://github.com/silvermine/cloudformation-custom-resources

I give a tutorial on how to get it up and running (using Serverless) in less than ten minutes here: https://serverless-training.com/articles/how-to-set-up-application-load-balancer-with-lambda/

Hope that helps. I'd love to see this feature eventually land in the framework. Basically, I think it would be like this: In the events area of your function definition, you specify HTTP(s) events similar to what was done for APIGW. You also provide an ARN to one or more existing ALB listeners (not the ALB itself). The ALB and its listener(s) be a ref to an ALB / listener in the same stack, or one in another stack. Then Serverless would create this:

• Target group for each function (example)
• Associate the function with the target (example)
• Permission for ELB to invoke the function (example)
• The path-mapping rule(s) for the function (example) which ties the function to the appropriate listener(s).

It's not 100% straight-forward to do, and there would need to be some well-thought-out design put into it, but it should be possible (once CloudFormation supports the necessary resources). If someone gets ready to implement this, please ping me - I'd love to be part of the design discussion because we're starting to run all our APIs through ALB. We launched our first one yesterday and assuming it continues doing well, we'll migrate our other dozen or so APIs in the next couple weeks (thus the reason I wrote all those blog posts about it).

Note on why you should share one ALB for multiple APIs: cost savings. An ALB costs hourly, so there's really no reason to run more than one for a collection of related APIs. Even though ALB costs hourly, it can still save you a ton of money if you have busy APIs. See my article on saving money with ALB / Lambda if you're interested in more details.

[benbunk]

In our organization we "Nuke-and-Pave" for all releases essentially destroying and recreating everything including ALBs. It would be beneficial for us if the ALB was created along with the functions so that we can avoid having two separate automation processes.

[tdmalone]

This might be useful to people in the meantime:
https://github.com/jthomerson/serverless-training-examples/blob/master/services/application-load-balancer-lambda/serverless.yml

[komalali]

Looks like lambda is a valid TargetType now, according to this: https://docs.aws.amazon.com/elasticloadbalancing/latest/APIReference/API_CreateTargetGroup.html#API_CreateTargetGroup_RequestParameters

Was this the only part that was missing cf support?

[jthomerson]

@komalali Even though TargetType is supported, the rest of the CloudFormation integration of ALB/Lambda is broken. As of last week, it still had these errors:

If you don't specify Port you will get this error from CloudFormation:

An error occurred: TargetGroup - Property Port cannot be empty

And if you do specify Port, you will get an error from AmazonElasticLoadBalancingV2:

An error occurred: TargetGroup - Port cannot be specified for target groups with target type 'lambda' (Service: AmazonElasticLoadBalancingV2; Status Code: 400; Error Code: ValidationError; Request ID: 8d5d15d0-2885-11e9-86e4-d91c062885c7).

I've got you covered with some custom resources and a how-to: https://serverless-training.com/articles/how-to-set-up-application-load-balancer-with-lambda/

[robert2d]

This issue was raised with AWS as far back as December 3rd 2018 The AWS docs are quite misleading which is disappointing if you've built a solution around this working 😞

For now I've just configured the target group and listener manually in the console 😮 until Cloud Formation supports this properly.

Edit: You can also still create the listener Rule and just associate it with the manually created target group which means a smaller change later:

    ListenerRule:
      Type: AWS::ElasticLoadBalancingV2::ListenerRule
      Properties:
        Actions:
        - Type: forward
          TargetGroupArn: ARN_FOR_TARGET_GROUP_WHICH_IS_LAMBDA
        Conditions:
        - Field: path-pattern
          Values:
          - "route-things-matching-this-path-prefix/*"
        ListenerArn: ARN_FOR_LISTENER
        Priority: 1

[triptec]

@robert2d I just looked at the AWS issue you linked and if I read the correctly it was solved in the Frankfurt region but not yet the Paris one. But could we use the Frankfurt one to validate the functionality in serverless?

[abadyan-vonage]

Proper cloudformation support is here!
aws/serverless-application-model#721 (comment)

[pmuens]

Hey everyone 👋

I just took a stab at this and implemented it via #6073.

Can some of you provide some feedback in the PR on what configurability is needed? I tried to start with a pretty simple setup where only existing load balancers can be used via an arn.

Thanks in advance and see you over there...