Can I use Consul Connect in AWS Lambda?

[rts-gordon]

Feature Description

Use Consul Connect in AWS Lambda.

Use Case(s)

We have launched a lots of mircoservices in AWS Lambda, we want to add Service Discovery and Service Registry feature in Lambda. Can we use Consul Connect in AWS Lambda, and would you like to give me some examples. Thanks a lot.

[nicholasjackson]

Hi @Chcp, there are a couple of solutions to do this. In all the options below I have assumed that you are interested in integrating non lambda based applications and that lambda to lambda calls would use standard ARN routing and security policy.

One of the core problems is the Lambda execution model. Lambda is designed to scale as the load on your system increases. All the facets of Lambda such as the recommended data stores you use with it are designed to work with this horizontal scale. In the instance that Lambda needs to connect to a system which is outside of this model the user must handle these requirements. To participate in a Service Mesh a Lambda function needs an identity which is in the form of an x509 certificate issued by the Consul server. While Lambda functions do not immediately scale to 0 after a request has finished the lifecycle is not controllable by the user. Consideration needs to be given that peaks in load on the system will cause new Lambda instances to be created, this will increase the load on the Server signing the identity requests.

Custom runtime
It is possible to create a custom runtime in Lambda, you could bundle Envoy as a side process which intercepts ingress and egress from the function. Envoy would connect to Consul in the standard way to obtain endpoints and identity.

https://github.com/awslabs/aws-lambda-container-image-converter

Advantages:
All of the features L7 and security can be leveraged thanks to Envoy proxy
Lambda retains single identity and access can be controlled through service mesh authentication (mTLS) and authorisation (Intentions)

Disadvantages:
Slower function start times
Potential load on Consul in the instance you have many functions which start simultaneously. For every function instance started Consul needs to sign certificates for mTLS
Additional complexity of over standard lambda approach

Go SDK
It is possible to use the Go SDK inside of AWS lambda to validate ingress and also provide egress using upstream model. I have done a couple of experiments with this approach, some old code can be found in the repo below.

(https://www.consul.io/docs/connect/native.html)[https://www.consul.io/docs/connect/native.html]

https://github.com/nicholasjackson/consul-connect-router/blob/master/lambda/main.go

Advantages:
Lambda retains single identity and access can be controlled through service mesh authentication (mTLS) and authorisation (Intentions)
Go SDK has very low latency

Disadvantages:
L4 only from the SDK, L7 capabilities such as routing and reliability need to be manually built
SDK is currently only available for GoLang, Consuls API does support all the features needed to obtain certificates and discovery, but again there would need to be engineering effort to make this work
Slower function start times
Potential load on Consul in the instance you have many functions which start simultaneously. For every function instance started Consul needs to sign certificates for mTLS
Complexity of over standard lambda approach

Ingress / Egress Gateway
It would be possible to de-couple the sidecar process and run Envoy in a VM autoscale group which would be used to proxy traffic to and from AWS lambda. This approach would require a trusted network zone between AWS lambda and the proxy VMs running the proxy.

https://aws.amazon.com/blogs/compute/announcing-improved-vpc-networking-for-aws-lambda-functions/

Advantages:
Decoupled proxy from lambda invocation gives predicable load on Consul
Proxy is only forwarding network traffic and should be capable of multiplexing many lambda connections
Standard autoscale mechanisms can be employed to control the gateway group size

Disadvantages:
Complexity of over standard lambda approach
Potentially reduced security as a Gateway approach treats all lambda functions as a single identity
Additional cost for running gateway VMs

[rts-gordon]

Hi @nicholasjackson ,
Thanks for your so detail answer, I will learn about this.

[schristoff]

Hey -
It looks like this issue has been answered at this time. I am going to go ahead and close this issue. If you think it should remain open, please comment back and let us know why!

Thanks!