Demo application for an AWS Fargate stack, without needing your own domain and certificate.
- ✔️ Requests can be securely routed through API Gateway to a containerised Express server behind a private load balancer
- ✔️ Secure cookies work fine with HTTP API Gateway and a little Express wrangling
- ❌ Secure cookies will not make it through REST API Gateway
While working on the Prompt Injection Mitigation repo, I was tasked with generating infrastructure-as-code templates for deploying resources to AWS. The project has a standard React SPA UI, and a containerised Node Express back-end. For the infrastructure, I am using AWS CDK framework as it makes the stacks super-easy and even enjoyable to define! Not something you can say about any of the YAML-based template solutions (yes I'm talking about you: CloudFormation, Terraform, Serverless et al).
For the UI, it is trivial to provision and deploy our bundle to an S3 bucket behind [CloudFront](https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/Introduction.html. But the API has a stumbling block: we are using a Secure Cookie to pass a session ID between UI and API, which Express will not send if it thinks the connection is insecure (i.e. http not https). While you can tell Express to trust proxy servers in between, this currently relies on X-Forwarded-* headers being correctly added by the proxy server(s) to represent the client origin.
Our API stack will be using AWS Fargate to deploy and manage our container, fronted by an Application Load Balancer (ALB), which does indeed append these headers. Our production environment turns out to be rather flexible to configure, because in addition to the ALB being secure (so Express can trust it) we will be using our own domain for both UI and API (via Route53) meaning they are classified as SameSite. This means our cookie doesn't even need to be secure, which would be fine for us as all it contains is a session ID, nothing sensitive.
Running the application locally is also fine, because UI and API are both on localhost - also classed as SameSite despite being on different ports (this ain't CORS, folks).
This all sounds lovely, but we don't yet have a subdomain registered for our application. We might decide to buy a fancy domain of its own, and use ACM for hassle-free certificate management, or we might choose to register a subdomain of scottlogic.com. In either case we want to show we can deploy our application to AWS, fully functional and robust, until we have an application ready to Go Live. And of course tear it down again super-fast.
We won't want to expose our ALB publicly if it is insecure, which leaves us needing some sort of secure proxy in front. Step forward, API Gateway!
There are now two ways to route traffic through API Gateway to a containerised server fronted by a load balancer.
The newer HTTP API from API Gateway is lightweight and costs less than REST API, and as a distinct bonus, it supports integration with ALB. For this, we define a VPC Link to access the ALB within our Virtual Private Cloud (VPC). It all fits together pretty smoothly, despite a hiccup or two with Security Groups.
But gosh darn, it turns out we have a disagreement between AWS services. Crucially, it's a disagreement about which headers to use for identifying client origin and proxies along the route. HTTP API uses the new "standardized" Forwarded header, whereas ALB uses the X-Forwarded- headers which MDN describes as "de-facto standard headers" ... Confused? Me too! But the end result is that our server receives both flavours of header, and even if Express was respecting the Forwarded header, which it currently isn't, it wouldn't know how to combine the headers to reproduce the correct order of client and proxies along the route. Furthermore, for good reason HTTP API does not allow adding or modifying either of these reserved headers, so we cannot even map the values from one header to the other.
So we're up shit creek then? Not quite... The lovely folks at Express provide what you might call a bypass to rewrite properties that Express itself derives from the incoming request. Luckily for us, two of these properties are ip and protocol. By default, Express derives the values of these from the X-Forwarded headers when they exist (and "trust proxy" is set), so because we know we are receiving the Forwarded header instead, we can parse it to extract and set the ip and protocol, ignore the X-Forwarded headers entirely, and fall back to default values derived from the request when the Forwarded header is not found.
Easy when you know how, and ... OMG it actually works! 🥳
Before HTTP API arrived, you could integrate a public REST API with a Network Load Balancer (NLB) inside a VPC, by defining a VPCLink. The NLB will route traffic to the ALB, which in turn routes it to your container(s). That's a lot of services (and costs) and a fair amount of config, so it's far from ideal. But despite a few gotchas along the way, it is possible.
However, it turns out this stack has a showstopper for our secure cookies use case: either REST API is not adding our secure client to the X-Forwarded headers (and REST API does not support modifying headers with a proxy integration), or the NLB is not preserving those headers along the way (which seems unlikely as NLB is a Transport Layer), but in either case there is no way to tell our server it is behind a secure, trusted proxy, and that the request originated from a secure client. So while we can successfully route our requests, we cannot transmit secure cookies, so we cannot preserve sessions.
I have not had time to fully investigate using only a NLB without an ALB to route to our Fargate container, but I do recall the first time I tried, I was unable to get my healthcheck working correctly, so I switched to ALB. Using proxy protocol on our Target Groups looks somewhat promising, if you want to give it a try.
The bottom line: this time we really are up shit creek 🛶
You can run the UI and API locally, to see how it should work. You can then deploy the two flavours of stack to your AWS account and see it working / breaking respectively.
# From this directory:
npm install
# Run these in separate terminals:
npm run start:api
npm run start:ui
If you have not used AWS CLI before, you will need to install it and then configure a profile.
If you have not used CDK to manage your resources before, you will need to bootstrap your AWS environment for CDK. See the cloud README for the commands you'll need.
Refer to the cloud README. In simplest terms: synth, deploy, test ... destroy!
Once the stacks are deployed, you should see the Site URL output on command-line, or you can log into the AWS console and find your distribution in CloudFront. Open the URL in your browser, and click away! Then open the URL in a different browser, or a private window, and see what happens... When it works, each browser or private tab will have its own independent session, and the current state is restored on page refresh. When sessions are broken, you'll likely see the counter becomes stuck on 1.