PRAwN Stack

🦐 🍤 🦐 🍤 🦐 🍤 🦐 🍤 🦐 🍤 🦐 🍤

A modern page view counter to see how unpopular my project is. Powered by a PRAwN stack (Postgres, React, AWs and Node) in the free tier, deployed using CDK.

• Postgres.
• React.
• Aws.
• Node.

We're also using Typescript and NextJS for the frontend.

Features

• Relational database so you can make cool stateful apps like a page view counter.
• Fun local development experience with docker-compose.
• Easy to deploy with AWS CDK.
• Almost completely in the AWS Free Tier. Turns out this is actually quite difficult with RDS and Lambda.

No Longer Available

My free tier has run out once again and I don't feel like setting up another account right now.

Here's the bill post-free tier.

Design

Mermaid Diagram

Performant Analytics with Rollups

We have a graph that shows the pageviews per hour and there's a few ways to make that happen.

1. Calculate the aggregates when they're requested. This is the easiest and will be up to date but it can be slow and resource intensive if it's requested often.
2. Setup a materialised view and refresh it on an interval. This is a natural extension of the first solution but will be much faster at the cost consistency because it might be stale. This also might be resource intensive if the table is big enough or the aggregates are complicated enough.
3. Iteratively calculate the aggregates and store them manually. The materialised view solution might recalculate aggregates that aren't going to change because they might be in the past. E.g the pageviews for last month aren't going to change to why recalculate them. This is the approach we've taken here.

I would recommend starting with the first one then escalating to more complicated solutions as required but I wanted to explore the third solution with this project.

Serverless and Lambda

I'm a serverless skeptic. A few timely things swayed me to try Lambda for this project.

1. serverless-express allows you to run an express app in Lambda, which means you can run the express part locally for a nice development experience.
2. I wanted to setup some cron jobs which are well suited to Lamdbda.
3. AWS CDK makes working with AWS easy less hard.

It turns out the simplest way to setup Lambda with RDS is quite expensive. You have to use a NAT Gateway which costs $0.059/h and $44.25/month. That's expensive!.

There's a few ways to get around this:

• Make your database public. This isn't secure.
• Make your Lambda private so it can't access the internet. The internet is useful though and you'll have to pay to access AWS services like Secret Manager.
• Setup a NAT Instance in the free tier.

This all feels very typical of AWS.

I went with the NAT Instance because thankfully CDK makes it easy.

Almost Completely in the Free Tier

The only thing you have to pay for is Secrets Manager which is $0.4/month.

When the free tier ends after 12 months you could move this onto a t3.micro for rds at $0.028/h, $21/month) and a t3.nano for ec2 (used as a cheap NAT) at $0.0066/h, $4.95/month.

There's also a budget configured with cdk to alert you when you spend more than $1 usd to remind you when the free tier ends.

Load Testing

With more load, our application can support a throughput of 570 requests per second on average or about 49.2 million per day at 350ms latency. 350ms is still pretty fast and if we had a higher tolerance for latency then this stack could probably do more.

You can install wrk with:

brew install wrk

Then run it with:

wrk -t1 -c1 -d60s https://prawn.cadell.dev/api/home

If you want to go further with load testing then maybe have a look at vegeta, wrk2 or k6.

Load testing goes really deep apparently but I found How percentile approximation works (and why it's more useful than averages) was a good introduction to percentiles, as well as the hacker news comments, which is where I found How NOT to Measure Latency by Gil Tene, which is amazing! There's also an article similar to the talk but I much prefer the talk (and I don't usually watch talks).

Vegeta, wrk2 and k6 all avoid the coordinated omission problem mentioned in Gil's talk, if you use constant throughput modes. It's called something slightly different in k6 though.

Scripts

• docker-compose up --build starts postgres, the node api development server and the nextjs development server.
• yarn start run the development Node server.
• yarn dev run the NextJS development server.
• yarn test perform the jest unit tests.
• yarn build build the NextJS frontend so it's ready to deploy.
• yarn cdk bootstrap --profile account-name prepare the AWS region for cdk deployments.
• yarn deploy --profile account-name build the frontend and deploy the cdk stack.
• yarn cdk destroy --profile account-name destroy the deployment.

Running Locally

1. Install dependencies.

   yarn
   

2. Run docker-compose.

   docker-compose up --build
   

   This will bring up:
   • Postgres database and run migrations on it using Flyway.
   • PgAdmin to access the Postgres database. Available at http://localhost:5050/.
   • Node API development server. Available at http://localhost:3001/api/.
   • NextJS development server for the frontend. Available at [http://localhost:3000](http://localhost:3000.

Going forward, you can make this faster by skipping the build step. You only need it if your dependencies change.

docker-compose up

Accessing the Local Database

1. Go to http://localhost:5050/.
2. Set a master password.
3. Click Add New Server.
4. Fill in the local server details.
   • General
     • Name: local
   • Connection
     • Host: postgres
     • Username: postgres
     • Password: changeme
     • Save password?: yes

First Deploy

Make sure you setup email forwarding for admin @yourdomain (e.g admin@cadell.dev) you receive the email from aws to validate that you own the domain so it can create a certificate.

1. Create an AWS account.
2. Click Signin.
3. Click Create a new AWS account.
4. Enter your details.
5. Go to IAM.
6. Enable MFA on your Root account. I recommend 1Password.
7. Create an IAM user with programmtic access and assign the AdministratorAccess permission.
8. Download the credentials.
9. Create a .aws folder if you don't already have one.

   mkdir ~/.aws
   

10. Create a credential file with vi ~/.aws/credentials.

    [account-name]
    aws_access_key_id=
    aws_secret_access_key=
    

11. Run yarn to install dependencies.
12. Set the region in bin/prawn-cdk.ts.
13. Update yourPublicIpAddress in lib/prawn-stack.ts with your public ip address so you can access the database.
14. Run yarn cdk bootstrap --profile account-name.
15. Run yarn cdk deploy CertificateStack --profile account-name.
    • This will send an email to admin@yourdomain to confirm you control the domain for the certificate it's creating. You can also go into CertificateManager in the us-east-1 region and resend it if you need to.
    • The command won't finish until you approve the email.
16. Copy the certificateArn from the output and add it to the bin/prawn-cdk.ts.
17. Run yarn deploy --profile account-name.
    • This takes about about 15 mins.

Future Deploys

This takes about 1-3 mins.

yarn deploy --profile account-name

Access the Database

You'll need to do this to setup the database initially.

Access is currently through a whitelisted ip address which isn't ideal but will work well enough for now.

1. Make sure the yourPublicIpAddress in lib/prawn-stack.ts is up to date and deployed.
2. Login to the AWS console.
3. Select the region, probably ap-southeast-2.
4. Go to Secret Manager.
5. Click PrawnStack-rds-credentials.
6. Go to the Secret value section then click Retrieve secret value.

You can put the values into PgAdmin to query the database.

Setup the Database

Run the following command to migrate the database with Flyway.

HOST=xxx USER=xxx PASSWORD=xxx; docker-compose run flyway -url=jdbc:postgresql://$HOST/postgres -user=$USER -password=$PASSWORD migrate

Setup a Custom Domain

1. Sign into your domain registrar. I use Google Domains.
2. Setup a new CNAME DNS record using the coudfront domain in the deploy output.
3. Make sure the CNAME aligns with the customDomain in bin/prawn-cdk.ts.

Destroy the Stack

yarn cdk destroy --profile account-name

Why is AWS Charging You?

1. Login to the root account.
2. On the account dropdown on the top right, click My Billing Dashboard.
3. Click Cost Explorer.
4. Click Daily Spend View.
5. Change the timeframe to the last 7 days.
6. Group by Usage Type.

The table below will give you a decent breakdown on your charges.

Inspiration

The LAMP Stack (Linux, Apache, MySQL, PHP) is the inspiration for this project. I made a terrible chat app with it when I was in year 10 to get around the school's internet filter but it worked and it was really fun.

The best parts:

• Free hosting with dodgy hosts.
• Good local development with MAMP.
• Relational database.
• Fun.

Some other people also agree.

I wanted to see if I could create a stack with similar qualities with more modern tools.

Roadmap

Ideas

1. Email me when the scheduled pageview fails, for basic monitoring.
2. Setup integration tests.
3. Set a maximum Lambda concurrency.
4. Use serverless-postgres for connecting to postgres.

Done

• Dark mode.
• Replaced Nivo with VisX for the graph.
• Use MDX for documentation and vanilla-extract for styling.
• Use arm64 in lambdas.
• Optimise the docker environment by excluding node_modules from the bind mount.
• Graph the pageviews per hour with Nivo.
• Setup an hourly pageview.
• Setup an hourly activity rollups for analytics.
• Setup activity table.
• Fixed the latency issue by tweaking cloudfront settings.
• Cache secrets from SecretManager. Didn't make much difference to latency but it does reduce compute time which reduces costs.
• Setup X-Ray tracing.
• Load testing results with wrk.
• Custom domain.
• Setup a budget in cdk.
• Moved setup the api and frontend in docker-compose. It's a bit slow though.
• Setup flyway for database migrations.
• Setup a NAT Instance to save on PrivateLink costs and allow internet access for Lambdas.
• Document the inspirations for this project.
• Setup frontend with nextjs.
• Setup pageViews in deployed environment.
• Set the NAT Gateways to 0 after being charged. Lucky I had a Budget!
• Setup pageViews with local postgres database.
• Setup serverless-express to make routing conistent for local and deployed environments.
• Setup Express to call the same handler as Lambda.
• Deploy a cdk stack with RDS and Lambda.

Related Projects

• cadbox1/backend-frontend-aws-cdk.
• cadbox1/internal-magic-links-aws-cdk.
• cadbox1/node-api-2020.
• cdk-patterns-serverless/the-rds-proxy.
• wclr/ts-node-dev.
• vendia/serverless-express.
• khezen/compose-postgres