Repository to showcase a cloud engineering project bringing together a simple FastAPI REST API app with database backing store using SQLAlchemy and targeting AWS infrastructure using Terraform.
The AWS infrastraucture is provisioned with Terraform for deploying the FastAPI Lambda function. It creates an API using an AWS API Gateway and Lambda function, based on this walkthrough. The setup uses:
Requests are sent to the API Gateway, which handles all requests using a proxy integration with the Lambda function. Mangum acts as a wrapper, which allows FastAPI to handle the requests and create responses the API gateway can serve. All logs are sent to CloudWatch log groups.
┌─── AWS region ───────────────────────────────────┐
│ ┌─── VPC: 1..n AZs ──────────┐ │
│ │ ┌──────────────────────┐ │ │
Request ───► API Gateway ──────► Lambda (FastAPI) │ │ │
│ │ └────────────────│─────┘ │ │
│ │ ┌────────│───┐ │ │
│ │ │ RDS ▼ │ │ │
│ │ └────────────┘ │ │
│ │ └───│────────────────────────┘ │
│ ┌───────│────────────│─────┐ │
│ │ ▼ CloudWatch ▼ │ │
│ └──────────────────────────┘ │
└──────────────────────────────────────────────────┘
CloudWatch is not currently implemented.
If performance becomes an issue, a CloudFront distribution could be added for API responses that are cacheable (i.e. the READ operations), however the cache would need to be refreshed after each WRITE operation.
Start by navigating into the infrastructure/
directory and performing the following commands:
cd infrastructure/
make create-deployment
For tearing down the deployment:
cd infrastructure/
make destroy-deployment
NOTE: The environment variables
$AWS_SECRET_ACCESS_KEY
and$AWS_ACCESS_KEY_ID
must be set in the user environment in order to make changes to AWS infrstructure.
If you wish to create a terraform state that is shared between users/machines (e.g. in DevOps scenarios) then first create an s3 bucket using this guide. Once done make main.tf
can be used to guide the user into configuring the TF backend to use S3.
Place your ca.crt
, certificate.crt
and certificate.key
files for your custom domain under infrastructure/certs
and add an infrastructure/overrides.tf
file containing values for the following variables:
api_gateway_domain
api_gateway_certificate_name
Finally do a make api_gateway_custom_domain.tf
to add the configuration to Terraform. Remove with make clean
.
NOTE: This feature is currently untested.
Development is assumed to be in a Linux environment with the following tools available:
- bash
- pyenv (or python3.8,
.python-version
contains the exact version used locally) - make
- terraform (use tfenv and version within
.terraform-version
)
If you are using Linux then the toolchain can be installed using make install-dev
.
Assuming the aforementioned tools are installed and available on the system PATH,
then setting up the python virtual environment for development is as simple as
make init
.
To start a local test server, do:
make serve
Provided the virtual environment has been initialised correctly, this will serve a local REST API under http://127.0.0.1:8000. The OpenAPI (Swagger) documentation is available at http://127.0.0.1:8000/docs and can be used to test endpoints with/without authentication.
To run the API tests locally (after setting up the dev environment as described above):
make clean test # clean is currently required due to async SQLite driver needing to point to a file.
- Implement SSO authentication with JWT
- Send an email notification / allow subscription to Asset creation events
- Use
alembic
for handling database migrations - Add FastAPI exception handlers to handle different python errors intuitively.
- CI for terraform module validation and security checks