lightrail

This document is your guide to Lightrail.

The 50 Foot View

Lightrail is a primarily a REST API. That API has both high-level documentation and an API reference. There is also a webapp that serves as a web console to the API and hosts an interface for the documentation.

The implementation is deployed on AWS and is centrally controlled by lightrail-cloudformation-infrastructure. The REST API is implemented by microservices that mostly don't talk to each other, but when they do they do it through the REST API itself or the Lightail SNS event topic.

The primary microservices are Rothschild covering /v2/currencies, /v2/contacts, /v2/programs, /v2/transactions, /v2/values; Edhi covering /v2/account, /v2/user; KVS covering /v1/storage; and Gutenberg covering /v2/webhooks. This is all defined in lightrail-cloudformation-infrastructure's cloudfront-api-distribution-template.yaml.

Philisophically the microservices could each have been implemented in different ways, but in practice they ended up the same for simplicity. Each microservice (links are to Rothschild as an example) is deployed by a CodePipeline created by lightrail-cloudformation-infrastructure that points to a specific commit of infrastructure/ci.yaml. The CodePipeline deploys infrastrcutrue/sam.yaml which includes resources like databases, and one or more Lambdas. The Lamba source code is in src/lambdas/. REST Lambdas handle multiple HTTP request paths, routing them with Cassava. Other Lambdas may be triggered on a schedule, by an SQS queue or an SNS topic.

Authentication is done with JWTs signed with a secret stored in the lightrailsecureconfig S3 bucket. All JWTs with valid signatures are 100% trusted with no extra steps such as checking that the userId exists. The exception is API keys blocklisted by the WAF WebACL.

The system is set up to be deployed in three accounts: dev, staging and production. Dev and staging are optional. Dev is for development and may have changes being actively developed. Staging is for a dry run of deployment and ironing out any issues before going live. Production is production.

Deploying

• fork all of the Lightrail repos
• for each AWS accounts (dev, staging, production) buy the domain that will host Lightrail in Route53
• create a GitHub account with CI access to your repos
• deploy the lightrail-cloudformation-infrastructure and follow the instructions therein
• for each of the services (rothscild, edhi, kvs, gutenberg, turnkey):
  • set the TemplateURL for the corresponding lightrail-cloudformation-infrastructure module
  • run the lightrail-cloudformation-infrastructure CodePipeline
  • run the service CodePipeline
  • set the RoleNames and SentryDsn in the corresponding lightrail-cloudformation-infrastructure module
  • run the lightrail-cloudformation-infrastructure CodePipeline again

Credentials

Users are defined in lightrail-cloudformation-infrastructure's groups.yaml.

Manual administration is done with the InfrastructureAdmin role.

Monitoring

Lightrail has configuration points for Sentry and Datadog.

Revoking a Customer's Access

Edhi has scripts documented in the README to make this easy.

Deploying a Backend Change

The backend is built on several microservices. Primarily they are Rothschild, Edhi, KVS and Gutenberg. Each of these services is deployed by CodePipeline. This includes both code and infrastructure (via CloudFormation). The staging account is deployed from the staging branch and the production account is deployed from the master branch.

The development flow is:

• develop locally against unit tests (npm run test)
• deploy to dev with ./dev deploy as necessary
• open a GitHub PR to the staging branch
• someone else approves the PR
• merge the PR which automatically starts the staging CodePipeline
• approve the CodePipeline changes in the staging account and watch for it to complete which automatically creates a PR from staging to master
• approve and merge the GitHub PR to master which automatically starts the production CodePipeline
• approve the CodePipeline changes in the production account and watch for it to complete

If you PR directly to master you can shortcut some steps above but I really don't recommend it. Don't be a cowboy.

Unifying infrastructure not in one of the above projects is defined in lightrail-cloudformation-infrastructure. This includes IAM, CloudFront, S3 buckets, SNS topics, KMS keys and CodePipelines. Unlike the above projects that have a CodePipeline in each environment, lightrail-cloudformation-infrastructure has a single CodePipeline called LightrailInfrastructureCI that lives in production. This CodePipeline has stages and permissions to deploy across AWS accounts.

The only infrastructure not managed by any of the above is infrastructure that has to live in the us-east-1 region: domain names, certificates, Lambda@Edge and WAF WebACL. I strongly recommend you manage all changes possible through CloudFormation templates deployed by CodePipeline to keep the system consistent. That is unless you're closing the account. Then you can go wreck house.

Deploying a Webapp Change

See lightrail-webapp.

Updating the Static Site

See lightrail-static.

Shutting it all Down

1. Take down the website by deleting the CloudFront distributions.
2. Backup all production data you may need to refer to later.
   1. Connect to the RDS instance through the bastion host as outlined in the Rothschild readme. Connect to the database with MySQL Workbench and use the data export wizard to export all the data. This data contains personally identifying information so store it somewhere secure.
   2. Download the entire Edhi database with the script export. This data contains personally identifying information so store it somewhere secure.
3. Transfer the domains out of the account.
4. Initiate close account on all 3 accounts (dev, staging, production).
5. Delete Lightrail monitors in DataDog and PagerDuty.