Population Service

Project

What

• Demonstrate an exponentially expensive task of extracting, transforming, loading, serving, and consuming data
• Specification

Why

• Demonstrate how to maximize efficiency by using appropriate languages, infrastructure, and concurrency
• Demonstrate a simple and delightful way to deliver scaleable software
• Highlight business value by demonstrating computational, and human process efficiencies which result in on-demand and extremely low operational costs

How

• This project is organized into three distinct services
  • Importer: Extract, transform, and load population data from CSV files into an embedded database
  • Server: For a given zip code, retrieve and respond with population metadata
  • Client: Consume Server API

+------------+        +------------+        +------------+
|            |        |            |        |            |
|            |        |            |        |            |
|  Importer  --------->   Server   <-------->   Client   |
|            |        |            |        |            |
|            |        |            |        |            |
+------------+        +------------+        +------------+

Arrows signify flow of data

• Architecture

KPIs / Goals

• Import processing time: < 5 minutes (~2s actual!)
• API request processing time: < 100ms (~4ms actual!)
• Autoscaling: true (true actual!)
• Deployment steps: 3 (2 actual!)
• Operational cost: < $10/mo ($0 actual!)

Stack

• Application
  • Ruby
  • Go
  • BoltDB
• Development
  • Github
  • CodeClimate
• Delivery
  • Cloud Build
  • Container Registry
• Infrastructure
  • Cloud Run

TODO

After this proof of concept has been vetted and approved, I would prioritize the following:

• Add tests
• Add CI/CD
• Refactor code
• Clarify documentation