Meteorologica TODO

#Here is a semi-structured brain dump of this repo's TODO list:

##Features

• It would be cool to extract resources which would each take in a date (ddmmyyyy) and return a .csv file of that day's usage. We need to determine how they would signal to Meteorologica that they had finished writing to the file. Perhaps they could move it to a directory, or signal with an HTTP request. This way we could extract the GCP, Azure, and AWS clients as separate repos with binaries to generate. This would also allow future resources to be added easily, such as a Pagerduty resource to track spending.
• Each usage should return Daily Normalized Usage instead of Monthly Normalized Usage
  • Challenge: AWS could never know which day it is getting usage for, only the month. The data does not provide daily granularity. Maybe AWS can return some clever errors depending on the day requested vs the day it is.
  • GCP is easy to get daily usages.
  • Azure can get daily usage because the day is listed in the file.

##Performance

• It would be neat to use streams of data to read from the csv and write to the database instead of doing it in large blocks

##Configuration

• merge flags with configuration struct to cleanup startup
• read db credentials from environment variables
• last-job-ran should be stored and retrieved from database
• use $PORT environment variable

##Testing

• backfill pending tests
• add integration tests
• test database locally (and migrations)
• rename database lock to be more robust (look into this)
• explore best data types for database columns