Rework the earlier API and launch as a v0.1 effort

[gbinal]

No description provided.

[jmhooper]

#152 contains the original work for the API

[jmhooper]

At this point, this is what I'm thinking:

We can add a flag to the command (similar to --publish) that tells the reporter to write the reports to a postgres db. The db will have one table for the data points described in the data array. Each entry element in the array will be an array in the table. I'm thinking something like this:

• id
• report_name: The name of the report the data is associated with
• agency_name: The name of the agency the data is associated with. Null if gov-wide
• date: The date associated with the data
• data: jsonb column with the data point, minus the data prop if present
• created_at
• updated_at

For reports that are not realtime, we can check for a ga:date dimension. For each report with one, we can merge into the existing data for that report. This means that for each point we look up whether a record already exists. If it does, we update it. If it doesn't we insert it. I'm a bit concerned about performance here b/c some of the reports are long, but we can try it and see what happens.

Realtime reports are easier b/c we do not need to check for existing records. We can simply set the date to the current date and insert.

We can build an API app that ties into the same database. I'm a fan of allowing that app to manage the db schema b/c a web app framework will likely have better tooling for managing schema.

[gbinal]

This is done.

[gbinal]

https://analytics.usa.gov/developer