Update EIA 176 archiver to grab CSVs, rather than bundled data

[e-belfer]

We're currently downloading the 'bundled' EIA 176 data (https://www.eia.gov/naturalgas/ngqs/all_ng_data.zip). This data large CSVs with all years of data, contains only 2017 data for form 757 and a LINE column requiring translation for EIA 176. Instead, let's shift to querying the EIA website for CSVs for individual years of data from the form partitions, as discussed here.

For 191 and 757 data, this will look like querying the subsections of the form (e.g., 191 monthly) into individual CSVs, and zipping them.

For 176, there are subsections of the form available but they don't seem to capture all lines. Instead, we'll use the "custom export" endpoint to iteratively download chunks of the form at a time, and then combine them all together into one annual CSV. This will look like querying the items endpoint, getting a list of all possible 176 line codes, and iteratively building URLs to query and compile all lines of the 176 form for a single year into one CSV, which is then zipped.

In the end, this should look like:

• three archivers, one each for EIA forms 176, 757, 191. 757 and 191 should be quite similar.
• one zipfile per year, containing one file per section with all form fields
• for 757: an archiver that tries all combinations of years and logs a note if they're not available (to capture irregular years of reporting for 757), or uses the report endpoint to generate start and end years

Tasks

Beta Give feedback

1. Write archiver for 176 that iteratively gets all items and compiles them
2. Add EIA Forms 191 and 757 to sources in PUDL metadata pudl#3304
   1 of 1
   eia176 new-data +2
   
3. Write archiver for 191
4. Write near-identical archiver for 757