Estimate redacted EIA 923 fuel prices

[TrentonBush]

Motivation

This project began with our desire to remove our external dependency on the EIA API (see epic #1491 and its issue #1343), but expanded into this Epic after we realized we could improve substantially on EIA's methodology.

Why impute? About a third of the fuel cost data are missing. This data can and has been used by advocates to identify plants with high fuel costs (particularly coal plants) to target for early retirement campaigns. The more complete and accurate this data is, the more opportunities for such action we can support.

Issues:

Systematic Biases

47% of plants have complete data, 47% have no data, only 6% have partial data. It turns out that the data is not missing at random:

• In general IPPs (merchant generators) redact all their fuel prices, and these generators are concentrated in competitive wholesale markets, especially the Northeastern US, where there are essentially no reported fuel prices.
  • In addition, the Northeast has a unique seasonality in its natural gas prices, which would be impossible to infer by sampling data elsewhere in the country.
  • This means we have to use the aggregate data from the EIA API to accurately estimate prices nationwide.
• Major discontinuities in data collection and processing methodology. See details in this comment.
  • The most severe (and most recent) was in 2013, when the temporal resolution changed from annual to monthly. Prior to 2013, monthly resolution data was collected for a sample of plants and the rest calculated via a regression model. Also, the reporting threshold for oil and gas fueled plants changed from 50MW to 200MW.

Scope

A major organizational concern is not necessarily whether we should do this project but rather how much effort to devote to it. Improving model accuracy is a potentially endless spiral of diminishing returns. Is there an accuracy threshold we can call good enough? Is there a certain accuracy improvement per time threshold that we use as a stopping point? We need to define this before getting sucked into the endless labyrinth of interesting technical problems.

Requirements

• Produce a fuel price estimate for every delivery in the fuel_receipts_costs_eia923 table.
• Do not rely on the EIA API in the pipeline, due to reliability issues in CI / testing and user setup difficulties.
• Estimates should be at least as accurate as the coarse aggregations that we've used historically.
• Estimates should be consistent with the spatial and temporal variation we see in the aggregated data (e.g. the seasonal variability that's observed in the aggregated natural gas prices from the Northeastern US).
• The model should be performant enough to run as part of our nightly builds (less than maybe 10 minutes of run time)
• We should avoid new software dependencies if possible

Phase 1: Replace EIA API w/ Bulk Data

Beta Give feedback

1. Create Cross Validation Framework #1709
   data-repair eia923 +2
   
2. Brainstorm Fuel Price Imputation Experiments #1710
   data-repair eia923 +2
3. Iterate on Imputation Model #1711
   data-repair eia923 +2
   
4. Explore external validation sources for fuel imputation #1712
   data-repair eia923 +2
   
5. Create a basic HistGBR Model for EIA-923 fuel price imputation #1714
   data-repair eia923 +2
   
6. Move slowly varying columns from plants_entity_eia to annual plants_eia860 #1748
   eia860 +1
   
7. Sanity check EIA fuel price predictions visually #1720
   analysis data-repair eia923 +3
   
8. Create EIA API v2 fuel price archiving script #1762
   eia923 new-data +2
   
9. Integrate EIA bulk electricity data into our Zenodo archives #1763
   datastore eia923 new-data zenodo +4
   
10. Create ETL functions for stored EIA API fuel price data #1764
    eia923 new-data +2
    

Phase 2: Impute Missing Values

Beta Give feedback

1. Implement fuel price filling using API aggregations #1765
   analysis data-repair eia923 inframundo +4
   
2. Validate EIA Bulk data vs original API source #1896
   +0
   
3. Add fuel price aggregates as features in FRC table #2525
   2 of 5
   analysis data-repair eia923 +3
   
4. Train HistGBDT model to predict fuel price residuals #1766
   analysis data-repair eia923 inframundo +4
   
5. Validate model estimates of redacted EIA fuel prices #1767
   analysis data-repair eia923 inframundo +4