Demand mapping

[yashkumar1803]

Added new visualization functions, which can be analyzed in the electricity-demand-mapping repo

[codecov]

Codecov Report

Merging #717 into sprint22 will decrease coverage by 3.17%.
The diff coverage is 6.73%.

@@             Coverage Diff              @@
##           sprint22     #717      +/-   ##
============================================
- Coverage     74.39%   71.22%   -3.17%     
============================================
  Files            39       39              
  Lines          4639     4819     +180     
============================================
- Hits           3451     3432      -19     
- Misses         1188     1387     +199     

Impacted Files	Coverage Δ
src/pudl/analysis/service_territory.py	21.60% <0.00%> (-0.35%)	⬇️
src/pudl/helpers.py	87.02% <ø> (ø)
src/pudl/output/ferc714.py	17.74% <ø> (ø)
src/pudl/transform/eia861.py	96.57% <ø> (ø)
src/pudl/analysis/demand_mapping.py	9.52% <6.80%> (-2.21%)	⬇️
src/pudl/workspace/datastore.py	43.02% <0.00%> (-17.88%)	⬇️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 200dab7...e5bab5f. Read the comment docs.

[zaneselvans]

Hey @yashkumar1803 it's not technically part of this PR, but I do want to check in about a few things in the most recent version of your notebook, over in the demand mapping repo.

• Is there a reason that you're pulling the tract level layer of the census geometries, and then dissolving them to the county level, rather than directly using the provided county layer of the census data?
• There's a function integrated into PUDL for obtaining the Census DP1 and storing it locally:
  • pudl.analysis.service_territory.get_census2010_gdf() -- and you can set the desired layer to state, county, or tract.
• The FERC 714 doesn't need to be obtained independently, as it will be downloaded automatically when you ask for the FERC 714 ETL to be run, which happens automatically inside of the PUDL output object (including inside of the FERC 714 Respondents class). All access to the FERC 714 data should be made through the PUDL output object, the interface ought to be pretty stable at this point. Also, it runs the ETL, so you don't need to do the extract / transform steps in the notebook anywhere. So for example...

pudl_settings = pudl.workspace.setup.get_defaults()
pudl_engine = sa.create_engine(pudl_settings['pudl_db'])
pudl_out = pudl.output.pudltabl.PudlTabl(pudl_engine)

# Get the county level Census geometries / data:
county_gdf = pudl.analysis.service_territory.get_census2010_gdf(pudl_settings, layer="counties")

ferc714_out = pudl.output.ferc714.Respondents(pudl_out)
ba_county_map = ferc714_out.georef_counties()

# FERC 714 hourly demand data:
dhpa_ferc714 = pudl_out.demand_hourly_pa_ferc714()

• pd.read_csv() is happy to take a pathlib.Path object as an argument -- you don't need to str() the path first.
• To select records from a given year, when you have a date field, you can use the datetime accessor for the datetime column so like ba_county_map_2010 = ba_county_map[ba_county_map.report_date.dt.year == 2010]

[zaneselvans]

I didn't get to everything, but there's a lot here already. We should probably merge it, but keep working on the module as a whole in smaller chunks, one function at a time.

[zaneselvans]

In the notebook output for this map, the PJM territory seems vastly overrepresented, since there are several FERC 714 respondent IDs that have been assigned to the PJM EIA ID. However, only one of them has any demand associated with it. Would it make sense to exclude any area that doesn't have any reported demand at all in the year in question from consideration? Probable sometime early on -- the annualized() version of the FERC714 respondents output includes the annual sum of that respondent's reported demand.

[zaneselvans]

This function does a very particular thing, and I think it might be better separated into a few smaller functions that are more reusable -- like one which, given a year of hourly data (a Datetime index + a data column) makes a heatmap, and another separate one which takes 2 hourly demand allocations and calculates the delta between them, the output for which can be fed into the plotting function. That we we can make other 365 x 24 heatmaps to show other kinds of variables that make sense that way too.

Visually, can we scale it down so it fits on one screen? I think the Y-axis can also be simplified to have day-of-year (integer) as the labels. Could label every 7 days to highlight the weekly pattern? Could just have the first letters of the months JFMAMJJASOND. Allowing the UTC time to be localized for display so that the zero hour is local zero would also be good, since it would make the familiar diurnal pattern clearer, and allow uniform visual comparison between different plots of this type.