Map PHMSA Natural Gas Transmission Part L columns

[e-belfer]

Overview

Closes #3253.

What problem does this address?
Extracts Part L in a raw format for PHMSA transmission data into one table, raw_phmsagas__yearly_miles_of_pipe_by_class_location. This table includes data from 2001-2022.

What did you change?
Mapped columns for the relevant table, and updated the remaining CSVs in src/pudl/package_data/phmsagas.

Testing

How did you make sure this worked? How can a reviewer verify this?
Materialize the raw asset in dagster. Also run the table through the notebook and verify columns are correctly mapped, for additional peace of mind.

To-do list

Edit tasklist title

Beta Give feedback Tasklist To-do list, more options

• Rename
• Copy markdown
• Delete tasklist

Delete tasklist

Delete tasklist block?

Are you sure? All relationships in this tasklist will be removed.

Cancel Delete

1. Make sure full ETL runs & `make pytest-integration-full` passes locally

   Make sure full ETL runs & make pytest-integration-full passes locally
   Options

   • Convert to issue
   • Toggle completion
   • Rename
   • Remove
2. Update the [release notes](../docs/release_notes.rst): reference the PR and related issues.

   Update the release notes: reference the PR and related issues.
   Options

   • Convert to issue
   • Toggle completion
   • Rename
   • Remove
3. Review the PR yourself and call out any questions or issues you have

   Review the PR yourself and call out any questions or issues you have
   Options

   • Convert to issue
   • Toggle completion
   • Rename
   • Remove

Add item to To-do list

[codecov]

Codecov Report

Attention: 6 lines in your changes are missing coverage. Please review.

Comparison is base (b3b1a26) 92.7% compared to head (848cfd4) 92.7%.
Report is 41 commits behind head on main.

Files	Patch %	Lines
src/pudl/extract/phmsagas.py	44.4%	5 Missing ⚠️
src/pudl/extract/excel.py	50.0%	1 Missing ⚠️

Additional details and impacted files

@@          Coverage Diff          @@
##            main   #3254   +/-   ##
=====================================
  Coverage   92.7%   92.7%           
=====================================
  Files        144     144           
  Lines      13087   13091    +4     
=====================================
+ Hits       12128   12134    +6     
+ Misses       959     957    -2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[jdangerx]

One non-blocking question about standardizing our column name orders.

[jdangerx]

I think we are a little inconsistent about the ordering of onshore/offshore and transmission/gathering:

> ag ^onshore_ -c 
yearly_miles_of_pipe_by_class_location.csv:25
yearly_miles_of_transmission_pipe_by_nps.csv:35
yearly_miles_of_gathering_pipe_by_nps.csv:95
> ag ^transmission_onshore -c
yearly_transmission_gathering_summary_by_commodity.csv:1
> ag ^gathering_onshore -c 
yearly_transmission_gathering_summary_by_commodity.csv:4

It appears that "shoreyness" comes before "transmissioniness" in most cases, should we standardize that everywhere?

[e-belfer]

I think in the transmission_pipe_by_nps and gathering_pipe_by_nps transmission/gathering are implied, so they're not explicitly spelled out. I can flip the summary_by_commodity usage to more closely reflect the one in class_location though.