Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update expected # of plant_in_service rows #2650

Merged
merged 2 commits into from
Jun 9, 2023
Merged

update expected # of plant_in_service rows #2650

merged 2 commits into from
Jun 9, 2023

Conversation

cmgosnell
Copy link
Member

@cmgosnell cmgosnell commented Jun 9, 2023

PR Overview

we now have an additional 156 lines from adding calculation corrections into this plant in service table.

this is how i checked what was going on before updating:

from dagster import AssetKey
from pudl.etl import defs, default_assets

table_name = "plant_in_service_ferc1"
pks = pudl.metadata.classes.Resource.from_id(table_name).schema.primary_key

# I downloaded the old version of the table as a csv and slapped it into my notebook's location
old_df = pd.read_csv(f"{table_name}.csv")
new_df = defs.load_asset_value(AssetKey(table_name))

old_df = old_df.set_index(pks).sort_index()
new_df = new_df.set_index(pks).sort_index()

diff_index = old_df.index.symmetric_difference(new_df.index)
assert(len([c for c in new_df.loc[diff_index].reset_index().ferc_account_label.unique() if "_correction" not in c]) == 0)

PR Checklist

  • Merge the most recent version of the branch you are merging into (probably dev).
  • All CI checks are passing. Run tests locally to debug failures
  • Make sure you've included good docstrings.
  • For major data coverage & analysis changes, run data validation tests
  • Include unit tests for new functions and classes.
  • Defensive data quality/sanity checks in analyses & data processing functions.
  • Update the release notes and reference reference the PR and related issues.
  • Do your own explanatory review of the PR to help the reviewer understand what's going on and identify issues preemptively.

@zaneselvans zaneselvans self-requested a review June 9, 2023 14:01
Copy link
Member

@zaneselvans zaneselvans left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Niiice check snippet.

@codecov
Copy link

codecov bot commented Jun 9, 2023

Codecov Report

Patch coverage: 94.3% and project coverage change: +0.2 🎉

Comparison is base (fe395c9) 86.9% compared to head (d86ecaf) 87.1%.

Additional details and impacted files
@@           Coverage Diff           @@
##             dev   #2650     +/-   ##
=======================================
+ Coverage   86.9%   87.1%   +0.2%     
=======================================
  Files         84      86      +2     
  Lines       9720   10001    +281     
=======================================
+ Hits        8447    8716    +269     
- Misses      1273    1285     +12     
Impacted Files Coverage Δ
src/pudl/metadata/fields.py 100.0% <ø> (ø)
src/pudl/metadata/resources/ferc1.py 100.0% <ø> (ø)
src/pudl/metadata/sources.py 100.0% <ø> (ø)
src/pudl/transform/params/ferc1.py 100.0% <ø> (ø)
src/pudl/extract/xbrl.py 94.2% <33.3%> (-2.8%) ⬇️
src/pudl/ferc_to_sqlite/cli.py 70.7% <71.4%> (-0.7%) ⬇️
src/pudl/workspace/datastore.py 73.8% <71.4%> (+0.6%) ⬆️
src/pudl/extract/dbf.py 88.8% <87.8%> (-0.2%) ⬇️
src/pudl/extract/ferc.py 91.6% <91.6%> (ø)
src/pudl/transform/ferc1.py 96.6% <98.6%> (+0.4%) ⬆️
... and 4 more

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

@zaneselvans zaneselvans merged commit f7d0566 into dev Jun 9, 2023
3 of 5 checks passed
@zaneselvans zaneselvans deleted the pis_len_update branch June 9, 2023 23:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

None yet

2 participants