Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge dev into dagster-asset-etl once again #2353

Merged
merged 107 commits into from
Mar 6, 2023

Conversation

jdangerx
Copy link
Member

@jdangerx jdangerx commented Mar 2, 2023

I branched daz/dagster-asset-etl-dev off of dagster-asset-etl, merged dev, and fixed the one merge conflict that was there. Hope CI passes!

e-belfer and others added 30 commits January 31, 2023 16:32
@codecov
Copy link

codecov bot commented Mar 2, 2023

Codecov Report

Patch coverage: 97.3% and project coverage change: +85.9 🎉

Comparison is base (15b3196) 0.0% compared to head (4702f91) 85.9%.

Additional details and impacted files
@@                 Coverage Diff                  @@
##           dagster-asset-etl   #2353      +/-   ##
====================================================
+ Coverage                   0   85.9%   +85.9%     
====================================================
  Files                      0      79      +79     
  Lines                      0    9638    +9638     
====================================================
+ Hits                       0    8284    +8284     
- Misses                     0    1354    +1354     
Impacted Files Coverage Δ
src/pudl/analysis/allocate_net_gen.py 97.0% <ø> (ø)
src/pudl/analysis/mcoe.py 90.4% <ø> (ø)
src/pudl/extract/eia860.py 100.0% <ø> (ø)
src/pudl/extract/excel.py 94.6% <ø> (ø)
src/pudl/glue/epacamd_eia.py 100.0% <ø> (ø)
src/pudl/metadata/codes.py 100.0% <ø> (ø)
src/pudl/metadata/fields.py 100.0% <ø> (ø)
src/pudl/metadata/resources/eia.py 100.0% <ø> (ø)
src/pudl/metadata/resources/eia860.py 100.0% <ø> (ø)
src/pudl/metadata/resources/glue.py 100.0% <ø> (ø)
... and 89 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

@jdangerx
Copy link
Member Author

jdangerx commented Mar 3, 2023

Whoops, need to delete load.py. That's ruining our patch coverage too.

Add boiler-associated attributes from EIA 860 6.2 EnvrEquip tables to ETL
@jdangerx jdangerx closed this Mar 3, 2023
@jdangerx jdangerx reopened this Mar 3, 2023
Copy link
Member Author

@jdangerx jdangerx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One other thing to note: my unit tests fail on the foreign key error still. When I removed pudl.sqlite they passed, but I think it's just because there's now no data in the pudl.sqlite...

setup.py Show resolved Hide resolved
src/pudl/transform/eia860.py Outdated Show resolved Hide resolved
@jdangerx jdangerx requested a review from e-belfer March 3, 2023 17:12
@jdangerx jdangerx changed the title Daz/dagster asset etl dev Merge dev into dagster-asset-etl once again Mar 3, 2023
@bendnorman
Copy link
Member

The CI is failing because the raw_boiler_particulate_eia860 table needs to be added to pudl.extract.eia860.raw_table_names.

Copy link
Member

@zaneselvans zaneselvans left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comments that @bendnorman made about the new post-harvesting boilers_eia860 table and the clean_boilers_eia860 asset (that doesn't get written into the DB) sound right to me.

@bendnorman bendnorman mentioned this pull request Mar 4, 2023
8 tasks
dependabot bot and others added 8 commits March 6, 2023 07:57
Updates the requirements on [gcsfs](https://github.com/fsspec/gcsfs) to permit the latest version.
- [Release notes](https://github.com/fsspec/gcsfs/releases)
- [Commits](https://github.com/fsspec/gcsfs/commits)

---
updated-dependencies:
- dependency-name: gcsfs
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Updates the requirements on [dask](https://github.com/dask/dask) to permit the latest version.
- [Release notes](https://github.com/dask/dask/releases)
- [Changelog](https://github.com/dask/dask/blob/main/docs/release-procedure.md)
- [Commits](dask/dask@2021.08.0...2023.3.0)

---
updated-dependencies:
- dependency-name: dask
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
…/gcsfs-gte-2021.7-and-lt-2023.3.1

Update gcsfs requirement from <2023.1.1,>=2021.7 to >=2021.7,<2023.3.1
…/dask-gte-2021.8-and-lt-2023.3.1

Update dask requirement from <2023.2.2,>=2021.8 to >=2021.8,<2023.3.1
Updates the requirements on [fsspec](https://github.com/fsspec/filesystem_spec) to permit the latest version.
- [Release notes](https://github.com/fsspec/filesystem_spec/releases)
- [Commits](fsspec/filesystem_spec@2021.07.0...2023.3.0)

---
updated-dependencies:
- dependency-name: fsspec
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
…/fsspec-gte-2021.7-and-lt-2023.3.1

Update fsspec requirement from <2023.1.1,>=2021.7 to >=2021.7,<2023.3.1
@jdangerx jdangerx merged commit b4f7c7c into dagster-asset-etl Mar 6, 2023
@jdangerx jdangerx deleted the daz/dagster-asset-etl-dev branch March 6, 2023 23:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

None yet

5 participants