Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Q4 CEMS data #3379

Merged
merged 4 commits into from Feb 10, 2024
Merged

Add Q4 CEMS data #3379

merged 4 commits into from Feb 10, 2024

Conversation

e-belfer
Copy link
Member

@e-belfer e-belfer commented Feb 9, 2024

Overview

Closes #3315

What problem does this address?
Adds Q4 2023 CEMS data.

What did you change?
Added Q4 data to extraction, updated docs.

Testing

How did you make sure this worked? How can a reviewer verify this?
Materialize the EPA CEMS dagster assets.

To-do list

Edit tasklist title
Beta Give feedback Tasklist To-do list, more options

Delete tasklist

Delete tasklist block?
Are you sure? All relationships in this tasklist will be removed.
  1. Ensure docs build, unit & integration tests, and test coverage pass locally with make pytest-coverage (otherwise the merge queue may reject your PR)
    Options
  2. Update the release notes: reference the PR and related issues.
    Options
  3. Review the PR yourself and call out any questions or issues you have
    Options

@e-belfer e-belfer added epacems Integration and analysis of the EPA CEMS dataset. new-data Requests for integration of new data. labels Feb 9, 2024
@e-belfer e-belfer self-assigned this Feb 9, 2024
@e-belfer e-belfer added this to the v2024.02 milestone Feb 9, 2024
@aesharpe
Copy link
Member

aesharpe commented Feb 9, 2024

Do we want to update the etl_fast.yml file to say cems year_quarters: ["2023q4"] instead of year_quarters: ["2022q1"]?

@e-belfer
Copy link
Member Author

e-belfer commented Feb 9, 2024

Do we want to update the etl_fast.yml file to say cems year_quarters: ["2023q4"] instead of year_quarters: ["2022q1"]?

I think we need to match the same year of CEMS and 860 so the plants are all matched, so I was deferring to comparing CEMS to the full year of EPA data rather than just the 860M data?

@aesharpe
Copy link
Member

aesharpe commented Feb 9, 2024

I was deferring to comparing CEMS to the full year of EPA data rather than just the 860M data?

Ok that's probably a good idea.

Copy link
Member

@aesharpe aesharpe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Everything looks good - I wanted to try running it locally to be sure, but it's taking a long time, so I'm going to approve! If you've loaded it locally / checked the tables to make sure everything is there as expected, I think you can go ahead and add it to the queue.

@e-belfer e-belfer added this pull request to the merge queue Feb 10, 2024
Merged via the queue into main with commit 0d9f396 Feb 10, 2024
14 checks passed
@e-belfer e-belfer deleted the new-cems-quarter branch February 10, 2024 01:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
epacems Integration and analysis of the EPA CEMS dataset. new-data Requests for integration of new data.
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

Q4-2023 Updates to CEMS Data
2 participants