Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New parser for Bangladesh #5381

Merged
merged 11 commits into from
May 12, 2023
Merged

Conversation

MuffinCompiler
Copy link
Contributor

@MuffinCompiler MuffinCompiler commented May 9, 2023

Issue

Closes #5370

Description

Rewrite the parser for Bangladesh to make use of the new data source, supporting consumption, generation, and exchanges with Northeastern and Eastern India.
Data is being updated only in batches, therefore, most of the time, it does not fulfill the real-time criteria of <2h delay. (But it can from time to time!) Also historical data can be fetched. I added capacities to the config file from their last annual report.

Supported commands:
poetry run test_parser BD generation
poetry run test_parser BD consumption
poetry run test_parser "BD->IN-NE" exchange
poetry run test_parser "BD->IN-EA" exchange
All work with --target_datetime YYYYMMDD as well.

Data also contains loadshed information, probably not important for us. The sum of consumption sources does not add up to the given total generation, I put the rest in 'unknown'.
Weird is also that the capacity for coal in the annual report (page 13) is lower than the current generation. Not sure what to make out of this.

Double check

  • I have tested my parser changes locally with poetry run test_parser "zone_key"
  • I have run pnpx prettier --write . and poetry run format to format my changes.

@github-actions github-actions bot added exchange config Pull request or issue for exchange configurations parser zone config Pull request or issue for zone configurations labels May 9, 2023
@VIKTORVAV99 VIKTORVAV99 self-requested a review May 9, 2023 22:35
Copy link
Member

@VIKTORVAV99 VIKTORVAV99 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR!

Overall this looks good and is working.
There is just a few small things that needs to be addressed. I think I got most of them but the type related changes might require further adjustment down the line.

parsers/BD.py Outdated Show resolved Hide resolved
parsers/BD.py Outdated Show resolved Hide resolved
parsers/BD.py Outdated Show resolved Hide resolved
parsers/BD.py Show resolved Hide resolved
parsers/BD.py Show resolved Hide resolved
parsers/BD.py Outdated Show resolved Hide resolved
parsers/BD.py Outdated Show resolved Hide resolved
@VIKTORVAV99 VIKTORVAV99 self-requested a review May 12, 2023 14:15
Copy link
Member

@VIKTORVAV99 VIKTORVAV99 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! 🎉

Thanks for creating the new parser! It's very much appreciated!
I pushed some final changes required by our build and CI pipeline (and a small fix for a type error). I hope that's okay.

@mathilde-daugy I know you are still working on estimating the India zones and perhaps the exchange data from this parser might be of use. There are historical data available since 2015-04-19.

@VIKTORVAV99 VIKTORVAV99 merged commit ec6920e into electricitymaps:master May 12, 2023
16 checks passed
parsers/BD.py Show resolved Hide resolved
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
exchange config Pull request or issue for exchange configurations parser zone config Pull request or issue for zone configurations
Projects
None yet
Development

Successfully merging this pull request may close these issues.

More up-to-date data source for Bangladesh
3 participants