Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CL parser: add snapshot test and remove usage of arrow module #6517

Merged

Conversation

gianantoniopini
Copy link
Contributor

@gianantoniopini gianantoniopini commented Feb 28, 2024

Issue

#6135

Description

This PR adds a snapshot test for the CL parser and removes usage of the arrow module from the CL parser.

Preview

Double check

  • I have tested my parser changes locally with poetry run test_parser "zone_key"
  • I have run pnpx prettier@2 --write . and poetry run format in the top level directory to format my changes.

@github-actions github-actions bot added parser python Pull requests that update Python code tests labels Feb 28, 2024
Copy link
Contributor Author

@gianantoniopini gianantoniopini Feb 28, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have tried to minify this .json mock file: after minification the file size went from 58K to 37K. But I decided to revert that commit and keep the file unminified, primarily because of readability. The commit with the minified version is 20ba103. Happy to revert back to the minified version, if it is needed.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think either is fine, the point of the snapshot tests is not really to see if they match the expected input data (I sure hope they do though) but rather to ensure no accidental regressions are made in future edits.

So readability is not really a concern IMO.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While working on the parser code, I have found it helpful to be able to see what the response from the external api actually looks like. So, at least for me, the .json mock file also served this purpose, as it contains a sample response from the external api.

Copy link
Member

@VIKTORVAV99 VIKTORVAV99 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work! Looks like it works as expected!

I'm not a huge fan of strptime and strftime so I left suggestions that use the iso builtin versions instead.

parsers/CL.py Outdated Show resolved Hide resolved
parsers/CL.py Outdated Show resolved Hide resolved
gianantoniopini and others added 2 commits February 29, 2024 11:01
Co-authored-by: Viktor Andersson <30777521+VIKTORVAV99@users.noreply.github.com>
Co-authored-by: Viktor Andersson <30777521+VIKTORVAV99@users.noreply.github.com>
@gianantoniopini
Copy link
Contributor Author

Just applied your suggestions related to the use of datetime isoformat methods. Thank you!

Copy link
Member

@VIKTORVAV99 VIKTORVAV99 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Thanks for doing this and special thanks for adding tests!

@VIKTORVAV99 VIKTORVAV99 merged commit 3f3bab9 into electricitymaps:master Feb 29, 2024
20 checks passed
@gianantoniopini gianantoniopini deleted the gp/6135-remove-arrow-CL branch February 29, 2024 10:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
parser python Pull requests that update Python code tests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants