Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor(parser structure): Refactor IS parser #6262

Merged
merged 7 commits into from
Dec 19, 2023
Merged

Conversation

unitrium
Copy link
Contributor

Description

Refactoring the IS parser to use the new datastructure.

Preview

Testing is currently pending because the website seems to be down so I can't get a test sample.

Double check

  • I have tested my parser changes locally with poetry run test_parser "zone_key"
  • I have run pnpx prettier --write . and poetry run format to format my changes.

@github-actions github-actions bot added parser python Pull requests that update Python code labels Dec 19, 2023
@unitrium unitrium removed the request for review from VIKTORVAV99 December 19, 2023 15:29
Copy link
Contributor

PR Analysis

  • 🎯 Main theme: Refactoring the IS parser to use a new data structure.
  • 📝 PR summary: The PR involves refactoring the IS parser to use a new data structure. The changes include removing the old IS.py file and adding a new file amper_landsnet.py with the updated parser. The new parser now uses the ProductionMix and ProductionBreakdownList models from the electricitymap.contrib.lib.models module.
  • 📌 Type of PR: Refactoring
  • 🧪 Relevant tests added: No
  • ⏱️ Estimated effort to review [1-5]: 2, because the PR is relatively small and straightforward, involving refactoring of a single parser. The changes are clear and well-structured.
  • 🔒 Security concerns: No security concerns found

PR Feedback

  • 💡 General suggestions: The PR is well-structured and the code is clean. However, it would be beneficial to include tests for the new parser to ensure its correct functionality. Also, it would be helpful to provide more context in the PR description about the reasons for the refactoring and the benefits of the new data structure.

  • 🤖 Code feedback:
    relevant fileparsers/amper_landsnet.py
    suggestion      Consider handling the case when the 'geothermal', 'hydro', or 'oil' keys are not present in the response JSON. This will make your code more robust to potential changes in the API response. [important]
    relevant linemix = ProductionMix(

How to use

Instructions

To invoke the PR-Agent, add a comment using one of the following commands:
/review: Request a review of your Pull Request.
/describe: Update the PR title and description based on the contents of the PR.
/improve [--extended]: Suggest code improvements. Extended mode provides a higher quality feedback.
/ask <QUESTION>: Ask a question about the PR.
/update_changelog: Update the changelog based on the PR's contents.
/add_docs: Generate docstring for new components introduced in the PR.
/generate_labels: Generate labels for the PR based on the PR's contents.
see the tools guide for more details.

To edit any configuration parameter from the configuration.toml, add --config_path=new_value.
For example: /review --pr_reviewer.extra_instructions="focus on the file: ..."
To list the possible configuration parameters, add a /config comment.

Comment on lines +16 to +23
def fetch_production(
zone_key: ZoneKey = ZoneKey("IS"),
session: Session | None = None,
target_datetime: datetime | None = None,
logger: Logger = getLogger(__name__),
) -> list[dict]:
"""Requests the last known production mix (in MW) of a given country."""
r = session or Session()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion: It's better to separate the creation of the Session object from the fetch_production function. This way, the same Session can be reused across multiple requests, which can improve performance by reusing the underlying TCP connection.

Suggested change
def fetch_production(
zone_key: ZoneKey = ZoneKey("IS"),
session: Session | None = None,
target_datetime: datetime | None = None,
logger: Logger = getLogger(__name__),
) -> list[dict]:
"""Requests the last known production mix (in MW) of a given country."""
r = session or Session()
def fetch_production(
zone_key: ZoneKey = ZoneKey("IS"),
session: Session,
target_datetime: datetime | None = None,
logger: Logger = getLogger(__name__),
) -> list[dict]:
"""Requests the last known production mix (in MW) of a given country."""

Comment on lines +34 to +39
obj = res.json()
mix = ProductionMix(
geothermal=obj["geothermal"],
hydro=obj["hydro"],
oil=obj["oil"],
)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion: It's a good practice to check if the response from the server is in the expected format before accessing its content. This can prevent unexpected errors.

Suggested change
obj = res.json()
mix = ProductionMix(
geothermal=obj["geothermal"],
hydro=obj["hydro"],
oil=obj["oil"],
)
obj = res.json()
if not all(key in obj for key in ["geothermal", "hydro", "oil"]):
raise ParserException(
parser="amper_landsnet.py",
message=f"Response from {SOURCE} is missing expected keys",
zone_key=zone_key,
)
mix = ProductionMix(
geothermal=obj["geothermal"],
hydro=obj["hydro"],
oil=obj["oil"],
)

parsers/amper_landsnet.py Show resolved Hide resolved
Copy link
Member

@VIKTORVAV99 VIKTORVAV99 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the endpoint is configured to only accept our production server IP and IPs in Iceland. So might be hard to get a a sample.

The changes look good though!

@unitrium
Copy link
Contributor Author

Ah ok that might explain some stuff! I will try to get a sample through a proxy

@github-actions github-actions bot added tests zone config Pull request or issue for zone configurations labels Dec 19, 2023
@unitrium unitrium merged commit ca6cefd into master Dec 19, 2023
19 checks passed
@unitrium unitrium deleted the robin/refactor-island branch December 19, 2023 17:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
parser python Pull requests that update Python code tests zone config Pull request or issue for zone configurations
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants