Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Switch NWM Crosswalk Source from CSV to URL #156

Merged
merged 7 commits into from Nov 9, 2021
Merged

Switch NWM Crosswalk Source from CSV to URL #156

merged 7 commits into from Nov 9, 2021

Conversation

jarq6c
Copy link
Collaborator

@jarq6c jarq6c commented Nov 9, 2021 •
edited

This PR changes the source of default National Water Model crosswalk information from packaged CSVs to an HDF5 file hosted on HydroShare.

https://www.hydroshare.org/resource/d154f19f762c4ee9b74be55f504325d3/

The crosswalk is no longer built on import. I implemented the crosswalk as a property of NWMClientDefaults. The interaction of this property with the NWMClient objects means the file is only downloaded and processed if a NWMClient is initialized without a user-specified location_metadata_mapping. The file is then downloaded via the FileDownloader and cached in the default ParquetCache like any other dataframe associated with the client.

Additions

  • NWMClientDefaults.ROUTELINK_URL specifies the location of the NWM RouteLink HDF5 file.

Removals

  • data directory and all RouteLink CSVs have been removed.

Changes

  • NWMClientDefaults.CROSSWALK is now a property that downloads and caches remote RouteLink data when called.

Testing

  1. Passes all existing tests.
  2. No new tests. Enhancement uses existing components.

Notes

Todos

  • None

Checklist

  • PR has an informative and human-readable title
  • PR is well outlined and documented. See #12 for an example
  • Changes are limited to a single goal (no scope creep)
  • Code can be automatically merged (no conflicts)
  • Code follows project standards (see CONTRIBUTING.md)
  • Passes all existing automated tests
  • Any change in functionality is tested
  • New functions are documented (with a description, list of inputs, and expected output) using numpy docstring formatting
  • Placeholder code is flagged / future todos are captured in comments
  • Reviewers requested with the Reviewers tool ➡️

@jarq6c jarq6c added the enhancement New feature or request label Nov 9, 2021
@jarq6c jarq6c self-assigned this Nov 9, 2021
aaraney
aaraney reviewed Nov 9, 2021
View reviewed changes
Copy link
Member

@aaraney aaraney left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In general this looks good! I have a few comments worth (i think) worth discussing. So, im looking forward to hearing back from you @jarq6c! Cheers.

python/nwm_client_new/src/hydrotools/nwm_client_new/NWMClient.py Outdated Show resolved Hide resolved
python/nwm_client_new/src/hydrotools/nwm_client_new/NWMClient.py Outdated Show resolved Hide resolved
@jarq6c
Copy link
Collaborator Author

jarq6c commented Nov 9, 2021

Resolved all comments. Passes all tests. Thanks for the review @aaraney !

@jarq6c jarq6c merged commit b31e522 into NOAA-OWP:main Nov 9, 2021
@jarq6c jarq6c deleted the nwm-xwalk branch November 15, 2021 15:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@aaraney aaraney aaraney left review comments

Assignees

@jarq6c jarq6c

Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Change crosswalk source to HydroShare
2 participants
@jarq6c @aaraney