Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add interface to nwm_client_new for Azure Blob Storage #236

Merged
merged 5 commits into from
Sep 19, 2023

Conversation

jarq6c
Copy link
Collaborator

@jarq6c jarq6c commented Sep 19, 2023

Adds a concrete interface for accessing NWM files on Azure Blob Storage.

Additions

AzureFileCatalog class for discovering files on Azure Blob Storage. Inherits from NWMFileCatalog.

Removals

  • None

Changes

  • Added example to README.md.

Testing

  1. Added basic tests commensurate with other concrete implementations of NWMFileCatalog.

Notes

  • None

Todos

  • None

Checklist

  • PR has an informative and human-readable title
  • PR is well outlined and documented. See #12 for an example
  • Changes are limited to a single goal (no scope creep)
  • Code can be automatically merged (no conflicts)
  • Code follows project standards (see CONTRIBUTING.md)
  • Passes all existing automated tests
  • Any change in functionality is tested
  • New functions are documented (with a description, list of inputs, and expected output) using numpy docstring formatting
  • Placeholder code is flagged / future todos are captured in comments
  • Reviewers requested with the Reviewers tool ➡️

@jarq6c jarq6c added the enhancement New feature or request label Sep 19, 2023
@jarq6c jarq6c self-assigned this Sep 19, 2023
@jarq6c
Copy link
Collaborator Author

jarq6c commented Sep 19, 2023

Closes #235

Copy link

@christophertubbs christophertubbs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My only 'complaint' is the name of the package nwm_client_new, but that's out of scope for this. Everything is well documented and the code is easy to read. If I were to be ruthless, I'd say that line 80 in AzureFileCatalog.py might read better as:

    return [
        os.path.join(self.server, url)
        for url in list(blobs)
        if must_contain in url
    ]

instead of

    return [self.server+b for b in list(blobs) if must_contain in b]

That's only if I try my hardest to be picky.

A+

@jarq6c
Copy link
Collaborator Author

jarq6c commented Sep 19, 2023

Thanks @christophertubbs !

@jarq6c
Copy link
Collaborator Author

jarq6c commented Sep 19, 2023

Passes all tests. Heeded advice for more explicit URL handling, but used our own handling from _restclient.

@jarq6c jarq6c merged commit ef6d0b6 into NOAA-OWP:main Sep 19, 2023
3 checks passed
@jarq6c
Copy link
Collaborator Author

jarq6c commented Sep 20, 2023

@jarq6c jarq6c deleted the add-azure-client branch September 20, 2023 17:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants