-
Notifications
You must be signed in to change notification settings - Fork 10
feature: polaris fsspec write implementation #73
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
cwognum
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This look great already! Thank you for taking the time to document everything! 🙏
For the failing code checks, it seems there is a mismatch between the line length we expect and the line length used. This should be automatic, as it is configured in the pyproject.toml. After navigating to the root directory, you should be able to just run:
ruff format .
It is a little weird though, because the error message states:
Would reformat: polaris/hub/client.py
Would reformat: polaris/utils/types.py
2 files would be reformatted, 43 files already formatted
Error: Process completed with exit code 1.
But types.py is fine for me locally. Let's try again and I'll look into it some more if it's not fixed.
cwognum
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @lmtroper ! Almost there!
cwognum
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Very polished! 💅
One minor comment still, but this is great work! Thank you @lmtroper !
|
Feel free to merge this PR whenever you're ready! 🙏 |
Changelogs
Integrated a basic write implementation for Zarr files with the Polaris FSSpec. Some of the major changes include:
rmandpipe_fileMore details of the FSSpec Implementation:
rmmethod is currently added as placeholder since it's required for the Zarr writing integration.pipe_filemethod calls the hub to get a signed URL for a PUT request, then writes the data to the bucket. The S3 bucket uses SHA-256 for data security but it is currently set to None.Checklist:
feature,fixortest(or ask a maintainer to do it for you).