New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: add dask_to_root #1085
feat: add dask_to_root #1085
Conversation
Just a stylistic comment but I think the function would be better situated within uproot as I think it would match dask-awkward and awkward better in that regard. |
You mean the function would be an attribute of another function, I've gone back and forth on that—there used to be an I understand the logic of writing-from-Dask being associated with reading-from-Dask. Uproot doesn't already have the flat bundle of functions that Awkward Array has, so there's less to maintain. But still, locating functions as attributes on other functions is unusual enough in Python as to be an antipattern—if someone is looking for it, they're unlikely to think of looking there. Perhaps it should just be a naming convention: |
pyproject.toml
Outdated
"typing_extensions>=4.1.0; python_version < \"3.11\"" | ||
"typing_extensions>=4.1.0; python_version < \"3.11\"", | ||
"dask >=2023.04.0", | ||
"dask-awkward>=2023.12.1" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
dask shouldn't be a top level dependency of the uproot package? @jpivarski
I suppose below we should import dask within dask_write
such that it complains when dask is not available, but doesn't impede more standard functioning of uproot.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Correct: dask
and dask-awkward
should not be strict dependencies for Uproot. These two lines should be removed.
@jpivarski ok that naming scheme makes sense, and I am fine with it! |
In Uproot and Awkward, name the tests by PR number:
|
OK I checked it with nanoevents, repartitioning, concatenating, all using a distributed Client and it seems to be robust. It touches data correctly and fills the output TTree well. Looks good to me :-) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is good; just be sure to
- remove version.py from this PR, now that it's auto-generated
- remove
dask
anddask-awkward
from the list of dependencies
After that, request a review from me and this will be mergeable.
pyproject.toml
Outdated
"typing_extensions>=4.1.0; python_version < \"3.11\"" | ||
"typing_extensions>=4.1.0; python_version < \"3.11\"", | ||
"dask >=2023.04.0", | ||
"dask-awkward>=2023.12.1" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Correct: dask
and dask-awkward
should not be strict dependencies for Uproot. These two lines should be removed.
…/uproot5 into feat-add-dask-to-root
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also remove the 7 output files before merging:
src/uproot/my-output/*.root
But, after they're removed (as well as the if __name__ == "__main__"
part), this is ready to merge!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nevermind; I was looking at the wrong diffs. The temporary output files have already been removed.
Co-authored-by: Jim Pivarski <jpivarski@users.noreply.github.com>
Co-authored-by: Jim Pivarski <jpivarski@users.noreply.github.com>
No description provided.