Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add __getstate__ and __setstate__ methods to allow pickling. Addresses #19 #20

Merged
merged 2 commits into from
Nov 11, 2021

Conversation

VolkerH
Copy link
Contributor

@VolkerH VolkerH commented Nov 10, 2021

As discussed in #19, the dask arrays currently returned by ND2file.to_dask() cannot be pickled, which limits the use of these returned dask arrays.

The reason why the pickling fails is due to the cythonized _rdr. This PR adds a __getstate__ method that removes _rdr from the items to be pickled and a __setstate__ method that re-creates a _rdr from the path.

I can't build the python package as I don't have the C code for the Nikon SDK. So far I tested this by monkey-patching a pip-installed nd2 package with these two methods.

I am not sure whether the way I hanlde the .closed attribute is correct.

@tlambert03
Copy link
Owner

thank you @VolkerH!

I can't build the python package as I don't have the C code for the Nikon SDK. So far I tested this by monkey-patching a pip-installed nd2 package with these two methods.

sorry about this. I think I'm going to just include the SDK here. The folks at laboratory imaging have given permission to distribute, and it would simplify a lot of things...

test fails are build-related (this is the first PR that's come outside of this repo)... so I'll deal with those too.

@tlambert03
Copy link
Owner

@VolkerH could you merge main here? I tried to do it for you but keep running into issues

@VolkerH
Copy link
Contributor Author

VolkerH commented Nov 11, 2021

Hi @tlambert03 , done !

@VolkerH
Copy link
Contributor Author

VolkerH commented Nov 11, 2021

I can now build locally.
Might add a short test, consisting of the following steps:

  • create dask array from nd2,
  • pickle/unpickle dask array,
  • run .compute() on a slice of unpickled dask array
  • assert correct values.

Is there somewhere I can fetch the nd2 files you are using for tests?

@tlambert03
Copy link
Owner

thanks @VolkerH! Will merge now and add a test later. I need to revamp the testing suite to make at least a subset of it easier for contributors to use. but let me do that in another PR

@tlambert03 tlambert03 merged commit 4eda9a0 into tlambert03:main Nov 11, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants