DM-19616: Add patch to support pickle/unpickle for multiprocessing #5
Conversation
patches/pickling.patch
Outdated
|
|
||
| + def test_pickle(self): | ||
| + h = htm.HTM(5) | ||
| + pickled = pickle.dumps(h, protocol=pickle.HIGHEST_PROTOCOL) |
There was a problem hiding this comment.
Why HIGHEST_PROTOCOL? I'm guessing (without having looked at the meas_algorithms code) that this would usually be called without a protocol argument, so it's probably most useful to test DEFAULT_PROTOCOL instead (or even range(HIGHEST_PROTOCOL+1) or something).
There was a problem hiding this comment.
HIGHEST_PROTOCOL results in much smaller and faster pickling, and is purely a pickle-internals thing; the pickle infrastructure here doesn't know or care about it. Though, in this case it shouldn't matter, since the only thing being pickled is an int and a function pointer. meas_algorithms doesn't do any pickling itself, multiprocessing does, and it would almost certainly use the best protocol for that.
There was a problem hiding this comment.
Multiprocessing uses the default protocol: https://github.com/python/cpython/blob/master/Lib/multiprocessing/reduction.py#L49
Just seems weird to force the test to use something else without explanation.
There was a problem hiding this comment.
Huh. I'm quite surprised they use the default there: I would have expected HIGHEST, because it's purely in-memory. Ah well, at least DEFAULT is now 3, which is good enough. I've removed the explicit protocol.
No description provided.