New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
pickler: Cython 3.0 compatibility #477
Conversation
Newer versions of Cython create functions that provide a __reduce__() implementation that does not play nicely with the strategies used by jsonpickle. Detect Cython functions and record references to them directly so that they are properly pickled. This fixes the ability to encode and decode pandas objects. We avoid recording "py/ref" references to Cython functions because they throw off the ref offsets and are not needed. "py/function" references, when unpickled, resolve to module globals that exist outside of the object graph. Fixes: jsonpickle#460
Remove the Cython "py/ref" avoidance by recording ref entries when unpickling "py/function" entities. Update the unpickler to do record ref entries for "py/function" so that we can remove the hacky self._objs handling for Cython functions in the pickler. Related-to: jsonpickle#460
Cython v3 produces functions that pass the util.is_object() tests. This complicated the unpickler because it needed to start tracking refs for functions, which it never needed to do before. Expand the meaning of is_module_function() so that it includes cython functions and use this check earlier in the pickling process. This makes the pickler handle Cython functions before they ever make it into the "if util.is_object(obj):" code path. These changes restore the pickler's original semantics by ensuring that functions are excluded from the referencing machinery completely. Related-to: jsonpickle#460
I split these changes up into several commits mostly to record the different approaches I encountered along the way. The first ones are kinda hacky but the later ones make things simple again. Let me know if you'd prefer that I squash them all together. |
a88ee0e
to
cfcc106
Compare
I believe this patch worked: https://copr.fedorainfracloud.org/coprs/churchyard/pandas2/build/7030817/ |
https://pypi.org/project/jsonpickle/3.0.3/ contains this fix. |
No description provided.