python 3.7 -> python 3.8 incompatible? SystemError: unknown opcode

[ugogon]

Following error happens when storing a dict that points to a function that contains a for loop or a try catch block (maybe other opcodes are problematic as well)

Reproduce:

I run the following file with python 3.7(.6 ):

import dill

def foo():
    for i in range(10):
        print(i)

d = dict(foo=foo)
f = open('foo.pkl', 'wb')

dill.dump(d, f)
f.close()

I run the following file with python 3.8(.10 ):

import dill

f = open("foo.pkl","rb")
d = dill.load(f)
d["foo"]()

The last run fails with:

XXX lineno: 4, opcode: 120
Traceback (most recent call last):
  File "load.py", line 5, in <module>
    d["foo"]()
  File "dump.py", line 4, in foo
    d = dill.load(f)
SystemError: unknown opcode

Is that expected behavior? Is dill not compatible between python versions?

[matsjoyce]

The way functions are pickled is python-version-dependent, so the pickled data is not in general compatible with different versions of python (it sometimes works, sometimes doesn't, depending on bytecode changes between versions). @mmckerns Maybe that should be added to the README?

[mmckerns]

dill attempts to ensure a function can be pickled and unpicked across versions of python, but it is in no way guaranteed to work. Specifically, the function "container" itself should pickle across versions, but beyond that it is highly dependent on what's inside and referred to outside of the function. A simple function like def(x): return x*x should work, but many others will not. (Note also that DEFAULT_PROTOCOL changed from 3 to 4 when going from 3.7 to 3.8, but that generally impacts transferring objects from 3.8 to 3.7.). We could add a statement to that effect in the README...