AttributeError: 'module' object has no attribute '__main__'

[jakirkham]

Seeing this error on the new release (dill 0.2.4), which I hadn't seen on the last release. Using Jupyter/iPython cluster. Related questions ( http://stackoverflow.com/questions/27597778/why-do-i-get-a-module-has-no-attribute-main-in-ipython-parallel-gather-state )

AttributeError: 'module' object has no attribute '__main__'

[mmckerns]

Can you post a reduced sample of your code, and the traceback, as well as the versions of python and modules you are using? Neither this ticket nor the link to stackoverflow gives enough information to reproduce an error… so I can't see what you are seeing, and thus can't ascertain if or what needs to be addressed.

[jakirkham]

This unfortunately is yet again another bug that occurs with very low frequency in a code base that interacts with the iPython cluster and dill at a high level of abstraction so it is difficult for me to give you a reproducer at present. I will see what I can do.

[mmckerns]

dill is quite deterministic, so this makes me think it's more of a bug resulting from something you are doing in IPython.parallel. That's the best I can do, unless you can give me version numbers and code that causes the error (even if it's infrequently).

[jakirkham]

Sorry this is so gross. This is the version info from conda.

anaconda-client 1.1.0
bokeh 0.10.0
boost 1.57.0
bottleneck 1.0.0
cairo 1.12.18
clyent 0.4.0
conda 3.18.3
conda-build 1.18.1
conda-env 2.4.4
decorator 4.0.4
dill 0.2.4
drmaa 0.7.6
fftw 3.3.4
fireworks 1.1.6
flask 0.10.1
flask-paginate 0.3.2
fontconfig 2.11.1
freetype 2.5.5
funcsigs 0.4
futures 3.0.3
h5py 2.5.0
hdf5 1.8.15.1
icu 54.1
ipykernel 4.1.1
ipyparallel 4.0.2
ipython 4.0.0
ipython-notebook 4.0.4
ipython_genutils 0.1.0
ipywidgets 4.1.0
itsdangerous 0.24
jinja2 2.8
jpeg 8d
jsonschema 2.4.0
jupyter_client 4.1.1
jupyter_core 4.0.6
libgcc 4.8.5
libpng 1.6.17
libsodium 1.0.3
libtiff 4.0.2
libxml2 2.9.2
markupsafe 0.23
matplotlib 1.4.3
mistune 0.7.1
mock 1.3.0
mongodb 2.4.6
monty 0.6.5
nanshe 0.1.0a29
nbconvert 4.0.0
nbformat 4.0.1
networkx 1.10
nose 1.3.7
notebook 4.0.6
numpy 1.9.3
openblas 0.2.14
openssl 1.0.2d
pandas 0.16.2
patchelf 0.6
path.py 8.1.2
pbr 1.3.0
pexpect 3.3
pickleshare 0.5
pillow 3.0.0
pip 7.1.2
pixman 0.26.2
psutil 3.2.2
ptyprocess 0.5
py2cairo 1.10.0
pycosat 0.6.1
pycrypto 2.6.1
pyfftw 0.9.2
pygments 2.0.2
pylibtiff 0.4.0
pymongo 3.0.3
pyparsing 2.0.3
pyqt 4.11.4
python 2.7.10
python-dateutil 2.4.2
pytz 2015.6
pyyaml 3.11
pyzmq 14.7.0
qimage2ndarray 1.4
qt 4.8.7
rank_filter 0.2.0
readline 6.2
requests 2.8.1
runipy 0.1.5
scikit-image 0.11.3
scipy 0.16.0
setuptools 18.4
simplegeneric 0.8.1
sip 4.16.9
six 1.10.0
spams v2.5
sqlite 3.8.4.1
ssl_match_hostname 3.4.0.2
tabulate 0.7.5
terminado 0.5
tk 8.5.18
tornado 4.2.1
traitlets 4.0.0
vigra ge4946fe
volumina v1.1.6
webcolors 1.5
werkzeug 0.10.4
wheel 0.26.0
yaml 0.1.6
zeromq 4.1.3
zlib 1.2.8

[mmckerns]

Ok, fine, I can deal with that. So how about some code?

[jakirkham]

From watching this bug arise, I think I am seeing this in relation to code like that seen in this issue ( #135 ). In particular, if I use the workaround proposed there, I run into an issue with code looking like this. However, this doesn't quite reproduce it.

from ipyparallel import Client

client = Client()
client[:].use_dill().get()

with client[:].sync_imports():
    import nest
    from nest import A, B

[mmckerns]

Can you also post the Traceback you are seeing?

[tcwalther]

I can confirm the error. I could work around it by importing pickle before calling use_dill().

import ipyparallel
engines = ipyparallel.Client()
balancer = engines.load_balanced_view()

%%px --local
import pickle

ipyparallel.canning.use_dill()
engines[:].apply_sync(ipyparallel.canning.use_dill);

[mmckerns]

@tcwalther: Thanks. I'll see if I can reproduce the error. Any code that reliably produces the error would be very helpful. If your workaround does indeed work, it might be require a patch for ipyparallel instead of dill… but that remains to be seen.

@jakirkham or @tcwalther, do you know if this is an existing issue/ticket for ipyparallel?

[jakirkham]

I don't believe there is an existing issue on ipyparallel.

[mmckerns]

I've punted this issue to ipyparallel, at least for the time being. Let's leave this ticket open, and play a little wait-and-see.

[max-sixty]

We just had the same issue.

We're using celery rather than ipyparallel. If we pickle a function in the notebook, when it's unpickled by the celery worker (on another box), this is the stack trace:

[2016-03-03 23:43:23,055: ERROR/Worker-99] Pool process <Worker(Worker-99, started daemon)> error: AttributeError("'module' object 
has no attribute '__main__'",)                                                                                                     
Traceback (most recent call last):                                                                                                 
  File "/usr/local/lib/python2.7/dist-packages/billiard/pool.py", line 295, in run                                                 
    sys.exit(self.workloop(pid=pid))                                                                                               
  File "/usr/local/lib/python2.7/dist-packages/billiard/pool.py", line 356, in workloop                                            
    req = wait_for_job()                                                                                                           
  File "/usr/local/lib/python2.7/dist-packages/billiard/pool.py", line 447, in receive                                             
    ready, req = _receive(1.0)                                                                                                     
  File "/usr/local/lib/python2.7/dist-packages/billiard/pool.py", line 419, in _recv                                               
    return True, loads(get_payload())                                                                                              
  File "/usr/local/lib/python2.7/dist-packages/billiard/common.py", line 77, in pickle_loads                                       
    return load(BytesIO(s))                                                                                                        
AttributeError: 'module' object has no attribute '__main__'                                                                        
[2016-03-03 23:43:23,059: ERROR/MainProcess] Process 'Worker-99' pid:10938 exited with 'exitcode 1'    

If we use dill.settings['recurse']=True, then it works fine.

Is there any other info that would be helpful?

[mmckerns]

@MaximilianR: Yes, this is helpful. Does it always occur? or is it infrequent, as when seen in ipyparallel?

I haven't been able to reproduce the error in ipyparallel (I don't have a minimal example), so if you have a full minimal example that I can test and reproduce the error, it would help immensely.

[mmckerns]

@jakirkham: I just noticed you said "seeing this error on the new release, which I hadn't seen on the last release"… you are referring to dill releases, correct? Can you be more specific in the version numbers (i.e. 0.2.6.dev0 versus 0.2.5)? That would help me narrow down the changes made to dill, and might provide some further strong insight to what's going on.

[jakirkham]

It was in the version listing above. Though admittedly that is not easy on the eyes. I have added it to the issue summary. The version was 0.2.4. Though other users here may have tried a more recent version.

[mmckerns]

@jakirkham: aha… I didn't see that. So, in version 0.2.4, but not in 0.2.3, correct? Hmm...

[jakirkham]

No worries. I think it was true of the previous version (0.2.3) so I went to update at the time to see if that might fix the problem. I didn't try any older versions though and I haven't tried 0.2.5. Though I wasn't aware of this dill.settings trick. Might be able to do this on top of ipyparallel with a bit of a hack.

[max-sixty]

This isn't conclusive, in trying to make a reproducible example I found this: I think it may be when pickle is used to unpickle the function.

(NB to those that don't know: you shouldn't use unserialize the strings I post here, you should generate your own, as it's a security risk for you)

import dill
In [2]:

def n(a):
    return a*2
In [3]:

dill.dumps(n)
Out[3]:
'\x80\x02cdill.dill\n_create_function\nq\x00(cdill.dill\n_unmarshal\nq\x01Uwc\x01\x00\x00\x00\x01\x00\x00\x00\x02\x00\x00\x00C\x00\x00\x00s\x08\x00\x00\x00|\x00\x00d\x01\x00\x14S(\x02\x00\x00\x00Ni\x02\x00\x00\x00(\x00\x00\x00\x00(\x01\x00\x00\x00t\x01\x00\x00\x00a(\x00\x00\x00\x00(\x00\x00\x00\x00s\x1e\x00\x00\x00<ipython-input-2-ac332b335aa3>t\x01\x00\x00\x00n\x01\x00\x00\x00s\x02\x00\x00\x00\x00\x01q\x02\x85q\x03Rq\x04c__builtin__\n__main__\nU\x01nq\x05NN}q\x06tq\x07Rq\x08.'

This works fine:

#restart process
In [1]:

import dill
In [2]:

dill.loads('\x80\x02cdill.dill\n_create_function\nq\x00(cdill.dill\n_unmarshal\nq\x01Uwc\x01\x00\x00\x00\x01\x00\x00\x00\x02\x00\x00\x00C\x00\x00\x00s\x08\x00\x00\x00|\x00\x00d\x01\x00\x14S(\x02\x00\x00\x00Ni\x02\x00\x00\x00(\x00\x00\x00\x00(\x01\x00\x00\x00t\x01\x00\x00\x00a(\x00\x00\x00\x00(\x00\x00\x00\x00s\x1e\x00\x00\x00<ipython-input-2-ac332b335aa3>t\x01\x00\x00\x00n\x01\x00\x00\x00s\x02\x00\x00\x00\x00\x01q\x02\x85q\x03Rq\x04c__builtin__\n__main__\nU\x01nq\x05NN}q\x06tq\x07Rq\x08.')
Out[2]:
<function __main__.n>

But this fails, as expected:

>>> import cPickle
>>> cPickle.loads('\x80\x02cdill.dill\n_create_function\nq\x00(cdill.dill\n_unmarshal\nq\x01Uwc\x01\x00\x00\x00\x01\x00\x00\x00\x02\x00\x00\x00C\x00\x00\x00s\x08\x00\x00\x00|\x00\x00d\x01\x00\x14S(\x02\x00\x00\x00Ni\x02\x00\x00\x00(\x00\x00\x00\x00(\x01\x00\x00\x00t\x01\x00\x00\x00a(\x00\x00\x00\x00(\x00\x00\x00\x00s\x1e\x00\x00\x00<ipython-input-2-ac332b335aa3>t\x01\x00\x00\x00n\x01\x00\x00\x00s\x02\x00\x00\x00\x00\x01q\x02\x85q\x03Rq\x04c__builtin__\n__main__\nU\x01nq\x05NN}q\x06tq\x07Rq\x08.')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'module' object has no attribute '__main__'

[mmckerns]

@jakirkham: So when you say, hadn't seen on the last release, did you mean the previous release of ipyparallel? Just trying to figure out some point where the critical change occurred…

[mmckerns]

@MaximilianR: Interesting… hmm. dill does indeed have a custom unpickler. it's very thin, but it't not the stock unpickler.

[mmckerns]

@MaximilianR: your example even fails if you use pickle instead of cPickle… which is significant because dill extends almost all of itself (except the Unpickler!) into pickle upon import.

[mmckerns]

def n(a):
    return a*2

import dill
# dill.settings['recurse'] = True
p = dill.dumps(n)
dill.loads(p)

import pickle
pickle.loads(p)

[max-sixty]

@MaximilianR: your example even fails if you use pickle instead of cPickle… which is significant because dill extends almost all of itself (except the Unpickler!) into pickle upon import.

Yup!

[mmckerns]

I'm trying to figure out how to best resolve this ticket…

I am still of the mind that this is not directly an issue with dill, but more so an issue that whatever package that is using dill is (inadvertently) calling dill.dump and pickle.load.

One possible resolution is that I: (1) close this ticket as it doesn't seem to be a dill problem… and (2) open a new ticket to explore the potential to inject the dill Unpickler behavior into the one used by pickle -- or to remove custom behavior from the Unpickler in dill. Note also that this behavior could however, potentially be resolved in dill, as the example above shows. The test does not fail when dill.settings['recurse'] = True. So an alternative to (2) above is to figure out what exactly is the issue when 'recurse' = False, and see if there's a workaround.

So… my inclination is to close this ticket saying "don't use pickle to unpickle something from dill, but if you do, then make sure you have set 'recurse' to True -- and I will also open a new ticket with regard to modifying/injecting the Unpickler behavior, or the behavior from 'recurse' = True.

Does that sound reasonable? Any other suggestions?

[diego-mazon]

As a pedestrian with the same issue since one week ago (it wasn't happening before), should I change the default value of 'recurse' to True in the python2.7/site-packages/dill/settings.py file ??

Thanks!

[mmckerns]

@diego-mazon: To me it smells like dill-centric bug in ipyparallel, but we are not sure just yet. So, with regard to your question, the answer is NO. You can set dill.settings['recurse'] = True. Don't edit the file. Import dill and set the entry in the settings dict to True.

[diego-mazon]

Thanks, @mmckerns . Just in case this helps: Now it's working fine for me with no need of setting 'recurse' to True. I can now load everything using either dill, pickle or cPickle. I wasn't able to do that yesterday until I changed that value in the settings dict. Thanks.