cutils fails when cutils_ext already present elsewhere in sys.path

[aaronstaley]

EDIT: Now it work with the c|py linker, but not with the cvm linker.

Theano's cutils moudle expects to find cutils_ext inside config.compiledir.

In cutils.py, cutils_ext.cutils_ext will be loaded. On an import error, this extension will be built and eventually cmodule.dlimport will re-import cutils_ext.cutils_ext. dlimport will inject config.compiledir into the front of sys,path, so the cutils_ext should always be loaded from config.compiledir.

Normally, this will then allow cutils_ext.cutils_ext to be loaded. The problem is if the cutils_ext package was found elsewhere on sys.path, but not the cutils_ext module (extension) within that package. cutils.py lacks the sys.path injection of dlimport. The consequence is that the import of cutils_ext.cutils_ext in dlimport fails because the reference to the outer cutils_ext (the package) will be the incorrect path!

Needless to say, when a user encounters this bug, it is a pain to solve.

The solution is to ensure that the cutils.py try statement starts with:

sys.path.insert(0,theano.config.compiledir)
with the finally doing:
del sys.path[0]

[jaberg]

A pain to solve indeed! Thanks for taking the time to track it down,
and report it. It's a long weekend in Canada, but hopefully Fred or
Pascal can patch this into the trunk next week.

• James

On Fri, Jun 29, 2012 at 10:11 PM, usaar33
reply@reply.github.com
wrote:

Theano's cutils moudle expects to find cutils_ext inside config.compiledir.

In cutils.py, cutils_ext.cutils_ext will be loaded.  On an import error, this extension will be built and eventually cmodule.dlimport will re-import cutils_ext.cutils_ext.  dlimport will inject config.compiledir into the front of sys,path, so the cutils_ext should always be loaded from config.compiledir.

Normally, this will then allow cutils_ext.cutils_ext to be loaded.  The problem is if the cutils_ext package was found elsewhere on sys.path, but not the cutils_ext module (extension) within that package.  cutils.py lacks the sys.path injection of dlimport.  The consequence is that the import of cutils_ext.cutils_ext in dlimport fails because the reference to the outer cutils_ext (the package) will be the incorrect path!

Needless to say, when a user encounters this bug, it is a pain to solve.

The solution is to ensure that the cutils.py try statement starts with:

sys.path.insert(0,theano.config.compiledir)
with the finally doing:
del sys.path[0]

---

Reply to this email directly or view it on GitHub:
#728

[nouiz]

Thanks for the report. In the trunk version of cutils, we append the path to sys.path if it is not present. I'll change that to an insert(0, path) as you suggest and delete it when we finish with it.

Also, what other project used the same name for as this module?

@jaberg, can you review the PR for this: gh-739?

thanks

[aaronstaley]

The bug wasn't raised during a project; it had to do with how PiCloud (http://www.picloud.com) was handling things. PiCloud automatically transports python modules it finds to the server. It didn't transport the .so of course, but a package (in the sense of having an init.py) called cutils_ext was created in its "automatic packages" directory. When a user then would use theano on PiCloud, the incorrect package would load.

[nouiz]

I don't understand everything you said, is the cutils_ext was created by PiCloud itself?

Anyway, can you look at gh-739 and tell us if you that this fix will solve the problem?

thanks

[aaronstaley]

The patch appears correct.

I didn't provide enough context in my explaination. Here is how PiCloud transports modules: http://docs.picloud.com/tech_overview.html#language-integration

So yes, a cutils_ext package (without the cutils_ext.so extension module) is created by PiCloud (as part of the automatic module transport from the client machine to its servers).

[nouiz]

thank for the information.

[jlowin]

I was experimenting with picloud this afternoon; I get the error described in this issue (importerror on cutils_ext)

This is my test code, if anyone cares to reproduce:

import theano
import theano.tensor as tt
import cloud
x = tt.scalar()
f = lambda y : theano.function([x], x)(y)
jid = cloud.call(f, 1)
cloud.result(jid)

[nouiz]

Witch version of Theano do you use? 0.6rc1?

[jlowin]

Yes, 0.6rc1.

[nouiz]

Sadly, I don't have the time to look at this shortly. Do you want to try to identify the problem? Can you test with the version when we closed this ticket, it was supposed to work at that time. If it work, can you bisect to find the commit that broke this?

thanks

[jlowin]

As far as I can tell, this actually didn't work at the time (may not have been fully tested). However, I worked with PiCloud to build a wrapper function that properly inserted cutils_ext. The wrapper function worked -- meaning the code runs and returns the correct result -- at the time this ticket was closed, but no longer works (something about a global 'CVM' not being defined that must be related to the CVM linker).

This isn't a priority for me right now, but when I have some time I will explore it and see if I can find why it doesn't work.

[nouiz]

If it is about the cvm, you can try to disable it with this theano flag:

linker=c|py

The CVM ask the compilation of a new file in: COMPILEDIR/lazylinker_ext/. I tell this just in case it help to understand the problem.

[jlowin]

Thanks @nouiz, that was helpful. I was unable to get the CVM linker to work, but I did get c|py to work (see PR). If I have more time I will try to readdress the CVM. In the meantime, anyone using c|py should be able to run distributed code on picloud now.

[jlowin]

Please note: anyone wishing to use PiCloud and Theano should install Theano in a PiCloud environment rather than use PiCloud's automatic dependency transfer (this is required in order to use the CVM linker). Theano is a dynamic library, and the local module will not always work once transmitted to the remote server.

[lamblin]

Would it make sense to add this warning and instructions in the installation documentation? Maybe adding a "PiCloud" subsection in "Linux", after "Using the GPU".

[nouiz]

As this is from 2012 and we don't have news of user of PiCloud, I will close this issue. If someone still have problems with PiCloud, then open a new issue.