Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cythonize failed in installation #6917

Closed
zcyang opened this issue Jan 1, 2016 · 14 comments

Comments

Projects
None yet
4 participants
@zcyang
Copy link

commented Jan 1, 2016

Hi,

I meet wit the following problem when I install from source:

Cythonizing sources
Processing numpy/random/mtrand/mtrand.pyx
Traceback (most recent call last):
  File "/share/apps/bin/cython", line 8, in <module>
    main(command_line = 1)
  File "/share/apps/lib/python2.7/site-packages/Cython/Compiler/Main.py", line 709, in main
    result = compile(sources, options)
  File "/share/apps/lib/python2.7/site-packages/Cython/Compiler/Main.py", line 684, in compile
    return compile_multiple(source, options)
  File "/share/apps/lib/python2.7/site-packages/Cython/Compiler/Main.py", line 657, in compile_multiple
    context = options.create_context()
  File "/share/apps/lib/python2.7/site-packages/Cython/Compiler/Main.py", line 582, in create_context
    self.cplus, self.language_level, options=self)
  File "/share/apps/lib/python2.7/site-packages/Cython/Compiler/Main.py", line 75, in __init__
    from . import Builtin, CythonScope
  File "/share/apps/lib/python2.7/site-packages/Cython/Compiler/CythonScope.py", line 5, in <module>
    from .UtilityCode import CythonUtilityCode
  File "/share/apps/lib/python2.7/site-packages/Cython/Compiler/UtilityCode.py", line 3, in <module>
    from .TreeFragment import parse_from_strings, StringParseContext
  File "/share/apps/lib/python2.7/site-packages/Cython/Compiler/TreeFragment.py", line 17, in <module>
    from .Visitor import VisitorTransform
  File "Cython/Compiler/Visitor.py", line 15, in init Cython.Compiler.Visitor (/share/apps/src/cython/Cython/Compiler/Visitor.c:19407)
  File "/share/apps/lib/python2.7/site-packages/Cython/Compiler/ExprNodes.py", line 4244, in <module>
    class SliceIndexNode(ExprNode):
  File "/share/apps/lib/python2.7/site-packages/Cython/Compiler/ExprNodes.py", line 4396, in SliceIndexNode
    "SliceObject", "ObjectHandling.c", context={'access': 'Get'})
  File "Cython/Compiler/Code.py", line 286, in Cython.Compiler.Code.UtilityCodeBase.load (/share/apps/src/cython/Cython/Compiler/Code.c:9181)
  File "Cython/Compiler/Code.py", line 525, in Cython.Compiler.Code.TempitaUtilityCode.__init__ (/share/apps/src/cython/Cython/Compiler/Code.c:15550)
  File "Cython/Compiler/Code.py", line 517, in Cython.Compiler.Code.sub_tempita (/share/apps/src/cython/Cython/Compiler/Code.c:15269)
  File "/share/apps/lib/python2.7/site-packages/Cython/Tempita/__init__.py", line 4, in <module>
    from ._tempita import *
  File "Cython/Tempita/_tempita.py", line 36, in init Cython.Tempita._tempita (/share/apps/src/cython/Cython/Tempita/_tempita.c:31807)
  File "/share/apps/lib/python2.7/cgi.py", line 50, in <module>
    import mimetools
  File "/share/apps/lib/python2.7/mimetools.py", line 6, in <module>
    import tempfile
  File "/share/apps/lib/python2.7/tempfile.py", line 35, in <module>
    from random import Random as _Random
  File "/share/apps/src/numpy-1.9.3/numpy/random/__init__.py", line 99, in <module>
    from .mtrand import *
ImportError: No module named mtrand
--fast-fail mtrand.c mtrand.pyx
Traceback (most recent call last):
  File "/share/apps/src/numpy-1.9.3/tools/cythonize.py", line 200, in <module>
    main()
  File "/share/apps/src/numpy-1.9.3/tools/cythonize.py", line 196, in main
    find_process_files(root_dir)
  File "/share/apps/src/numpy-1.9.3/tools/cythonize.py", line 188, in find_process_files
    process(cur_dir, fromfile, tofile, function, hash_db)
  File "/share/apps/src/numpy-1.9.3/tools/cythonize.py", line 162, in process
    processor_function(fromfile, tofile)
  File "/share/apps/src/numpy-1.9.3/tools/cythonize.py", line 73, in process_pyx
    raise Exception('Cython failed')
Exception: Cython failed
Traceback (most recent call last):
  File "setup.py", line 251, in <module>
    setup_package()
  File "setup.py", line 239, in setup_package
    generate_cython()
  File "setup.py", line 191, in generate_cython
    raise RuntimeError("Running cythonize failed!")
RuntimeError: Running cythonize failed!

I am using

Cython version 0.24.0a0

Many thanks!

@maniteja123

This comment has been minimized.

Copy link
Contributor

commented Jan 1, 2016

Hi, I remember facing this issue when installing scipy, do try installing cython using pip or easy_install using easy_install cython or pip install cython. Hope it helps.

@zcyang

This comment has been minimized.

Copy link
Author

commented Jan 1, 2016

I installed the cython from source too, so you mean it's the problem of source installation of cython?

@maniteja123

This comment has been minimized.

Copy link
Contributor

commented Jan 1, 2016

I really am a beginner with cython and am not aware of the internals. But I faced the issue when I installed it directly from source. Please do try installing using pip or easy_install.

@zcyang

This comment has been minimized.

Copy link
Author

commented Jan 1, 2016

seems no difference, I reinstall cython with pip, the same problem occurs

@maniteja123

This comment has been minimized.

Copy link
Contributor

commented Jan 1, 2016

Oh, I am sorry but this is the best of my knowledge. Will try installing in a venv and get back to you.

@maniteja123

This comment has been minimized.

Copy link
Contributor

commented Jan 1, 2016

I did try installing cython directly from the source with the dev version 0.24.0a0 in a virtualenv, but couldn't reproduce the error. I better leave this to the experts rather than wasting your time. Apologies.

@rgommers

This comment has been minimized.

Copy link
Member

commented Jan 1, 2016

From the traceback it looks like tempfile somehow manages to confuse numpy.random with stdlib random:

File "/share/apps/lib/python2.7/tempfile.py", line 35, in <module>
  from random import Random as _Random
File "/share/apps/src/numpy-1.9.3/numpy/random/__init__.py", line 99, in <module>
  from .mtrand import *

Can you try just from the command line:

$ cython numpy/random/mtrand/mtrand.pyx

If that fails as well, something may be broken in Cython. Next step then would be to check that the problem disappears if you go back to Cython 0.23.4

@zcyang

This comment has been minimized.

Copy link
Author

commented Jan 1, 2016

Many thanks!

$ cython numpy/random/mtrand/mtrand.pyx works, but how can I run python setup.py build and get correct result

@zcyang

This comment has been minimized.

Copy link
Author

commented Jan 1, 2016

when I run cython mtrand.pyx, I get the following error:

Traceback (most recent call last):
  File "/share/apps/bin/cython", line 5, in <module>
    from pkg_resources import load_entry_point
  File "/share/apps/lib/python2.7/site-packages/pkg_resources/__init__.py", line 37, in <module>
    import email.parser
  File "/share/apps/lib/python2.7/email/parser.py", line 12, in <module>
    from email.feedparser import FeedParser
  File "/share/apps/lib/python2.7/email/feedparser.py", line 27, in <module>
    from email import message
  File "/share/apps/lib/python2.7/email/message.py", line 16, in <module>
    import email.charset
  File "/share/apps/lib/python2.7/email/charset.py", line 13, in <module>
    import email.base64mime
  File "/share/apps/lib/python2.7/email/base64mime.py", line 40, in <module>
    from email.utils import fix_eols
  File "/share/apps/lib/python2.7/email/utils.py", line 27, in <module>
    import random
  File "/share/apps/src/numpy-1.9.3/numpy/random/__init__.py", line 99, in <module>
    from .mtrand import *
ImportError: No module named mtrand

which seems to the problem with from pkg_resources import load_entry_point

Yes, when it import random, we are already in the directory numpy/random/mtrand, it loads numpy/random first, which is the reason of the problem.

How to solve it then?

@rgommers

This comment has been minimized.

Copy link
Member

commented Jan 1, 2016

That looks like the same problem again. Do you perhaps have /share/apps/src/numpy-1.9.3/numpy on your PATH or PYTHONPATH? If so, you have to remove it from there. If not, I'm not sure - you'll have to debug the issue; figure out what is wrong in your environment. Maybe you can set a breakpoint above line 99 of /share/apps/src/numpy-1.9.3/numpy/random/__init__.py with pdb or with import IPython; IPython.embed() and see why import random doesn't find the random module in the stdlib.

@zcyang

This comment has been minimized.

Copy link
Author

commented Jan 1, 2016

oh, I made a stupid error in setting the environment variables... removing the path solves the problem. Many thanks!!

@maniteja123

This comment has been minimized.

Copy link
Contributor

commented Jan 1, 2016

Nice to hear the problem in solved. I need to learn the debugging ideas from @rgommers :). Sorry again for being unable to help before.

@rgommers

This comment has been minimized.

Copy link
Member

commented Jan 1, 2016

no worries, glad it's solved now

@njsmith

This comment has been minimized.

Copy link
Member

commented Jan 1, 2016

Sounds like we can close this then.

@njsmith njsmith closed this Jan 1, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.