Cython sometimes fails to build on Travis

[certik]

Example happened here: https://travis-ci.org/sympy/csympy/jobs/39208222, I restarted it. The failing log is here: https://gist.github.com/certik/de4811239380c3bc9210.

    gcc -pthread -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -g -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -fPIC -I/opt/python/3.2.5/include/python3.2mu -c /home/travis/virtualenv/python3.2.5/build/cython/Cython/Runtime/refnanny.c -o build/temp.linux-x86_64-3.2/home/travis/virtualenv/python3.2.5/build/cython/Cython/Runtime/refnanny.o
    gcc -pthread -shared -L/opt/python/3.2.5/lib -Wl,-rpath=/opt/python/3.2.5/lib build/temp.linux-x86_64-3.2/home/travis/virtualenv/python3.2.5/build/cython/Cython/Runtime/refnanny.o -L/opt/python/3.2.5/lib -lpython3.2mu -o build/lib.linux-x86_64-3.2/Cython/Runtime/refnanny.cpython-32mu.so
    cythoning /home/travis/virtualenv/python3.2.5/build/cython/Cython/Tempita/_tempita.py to /home/travis/virtualenv/python3.2.5/build/cython/Cython/Tempita/_tempita.c
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/home/travis/virtualenv/python3.2.5/build/cython/setup.py", line 337, in <module>
        **setup_args
      File "/opt/python/3.2.5/lib/python3.2/distutils/core.py", line 148, in setup
        dist.run_commands()
      File "/opt/python/3.2.5/lib/python3.2/distutils/dist.py", line 917, in run_commands
        self.run_command(cmd)
      File "/opt/python/3.2.5/lib/python3.2/distutils/dist.py", line 936, in run_command
        cmd_obj.run()
      File "/home/travis/virtualenv/python3.2.5/lib/python3.2/site-packages/setuptools/command/install.py", line 54, in run
        return _install.run(self)
      File "/opt/python/3.2.5/lib/python3.2/distutils/command/install.py", line 569, in run
        self.run_command('build')
      File "/opt/python/3.2.5/lib/python3.2/distutils/cmd.py", line 313, in run_command
        self.distribution.run_command(command)
      File "/opt/python/3.2.5/lib/python3.2/distutils/dist.py", line 936, in run_command
        cmd_obj.run()
      File "/opt/python/3.2.5/lib/python3.2/distutils/command/build.py", line 126, in run
        self.run_command(cmd_name)
      File "/opt/python/3.2.5/lib/python3.2/distutils/cmd.py", line 313, in run_command
        self.distribution.run_command(command)
      File "/opt/python/3.2.5/lib/python3.2/distutils/dist.py", line 936, in run_command
        cmd_obj.run()
      File "Cython/Distutils/build_ext.py", line 163, in run
        _build_ext.build_ext.run(self)
      File "/opt/python/3.2.5/lib/python3.2/distutils/command/build_ext.py", line 344, in run
        self.build_extensions()
      File "/home/travis/virtualenv/python3.2.5/build/cython/setup.py", line 169, in build_extensions
        build_ext_orig.build_extensions(self)
      File "Cython/Distutils/build_ext.py", line 170, in build_extensions
        ext.sources = self.cython_sources(ext.sources, ext)
      File "Cython/Distutils/build_ext.py", line 319, in cython_sources
        full_module_name=module_name)
      File "/home/travis/virtualenv/python3.2.5/build/cython/build/lib.linux-x86_64-3.2/Cython/Compiler/Main.py", line 627, in compile
        return compile_single(source, options, full_module_name)
      File "/home/travis/virtualenv/python3.2.5/build/cython/build/lib.linux-x86_64-3.2/Cython/Compiler/Main.py", line 580, in compile_single
        return run_pipeline(source, options, full_module_name)
      File "/home/travis/virtualenv/python3.2.5/build/cython/build/lib.linux-x86_64-3.2/Cython/Compiler/Main.py", line 406, in run_pipeline
        context = options.create_context()
      File "/home/travis/virtualenv/python3.2.5/build/cython/build/lib.linux-x86_64-3.2/Cython/Compiler/Main.py", line 527, in create_context
        self.cplus, self.language_level, options=self)
      File "/home/travis/virtualenv/python3.2.5/build/cython/build/lib.linux-x86_64-3.2/Cython/Compiler/Main.py", line 69, in __init__
        from . import Builtin, CythonScope
      File "/home/travis/virtualenv/python3.2.5/build/cython/build/lib.linux-x86_64-3.2/Cython/Compiler/CythonScope.py", line 5, in <module>
        from .UtilityCode import CythonUtilityCode
      File "/home/travis/virtualenv/python3.2.5/build/cython/build/lib.linux-x86_64-3.2/Cython/Compiler/UtilityCode.py", line 3, in <module>
        from .TreeFragment import parse_from_strings, StringParseContext
      File "/home/travis/virtualenv/python3.2.5/build/cython/build/lib.linux-x86_64-3.2/Cython/Compiler/TreeFragment.py", line 20, in <module>
        from . import Parsing
      File "Cython/Plex/Actions.pxd", line 2, in init Cython.Compiler.Parsing (/home/travis/virtualenv/python3.2.5/build/cython/Cython/Compiler/Parsing.c:55686)
        cdef class Action:
    ValueError: Cython.Plex.Actions.Action has the wrong size, try recompiling
    Complete output from command /home/travis/virtualenv/python3.2.5/bin/python -c "import setuptools, tokenize;__file__='/home/travis/virtualenv/python3.2.5/build/cython/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-gao9ra-record/install-record.txt --single-version-externally-managed --compile --install-headers /home/travis/virtualenv/python3.2.5/include/site/python3.2:
    running install

[thilinarmtb]

I found this by googling using the error message. Is this caused by the same error as one mentioned the thread, i.e. .c files are not updated to accommodate changes in .pxd files?

[argriffing]

This is also happening on some recent TravisCI scipy builds. I've only seen it with Python3.

[certik]

I don't understand how the .c files can not be updated. It's still the
same tarball and just restarting the build fixes it. It looks like some bug
in Cython to me.

Sent from my mobile phone.
On Oct 28, 2014 1:20 AM, "Thilina Bandara Rathnayake" <
notifications@github.com> wrote:

I found this
https://groups.google.com/forum/#!topic/cython-users/cOAVM0whJkY by
goggling using the error message. Is this caused by the same error as one
mentioned the thread, i.e. .c files are not updated to accommodate
changes in .pxd files?

—
Reply to this email directly or view it on GitHub
#308 (comment).

[argriffing]

Here's an example of a similar scipy TravisCI failure.
scipy/scipy#4126 (comment)
I also assume it's a Cython bug and that the same bug is affecting both the csympy and scipy builds.

[certik]

Thanks @argriffing, indeed, that's exactly the same problem. I think it's a Cython bug.

[certik]

@argriffing I reported the bug to the Cython mailinglist: https://mail.python.org/pipermail/cython-devel/2014-November/004261.html, I'll keep this issue posted with any updates.

[certik]

@njsmith posted a great suggestion here: https://mail.python.org/pipermail/cython-devel/2014-November/004263.html, so I've implemented it in #313.

[certik]

@argriffing, so I think the PR fixes it for us, so the solution is to use pip install --install-option="--no-cython-compile" cython (#313).

[argriffing]

According to the recent numpy and scipy PRs by @matthew-brett it looks TravisCI for like these projects will switch to the prebuilt cython binaries used by scikit-image, bypassing compilation of cython while still using the compiled version.

Update or not lol.