Compilation fails after moving to 0.27

[Berserker66]

Updated from 0.26.1 to 0.27.0 and a module of mine has now stopped compiling properly.

E:\PySideFactory\cython_setup.bat

E:\PySideFactory>call "E:\Visual Studio 2017 Community\VC\Auxiliary\Build\vcvarsx86_amd64.bat" 
**********************************************************************
** Visual Studio 2017 Developer Command Prompt v15.0.26730.3
** Copyright (c) 2017 Microsoft Corporation
**********************************************************************
[vcvarsall.bat] Environment initialized for: 'x86_x64'
running build_ext
cythoning accelerate.pyx to accelerate.c
building 'accelerate' extension
E:\Visual Studio 2017 Community\VC\Tools\MSVC\14.11.25503\bin\HostX86\x64\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD "-IE:\Visual Studio 2017 Community\VC\Redist\MSVC\14.10.25017\onecore\x64\Microsoft.VC150.CRT" "-IC:\Program Files\Python36\lib\site-packages\numpy\core\include" "-IC:\Program Files\Python36\include" "-IC:\Program Files\Python36\include" "-IC:\Program Files (x86)\Windows Kits\10\Include\10.0.15063.0\ucrt" "-IC:\Program Files (x86)\Visual C++ BuildTools\VC\Tools\MSVC\14.11.25503\include" "-IE:\Visual Studio 2017 Community\SDK\ScopeCppSDK\SDK\include\shared" /Tcaccelerate.c /Fobuild\temp.win-amd64-3.6\Release\accelerate.obj
accelerate.c
c:\program files\python36\lib\site-packages\numpy\core\include\numpy\npy_1_7_deprecated_api.h(12) : Warning Msg: Using deprecated NumPy API, disable it by #defining NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION
accelerate.c(2795): warning C4244: '=': conversion from 'double' to 'float', possible loss of data
accelerate.c(2950): warning C4244: '=': conversion from 'Py_ssize_t' to 'int', possible loss of data
accelerate.c(3153): warning C4244: 'function': conversion from 'int' to 'float', possible loss of data
accelerate.c(3173): warning C4244: 'function': conversion from 'int' to 'float', possible loss of data
accelerate.c(3182): warning C4244: 'function': conversion from 'int' to 'float', possible loss of data
accelerate.c(3324): warning C4244: '=': conversion from 'int' to 'float', possible loss of data
accelerate.c(5664): warning C4244: '=': conversion from 'Py_ssize_t' to 'int', possible loss of data
accelerate.c(26561): error C2065: '__pyx_7genexpr__pyx_v_10accelerate_12TileRenderer_i': undeclared identifier
accelerate.c(26571): error C2065: '__pyx_7genexpr__pyx_v_10accelerate_12TileRenderer_x': undeclared identifier
accelerate.c(26571): warning C4312: 'type cast': conversion from 'int' to 'PyObject *' of greater size
accelerate.c(26571): warning C4047: '=': 'int' differs in levels of indirection from 'PyObject *'
accelerate.c(26573): error C2065: '__pyx_7genexpr__pyx_v_10accelerate_12TileRenderer_i': undeclared identifier
accelerate.c(26575): error C2065: '__pyx_7genexpr__pyx_v_10accelerate_12TileRenderer_x': undeclared identifier
accelerate.c(26575): warning C4047: 'function': 'PyObject *' differs in levels of indirection from 'int'
accelerate.c(26575): warning C4024: 'PyNumber_Add': different types for formal and actual parameter 1
accelerate.c(26583): error C2065: '__pyx_7genexpr__pyx_v_10accelerate_12TileRenderer_x': undeclared identifier
accelerate.c(26583): warning C4312: 'type cast': conversion from 'int' to 'PyObject *' of greater size
accelerate.c(26586): error C2065: '__pyx_7genexpr__pyx_v_10accelerate_12TileRenderer_x': undeclared identifier
accelerate.c(26586): warning C4312: 'type cast': conversion from 'int' to 'PyObject *' of greater size
accelerate.c(29927): warning C4244: 'initializing': conversion from 'double' to 'float', possible loss of data
accelerate.c(29933): warning C4244: 'initializing': conversion from 'double' to 'float', possible loss of data
error: command 'E:\\Visual Studio 2017 Community\\VC\\Tools\\MSVC\\14.11.25503\\bin\\HostX86\\x64\\cl.exe' failed with exit status 2

Process finished with exit code 1

The module, as well as its .c is here:
https://www.dropbox.com/sh/cmv6z8hjlygwyn3/AACXPFPGzmAEQVnVmrsZXdvra?dl=0

Help in resolving this would be much appreciated. I looked around if someone else has the same issue, but I don't think so - 400 open issues is a bit to try and filter through.

Win 10, python 3.6.2 64bit

[Berserker66]

Turns out removing language-level=3 makes it work.

[scoder]

Thanks for the report. Looks like something is going wrong with the comprehension loop variables in the class namespace here.

I think the best work-around for you is to move the complex NumPy array creation into a cdef helper function. Function scopes are generally more efficient than class scopes.

[scoder]

The problem is that the loop variables end up in the class namespace:

/* "accelerate.pyx":19
 * cdef class TileRenderer:             # <<<<<<<<<<<<<<
 *     npindexes = numpy.array([x + i for i in range(0, 256 * 4, 4) for x in [0, 1, 2, 2, 3, 0]], dtype=numpy.uint32)
 */
struct __pyx_obj_10accelerate_TileRenderer {
  PyObject_HEAD
  struct __pyx_vtabstruct_10accelerate_TileRenderer *__pyx_vtab;
  [...]
  int chunksinregion;
  long __pyx_7genexpr__pyx_v_10accelerate_12TileRenderer_i;
  PyObject *__pyx_7genexpr__pyx_v_10accelerate_12TileRenderer_x;
};

But then they are not properly used from there but as global names:

  { /* enter inner scope */
    __pyx_t_4 = PyList_New(0); if (unlikely(!__pyx_t_4)) __PYX_ERR(0, 21, __pyx_L4_error)
    __Pyx_GOTREF(__pyx_t_4);
    for (__pyx_t_5 = 0; __pyx_t_5 < 0x400; __pyx_t_5+=4) {
      __pyx_7genexpr__pyx_v_10accelerate_12TileRenderer_i = __pyx_t_5;

With language_level=3, they should not end up in the class scope at all.

[scoder]

Fixed in acd03a2.

[Berserker66]

Thank you!

[scoder]

Fixed in 0.27.1b1. Please give it a try.

[Berserker66]

I downloaded the tarball, and ran the install; this is the result:

https://pastebin.com/4xUG96F7
Seems unrelated. Should I open a new issue for that, or how should we proceed?

Speculation here; I don't know much at all about the inner workings of cython, but it seems to not find io.h; maybe related to visual studio being on E: ?

[scoder]

Hmm - that does seem unrelated. I suggest ignoring it until it proves to be relevant. ;)

[Berserker66]

It might be a general problem - I usually install modules, such as Cython through pycharm. Which I think just pulls pre-compiled wheels, if they are available.

I got 0.27.0 and tried manually installing it and that too fails like it.

When I actually compile modules I have to call vcvarsall manually as well (well, as the start of my batch file), from the documentation it kind of sounds like that should be automatic?