BUG: Unable to build 1.25.1 (or latest) from source on Windows 10, Visual Studio 2017, Python 3.10 and Cython 0.29.36. #24269
Comments
|
Somehow in your build the include directives are repeated over and over, here are just 3 of the many many repeats: I have seen this before when setting up the compiler calls Current best practices is that you should be running |
|
Hi @mattip, Thank you! The result is the same with I tried the editable build (with Trimmed log for |
|
How can you successfully execute |
If you are distributing this, you should be packaging with a BLAS backend. We package our wheels with OpenBLAS. |
|
I did not notice it originally because my terminal was filled up but the MSVC includes are added to the previous MSVC includes every single The first
Second
Third
As you can see, the |
|
Yes, I noticed the include repetitions in the first reply. Do you have any INCLUDE environment variable set before invoking the build command? We are moving away from You still have not clarified whether your goal is to build a wheel (in which case |
|
I unfortunately did not notice that it was growing after each invocation to Noted for Meson although this is at odds with what the building from source guide: https://numpy.org/doc/stable/user/building.html To clarify: We are not building wheels, we are deploying the built package to our network and use Rez to make it available to a particular environment. We also do build the package for different Python and VS versions as it can be used in different applications, e.g. Autodesk Maya, The Foundry Nuke that have varied Python version. There is a very keen interest for being able to continue to do so. Note that the I just tested the following permutations:
I will build an earlier Cython version and report back (Note that Cython 0.29.30 is the minimum version for Numpy 1.23.0 onward). |
|
Confirming that v1.25.1 fails with Cython 0.29.30, Cython 0.29.33, Cython 0.29.34, Cython 0.29.35 and Cython 0.29.36. |
|
Allright, after almost two handfuls of hours, I found out the issue! It was not Cython but Setuptools! I thought it was caused by Cython because our Cython 0.29.28 is built against Python 3.9 whereas our Cython 0.29.36 is built against Python 3.10, Our build system ended up using a different interpreter depending on the Cython version. The actual main difference was the version of Setuptools that is embedded with Python when installed on Windows. In the def _fix_compile_args(self, output_dir, macros, include_dirs):
"""Typecheck and fix-up some of the arguments to the 'compile()'
method, and return fixed-up values. Specifically: if 'output_dir'
is None, replaces it with 'self.output_dir'; ensures that 'macros'
is a list, and augments it with 'self.macros'; ensures that
'include_dirs' is a list, and augments it with 'self.include_dirs'.
Guarantees that the returned values are of the correct type,
i.e. for 'output_dir' either string or None, and for 'macros' and
'include_dirs' either list or None.
"""
if output_dir is None:
output_dir = self.output_dir
elif not isinstance(output_dir, str):
raise TypeError("'output_dir' must be a string or None")
if macros is None:
macros = self.macros
elif isinstance(macros, list):
macros = macros + (self.macros or [])
else:
raise TypeError("'macros' (if supplied) must be a list of tuples")
if include_dirs is None:
include_dirs = self.include_dirs
elif isinstance(include_dirs, (list, tuple)):
include_dirs = list(include_dirs) + (self.include_dirs or [])
else:
raise TypeError("'include_dirs' (if supplied) must be a list of strings")
# add include dirs for class
include_dirs += self.__class__.include_dirs
return output_dir, macros, include_dirsif As I looked at the Setuptools blame, I noted that the code has been fixed a few months ago which points to this PR: pypa/setuptools#3591 Note the coincidental comment from @mrbean-bremen: Originally posted by @mrbean-bremen in pypa/setuptools#3591 (comment) |
Hm, in hindsight it might have helped if I had opened an issue here for tracking... |
rgommers
added
32 - Installation
|
We're pinning to I think we can close this, since the problem was identified and there is nothing left to fix? |
|
Hey @rgommers, Sounds good with me! Cheers, Thomas |
|
Thanks for confirming! |
Describe the issue:
Hello,
I'm unable to build Numpy 1.25.1 (or latest) from source on Windows 10, Visual Studio 2017, Python 3.10 and Cython 0.29.36. The build either hangs or errors out. It took me quite a bit of time to figure out what is going on because INFO logging is disabled during that step (but can be re-enabled by commenting the relevant lines here:
numpy/numpy/distutils/command/build_src.py
Line 374 in 080cf82
clcompilation command is just too long because of the includes and yields a[WinError 206] The filename or extension is too long. There are 249512 characters when I think the limit on Windows is 32767.Reproduce the code example:
Error message:
Partial (~10% of the total length)
clcommand with some of the includes:The text was updated successfully, but these errors were encountered: