GitHub is home to over 20 million developers working together to host and review code, manage projects, and build software together.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
Already on GitHub? Sign in to your account
The possibility for a silent numpy and/or scipy installer has been requested a number of times.
I have made a number of changes that should enable this, please review this. I'm not familiar with compiling numpy, so this is untested!
The basic idea is to create MSI installers instead of exe's. Then we can use msiexec and pass it a silent flag if the NSIS installer was passed a silent flag. This can be achieved using the NSIS 'IfSilent' directive.
Thanks a lot for having a go at this! I'll try to do some testing next week (only under Wine though).
Any news on this?
Sorry, didn't get around to it yet. Just tried, first issue:
$ paver bdist_wininst_simple -p 2.7 -m 1
optparse.OptionError: invalid long option string '-msi': must start with --,followed by non-dash
This needs to be ("make_msi=", "-msi", .... Same for other lines like this.
("make_msi=", "-msi", ...
I disagree ;)
Otherwise the line above should also be ("python-version", "-p", ... instead of ("python-version", "p", ...
But you are correct this line is the culprit, it should be:
("make_msi", "m", "...")
Because single-dash options need to be single-character options, double-dash options are long options. Will change this accordingly.
("python-version", "-p", ...
("python-version", "p", ...
("make_msi", "m", "...")
Traceback (most recent call last):
File "setup.py", line 251, in <module>
File "setup.py", line 243, in setup_package
File "Z:\Users\rgommers\Code\numpy\numpy\distutils\core.py", line 169, in setup
File "C:\Python27\lib\distutils\core.py", line 152, in setup
File "C:\Python27\lib\distutils\dist.py", line 953, in run_commands
File "C:\Python27\lib\distutils\dist.py", line 972, in run_command
File "C:\Python27\lib\distutils\command\bdist_msi.py", line 232, in run
sversion = "%d.%d.%d" % StrictVersion(version).version
File "C:\Python27\lib\distutils\version.py", line 40, in __init__
File "C:\Python27\lib\distutils\version.py", line 107, in parse
raise ValueError, "invalid version number '%s'" % vstring
ValueError: invalid version number '1.9.0.dev-Unknown'
Will only happen when you run a non-released build in a way where git cannot be found (like building master under Wine, with git not installed in Wine).
File "numpy\core\setup_common.py", line 92, in check_api_version
setup_common.MismatchCAPIWarning: API mismatch detected, the C API version numbers have to be updated. Current C api version is 9, with checksum 3e1c969a871ad01c622aa1432a04cdc5, but recorded checksum for C API version 9 in codegen_dir/cversions.txt is 49b27dc2dc7206a775a7376fdbc3b80c. If functions were added in the C API, you have to update C_API_VERSION in numpy\core\setup_common.py.
I don't see so quickly why this warning results in an exception. Looks build_msi specific. Would be nice to have fixed, because we don't always update the C_API_VERSION in master (only before a release).
EDIT: also happens for a normal build. Looks like a regression in master (?!).
Update to fix the short build_msi command line option to "m" instead of the invalid "msi". See [this comment](numpy#4727 (comment))
I'm interpreting there's not much I can do about the last two errors? If so, I'll be impatiently awaiting a slot of your time for further testing after these are resolved ;-)
I'm currently testing the build process on my Linux machine using wine and MingW, and this seems to be working (albeit without the blas/lapack/atlas support, by forcing a "release" or distutils keeps complaining and by updating the C API checksums).
Can you confirm the issues are resolved?
I have been able to compile and install the superpack in and on Wine. This confirms the changes I've made are working. The issues with 'actually' compiling numpy are not related to my changes so I see no more reason not to move forward with the merge?
For anyone interested in how I got it to build, this is a script I wrote to keep track of everything I needed to do. It is untested, but should give you a rough idea of what to do and where to find everything.
echo "CLEANING WINEPREFIX:"
rm -Rfv "`pwd`/.wine"
echo "DONE CLEANING";
# Download & install SCONS:
# Download & install git:
# Download & install python and add to path:
wine msiexec /i python-2.7.7.msi /qn
# This is a registry file exported using regedit where I added the Python,
# MinGW and NSIS paths to the PATH environemnt variable in
wine regedit python27+mingw32+NSIS.reg
# Download & install setuptools and pip:
wine python ez_setup.py
wine easy_install pip
# Download & install MingW and set Python default compiler to MingW:
printf "[build]\ncompiler = mingw32" > /home/mathijs/python_wine_prefix/.wine/drive_c/Python27/Lib/distutils/distutils.cfg
# Download & install cython
wine pip install cython
# Download & install msvc dll:
# Download & install paver:
wine easy_install paver
# Download & install NSIS:
# TODO you also need to install the CpuCaps dll. I managed to compile it from µ
# the numpy source files using MingW by first installing this:
# mingw-get.exe upgrade "w32api=3.17-2"
# Second step is to add a header file to the MingW installation,
# called "sdkddkver.h". Put it in the C:\MinGW\include\ of the wine prefix.
# Contents are:
# ifdef WINVER
# define _WIN32_WINNT WINVER
# ifdef _WARN_DEFAULTS
# warning _WIN32_WINNT is defaulting to _WIN32_WINNT_WIN2K
# define _WIN32_WINNT _WIN32_WINNT_WIN2K
# define WINVER _WIN32_WINNT
# Then I could cd into the source dir:
# cd numpy/tools/win32build
# wine python build-cpucaps.py
# If all goes well, this should install the CpuCaps.dll file where it should be.
# Finally clone git & build numpy:
git clone https://github.com/mathijs-dumon/numpy.git
wine paver bdist_superpack -p 2.7 -m 1
#wine python setup.py build --compiler=mingw32 bdist_wininst
# If the installer builds the exe's but fails at making the superscript, you can
# run this last part seperately like this;
# cd build-superpack/
# wine makensis numpy-superinstaller.nsi
@rgommers: sorry for bugging you, but what's the status on this? If at all possible I'd like to see this land for the next release.
I built msi binaries using this patch on our wine build setup and tried installing them but it immediately gives an error with no further useful information.
the regular exe installers work fine
nope made a mistake the regular exe installers are broken with this patch too
That's strange, when I tested the installation went fine. I won't be able to test again until tuesday (maybe), if I can test I'll report back (and also test on a windows install).
Changes related to solving issue #572 by creating MSI installers and …
…adjusting the NSIS installer script to allow for a silent install on windows.
Update to fix the short build_msi command line option to "m" instead of the invalid "msi". See [this comment](#4727 (comment))
Removed global variable and replaced by proper function arguments.
Alright, time for some more work on this:
I'll get back in a few minutes if this build works out
@mathijs-dumon Is this something that might get worked out at Euroscipy?
Sadly I'm not able to go to Euroscipy, so unless somebody else has a go at this there, it'll have to go the electronic way. I just need to find the time to sit down and figure out what is causing this, but with the start of the academic year in a few weeks, I'm pretty tied up for the time being.
@charris @rgommers @juliantaylor
I have been able to successfully produce an msi installer with the current patch. However, building this installer under wine (either linux or mac) does not work and produces the error we've seen above.
Silent install also worked as expected when built under windows using MinGW.
Can an 'official' builder confirm wether this works or not?
@mathijs-dumon @rgommers @juliantaylor I'm tempted to close this. Is the Wine failure important? Is there another way to get the same result?
Well, I mostly wanted this for my own project, but have resorted to manually unpacking the three versions and using a python script to detect the capabilities of the user's PC and select the correct version. It works, but I would have rather had a way supported by the project.
So you may close this, I have no idea why this doesn't work under wine or how to solve it anyway, and I don't have enough time to figure it out anytime soon.
If we get the new OpenBLAS / mingw-w64 binaries working then we won't have this problem any more because we won't need the separate installer step to choose different versions according to the CPU, because OpenBLAS detects the CPU at run-time.
It feels like a shame to close this, because a silent installer is nice to have. However, I don't really have the time to dig into Wine issues......
OK, closing this. It can be found with a search after if someone wants to open it again.
@mathijs-dumon thanks for the effort! I hope this'll still make it in somehow in the future