Windows wheel package (.whl) on Pypi

[mcarans]

Please make Windows wheel packages and put them on Pypi.

Currently it is possible to download Windows wheel packages for numpy here: http://www.lfd.uci.edu/~gohlke/pythonlibs/#numpy

It would be great if the wheels were directly available on the Pypi server https://pypi.python.org/pypi/ so that they can be installed with pip.

[matthew-brett]

Well said - and indeed there is a lot of work by @carlkl going on behind the scenes to make this happen. I believe we are nearly there now - @carlkl - when will you go public, do you think?

[njsmith]

For context: the reason this isn't trivial is that the binaries you linked
to depend on Intel's proprietary runtime and math library, which
complicates redistributing them.

[carlkl]

I deployed the recent OpenBLAS based numpy and scipy wheels on binstar. You can install them with:

pip install -i https://pypi.binstar.org/carlkl/simple numpy
pip install -i https://pypi.binstar.org/carlkl/simple scipy

This works for python-2.7 and for python-3.4. The wheels are marked as 'experimental'. Feedback is welcome.

[njsmith]

If you want widespread testing then you should send this to the list :-)

On Thu, Jan 22, 2015 at 8:54 PM, carlkl notifications@github.com wrote:

I deployed the recent OpenBLAS based numpy and scipy wheels on binstar.
You can install them with:

pip install -i https://pypi.binstar.org/carlkl/simple numpy
pip install -i https://pypi.binstar.org/carlkl/simple scipy

This works for python-2.7 and for python-3.4. The wheels are marked as
'experimental'. Feedback is welcome.

—
Reply to this email directly or view it on GitHub
#5479 (comment).

Nathaniel J. Smith
Postdoctoral researcher - Informatics - University of Edinburgh
http://vorpus.org

[juliantaylor]

fwiw I personally would like to change the size of the default integer in win64 before we actually provide official binaries, though there was some resistance too it when I last proposed it, also possibly with anaconda and other third party binaries its probably already too late :(

also speaking of openblas, someone fancy some debugging, I'm tired of it (looks like the same failure that breaks scipy with openblas):

test_einsum_sums_float64 (test_einsum.TestEinSum) ... ==31931== Invalid read of size 16
==31931==    at 0x7B28EB9: ddot_k_NEHALEM (in /usr/lib/libopenblasp-r0.2.10.so)
==31931==    by 0x6DBDA90: DOUBLE_dot (arraytypes.c.src:3127)
==31931==    by 0x6E93DEC: cblas_matrixproduct (cblasfuncs.c:528)
==31931==    by 0x6E6B7B3: PyArray_MatrixProduct2 (multiarraymodule.c:994)
==31931==    by 0x6E6E29B: array_matrixproduct (multiarraymodule.c:2276)

[carlkl]

OpenBLAS version used is 0.2.12. I didn't experienced significant problems with this version yet.

The scipy failures are copied to https://gist.github.com/carlkl/b05dc6055fd42eba8cc7.

32bit only numpy failures due to http://sourceforge.net/p/mingw-w64/bugs/367

======================================================================
FAIL: test_nan_outputs2 (test_umath.TestHypotSpecialValues)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:\tools\wp_279\python-2.7.9rc1\lib\site-packages\numpy\core\tests\test_umath.py", line 411, in test_nan_outputs2
    assert_hypot_isinf(np.nan, np.inf)
  File "D:\tools\wp_279\python-2.7.9rc1\lib\site-packages\numpy\core\tests\test_umath.py", line 402, in assert_hypot_isinf
    "hypot(%s, %s) is %s, not inf" % (x, y, ncu.hypot(x, y)))
  File "D:\tools\wp_279\python-2.7.9rc1\lib\site-packages\numpy\testing\utils.py", line 53, in assert_
    raise AssertionError(smsg)
AssertionError: hypot(nan, inf) is nan, not inf

======================================================================
FAIL: test_umath_complex.TestCabs.test_cabs_inf_nan(<ufunc 'absolute'>, inf, nan, inf)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:\tools\wp_279\python-2.7.9rc1\lib\site-packages\nose\case.py", line 197, in runTest
    self.test(*self.arg)
  File "D:\tools\wp_279\python-2.7.9rc1\lib\site-packages\numpy\core\tests\test_umath_complex.py", line 523, in check_real_value
    assert_equal(f(z1), x)
  File "D:\tools\wp_279\python-2.7.9rc1\lib\site-packages\numpy\testing\utils.py", line 275, in assert_equal
    return assert_array_equal(actual, desired, err_msg, verbose)
  File "D:\tools\wp_279\python-2.7.9rc1\lib\site-packages\numpy\testing\utils.py", line 739, in assert_array_equal
    verbose=verbose, header='Arrays are not equal')
  File "D:\tools\wp_279\python-2.7.9rc1\lib\site-packages\numpy\testing\utils.py", line 628, in assert_array_compare
    chk_same_position(x_isnan, y_isnan, hasval='nan')
  File "D:\tools\wp_279\python-2.7.9rc1\lib\site-packages\numpy\testing\utils.py", line 608, in chk_same_position
    raise AssertionError(msg)
AssertionError: 
Arrays are not equal

x and y nan location mismatch:
 x: array([ nan])
 y: array(inf)

======================================================================
FAIL: test_umath_complex.TestCabs.test_cabs_inf_nan(<ufunc 'absolute'>, -inf, nan, inf)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:\tools\wp_279\python-2.7.9rc1\lib\site-packages\nose\case.py", line 197, in runTest
    self.test(*self.arg)
  File "D:\tools\wp_279\python-2.7.9rc1\lib\site-packages\numpy\core\tests\test_umath_complex.py", line 523, in check_real_value
    assert_equal(f(z1), x)
  File "D:\tools\wp_279\python-2.7.9rc1\lib\site-packages\numpy\testing\utils.py", line 275, in assert_equal
    return assert_array_equal(actual, desired, err_msg, verbose)
  File "D:\tools\wp_279\python-2.7.9rc1\lib\site-packages\numpy\testing\utils.py", line 739, in assert_array_equal
    verbose=verbose, header='Arrays are not equal')
  File "D:\tools\wp_279\python-2.7.9rc1\lib\site-packages\numpy\testing\utils.py", line 628, in assert_array_compare
    chk_same_position(x_isnan, y_isnan, hasval='nan')
  File "D:\tools\wp_279\python-2.7.9rc1\lib\site-packages\numpy\testing\utils.py", line 608, in chk_same_position
    raise AssertionError(msg)
AssertionError: 
Arrays are not equal

x and y nan location mismatch:
 x: array([ nan])
 y: array(inf)

[njsmith]

I don't disagree about changing the win64 integer size, but I think that's
a separate issue that should be decoupled from the wheels. If this were the
first time win64 numpy builds were becoming widely available, then it'd
make sense to link them, but at this point there are already tons of users
for years, they're just using cgholke or anaconda or whatever. So let's
treat that as an independent discussion?

(Strictly speaking it's a backcompat break, but even so it seems reasonable
that we might be able to pull it off, since it actually reduces
incompatibility between platforms -- all portable code has to handle 64-bit
dtype=int already.)

On Thu, Jan 22, 2015 at 8:59 PM, Julian Taylor notifications@github.com
wrote:

fwiw I personally would like to change the size of the default integer in
win64 before we actually provide official binaries, though there was some
resistance too it when I last proposed it, also possibly with anaconda and
other third party binaries its probably already too late :(

also speaking of openblas, someone fancy some debugging, I'm tired of it
(looks like the same failure that breaks scipy with openblas):

test_einsum_sums_float64 (test_einsum.TestEinSum) ... ==31931== Invalid read of size 16
==31931== at 0x7B28EB9: ddot_k_NEHALEM (in /usr/lib/libopenblasp-r0.2.10.so)
==31931== by 0x6DBDA90: DOUBLE_dot (arraytypes.c.src:3127)
==31931== by 0x6E93DEC: cblas_matrixproduct (cblasfuncs.c:528)
==31931== by 0x6E6B7B3: PyArray_MatrixProduct2 (multiarraymodule.c:994)
==31931== by 0x6E6E29B: array_matrixproduct (multiarraymodule.c:2276)

—
Reply to this email directly or view it on GitHub
#5479 (comment).

Nathaniel J. Smith
Postdoctoral researcher - Informatics - University of Edinburgh
http://vorpus.org

[TFenby]

I'm also interested in this. Is there some way of assisting with the process?

[carlkl]

OpenBLAS can be compiled with INTERFACE64=1 and numpy can be compiled with -fdefault-integer-8 for a first try.

[tkelman]

Just a heads up. Using 64 bit integers in blas is a terrible idea. Stop before you get too far down that road. Matlab, and Julia before I went and fixed it, did this, and it breaks any third-party library that assumes conventional 32-bit integers in blas.

What we've been doing in Julia for the past ~5 months is actually renaming all the symbols in openblas to add a _64 suffix to them for the 64-bit-int version, that way you can do linear algebra on really huge arrays if you want, but loading external libraries into the same process won't segfault by name shadowing and trying to call dgemm with the wrong ABI.

[guyverthree]

Hey guys is there any update on the wheels files being made available for Numpy ?

[njsmith]

Not that I'm aware of right now.
On Jun 25, 2015 4:27 AM, "guyverthree" notifications@github.com wrote:

Hey guys is there any update on the wheels files being made available for
Numpy ?

—
Reply to this email directly or view it on GitHub
#5479 (comment).

[mikofski]

@guyverthree Christoph Gohlke has been releasing NumPy using Intel's MKL as wheels for a while now.

Also, see my blog post on NumPy wheels. I made some NumPy wheels in my Dropbox using Carl Kleffner's modified mingw-w64 toolchain and Zhang Xianyi's OpenBLAS port of GotoBLAS. Olivier Grisel was looking for help modifying the NumPy buildbot to repeat the same steps used in the OpenBLAS google groups thread I post to.

[carlkl]

My latest version is available on binstar.org though I'm not sure if anaconda.org is the new prefered name now.
The wheels for py-2.6 .. 3.4 (32/64bit) are about 2 months old:

• numpy-1.9.2
• scipy-0.15.1
• scikit-image-0.11.2

build with my https://bitbucket.org/carlkl/mingw-w64-for-python and a more or less recent OpenBLAS.
pip install:

• pip install -i https://pypi.binstar.org/carlkl/simple numpy
• pip install -i https://pypi.binstar.org/carlkl/simple scipy