Manual scipy build for win64 py38

[mariusvniekerk]

I've built scipy manually on an azure vm provisioned with VS2017 + 2019 and Intel IFort.

The only changes to the recipe needed were for compiler activations.

Almost all the tests pass natively aside from the following

2 failed, 14587 passed, 1285 skipped, 79 xfailed, 8 xpassed, 40 warnings in 372.07s (0:06:12)

================================== FAILURES ===================================
__________________ TestDualAnnealing.test_slsqp_ls_minimizer __________________
[gw0] win32 -- Python 3.8.0 %PREFIX%\python.exe

self = <scipy.optimize.tests.test__dual_annealing.TestDualAnnealing testMethod=test_slsqp_ls_minimizer>

    def test_slsqp_ls_minimizer(self):
        minimizer_opts = {
            'method': 'SLSQP',
        }
        ret = dual_annealing(self.func, self.ld_bounds,
                             local_search_options=minimizer_opts)
>       assert_allclose(ret.fun, 0., atol=1e-7)
E       AssertionError:
E       Not equal to tolerance rtol=1e-07, atol=1e-07
E
E       Mismatch: 100%
E       Max absolute difference: 1.67455276e-07
E       Max relative difference: inf
E        x: array(1.674553e-07)
E        y: array(0.)

minimizer_opts = {'method': 'SLSQP'}
ret        =      fun: 1.6745527631201185e-07
 message: ['Maximum number of iteration reached']
    nfev: 4045
    nhev: 0
     nit: 1000
    njev: 8
  status: 0
 success: True
       x: array([2.90527124e-05, 4.65601445e-08])
self       = <scipy.optimize.tests.test__dual_annealing.TestDualAnnealing testMethod=test_slsqp_ls_minimizer>

..\_test_env\Lib\site-packages\scipy\optimize\tests\test__dual_annealing.py:259: AssertionError
____________________ TestQRdelete_F.test_delete_last_p_col ____________________
[gw2] win32 -- Python 3.8.0 %PREFIX%\python.exe

self = <scipy.linalg.tests.test_decomp_update.TestQRdelete_F object at 0x0000019C23FC40A0>

    def test_delete_last_p_col(self):
        a, q, r = self.generate('tall', 'full')
        q1, r1 = qr_delete(q, r, 0, a.shape[1], 'col')
>       assert_unitary(q1)

a          = array([[0.36852294+0.7065659j , 0.9763755 +0.8815727j ,
        0.73179656+0.5390478j , 0.6848747 +0.2577209j ,
      ...0.8069605j ,
        0.2923851 +0.9023696j , 0.08459105+0.7541054j ,
        0.8391204 +0.23152167j]], dtype=complex64)
q          = array([[-0.11519074-0.22085421j,  0.4031076 +0.25545135j,
         0.20641464-0.02725353j,  0.04916074+0.12054591j,
  ....17945503j,  0.10257003-0.11285198j,
        -0.05898733+0.03267026j,  0.5694973 -0.06679238j]],
      dtype=complex64)
q1         = array([[-0.11519074-0.22085421j,  0.4031076 +0.25545135j,
         0.20641464-0.02725353j,  0.04916074+0.12054591j,
  ....17945503j,  0.10257003-0.11285198j,
        -0.05898733+0.03267026j,  0.5694973 -0.06679238j]],
      dtype=complex64)
r          = array([[-3.1992414 +0.j        , -2.1899545 +0.23892963j,
        -1.6467261 +0.47293147j, -2.4033258 +0.41776413j,
  ...        ,
         0.        +0.j        ,  0.        +0.j        ,
         0.        +0.j        ]], dtype=complex64)
r1         = array([], shape=(12, 0), dtype=complex64)
self       = <scipy.linalg.tests.test_decomp_update.TestQRdelete_F object at 0x0000019C23FC40A0>

..\_test_env\Lib\site-packages\scipy\linalg\tests\test_decomp_update.py:327:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

a = array([[-0.11519074-0.22085421j,  0.4031076 +0.25545135j,
         0.20641464-0.02725353j,  0.04916074+0.12054591j,
  ....17945503j,  0.10257003-0.11285198j,
        -0.05898733+0.03267026j,  0.5694973 -0.06679238j]],
      dtype=complex64)
rtol = 0.0001, atol = 2.384185791015625e-07, assert_sqr = True

    def assert_unitary(a, rtol=None, atol=None, assert_sqr=True):
        if rtol is None:
            rtol = 10.0 ** -(np.finfo(a.dtype).precision-2)
        if atol is None:
            atol = 2*np.finfo(a.dtype).eps

        if assert_sqr:
            assert_(a.shape[0] == a.shape[1], 'unitary matrices must be square')
        aTa = np.dot(a.T.conj(), a)
>       assert_allclose(aTa, np.eye(a.shape[1]), rtol=rtol, atol=atol)
E       AssertionError:
E       Not equal to tolerance rtol=0.0001, atol=2.38419e-07
E
E       Mismatch: 1.39%
E       Max absolute difference: 3.57627869e-07
E       Max relative difference: 3.57627869e-07
E        x: array([[ 1.000000e+00+0.000000e+00j, -1.676381e-08+1.490116e-08j,
E                0.000000e+00+1.490116e-08j, -2.793968e-08-2.980232e-08j,
E               -2.235174e-08-1.862645e-08j, -2.095476e-08+5.587935e-09j,...
E        y: array([[1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
E              [0., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
E              [0., 0., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0.],...

a          = array([[-0.11519074-0.22085421j,  0.4031076 +0.25545135j,
         0.20641464-0.02725353j,  0.04916074+0.12054591j,
  ....17945503j,  0.10257003-0.11285198j,
        -0.05898733+0.03267026j,  0.5694973 -0.06679238j]],
      dtype=complex64)
aTa        = array([[ 1.0000001e+00+0.0000000e+00j, -1.6763806e-08+1.4901161e-08j,
         0.0000000e+00+1.4901161e-08j, -2.793967...3484e-08-4.4703484e-08j,
        -8.3819032e-09-2.9802322e-08j,  9.9999994e-01+0.0000000e+00j]],
      dtype=complex64)
assert_sqr = True
atol       = 2.384185791015625e-07
rtol       = 0.0001

These tests are minor enough in failure that i'm uploading the artifact built to Anaconda.org

[mariusvniekerk]

Uploaded as https://anaconda.org/conda-forge/scipy/1.3.2/download/win-64/scipy-1.3.2-py38h582fac2_0.tar.bz2

[rgommers]

Awesome, thanks @mariusvniekerk. I'll fix those test failures.

[rgommers]

The linalg test was already fixed, and reducing test precision for the other failure is done in scipy/scipy#11054

[jakirkham]

Thanks @mariusvniekerk! 😀

[ocefpaf]

I've built scipy manually on an azure vm provisioned with VS2017 + 2019 and Intel IFort.

Cool. It would be nice to have that documented that for cfep-03

[peterdhansen]

Is this the change that broke everything for me?

I am using conda-forge with strict priority as recommended so this build replaced the default channel, but the requirements for this package conflict with a lot of things that depend on scipy (specifically scikit-learn for example)

[isuruf]

@peterdhansen, please open another issue with the requested information in the issue template.

[ocefpaf]

Is this the change that broke everything for me?

I am using conda-forge with strict priority as recommended so this build replaced the default channel, but the requirements for this package conflict with a lot of things that depend on scipy (specifically scikit-learn for example)

Are you using Python 3.8 for that? If not, this change did should not affect you.

[h-vetinari]

@mariusvniekerk Was the build time below the 6h CI limit (or is it expected to be)?

If not (fingers crossed), why not use try to use this already for the windows builds in the recipe?

[tboerstad]

Is there a way I can help to manually build and upload a new version of scipy for win64 to conda-forge?

Considering Scipy 1.4 is now out, it would be helpful to have it for win64 as well.

[h-vetinari]

conda-forge/cfep#5 has been merged. Could we retry the windows builds for scipy? :)

[mariusvniekerk]

I'll have to go resurrect / rebuild the azure machine i used to do this.

[h-vetinari]

@mariusvniekerk: I'll have to go resurrect / rebuild the azure machine i used to do this.

With the impending release of scipy 1.5.0, I thought I'd ping on this once more...