Upgrade NumPy to 1.18.5, scipy to 1.4.1, add pybind11 package

[mkoeppe]

Follow-up from the update of numpy to 1.16.6 (#29429):

We update

• NumPy to the latest release, 1.18.5
• scipy to latest release, 1.4.1.
  (latest as of 2020-06-05; both versions (and also the upcoming numpy 1.19) support python >= 3.6)

Depends on #29480

CC: @isuruf @dimpase @jdemeyer @embray @slel @vbraun

Component: porting

Author: Matthias Koeppe

Branch/Commit: u/mkoeppe/trac/local_conda_forge_macos_standard__build_error_with_numpy @ 56a6fe1

Reviewer: Isuru Fernando, Dima Pasechnik

Issue created by migration from https://trac.sagemath.org/ticket/29425

[isuruf]

comment:1

Can you try setting NPY_DISTUTILS_APPEND_FLAGS=1 in the numpy build script?

[mkoeppe]

comment:2

This didn't make a difference

[mkoeppe]

comment:3

Upgrading numpy to 1.18.2 fixes it; but this will require a Cython upgrade too

[mkoeppe]

Branch: u/mkoeppe/local_conda_forge_macos_standard__build_error_with_numpy

[mkoeppe]

Commit: 6170b87

[mkoeppe]

Author: Matthias Koeppe

[mkoeppe]

New commits:

adf988d	build/pkgs/cython: Update to 0.29.16
6170b87	build/pkgs/numpy: Update numpy to 1.18.2, back out numpy-1.10.2-no-hardcode-blas.patch (does not apply)

[mkoeppe]

comment:7

Tests run at https://github.com/mkoeppe/sage/actions/runs/66127777

[isuruf]

comment:8

Cython URL can be set to https://pypi.io/packages/source/C/Cython/Cython-VERSION.tar.gz

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

078a653

build/pkgs/cython/checksums.ini: Use predictable URL for upstream_url pattern

[sagetrac-git]

Changed commit from 6170b87 to 078a653

[mkoeppe]

comment:10

Thanks!

[isuruf]

Reviewer: Isuru Fernando

[mkoeppe]

comment:12

Breaks py2 builds

Traceback (most recent call last):
  File "setup.py", line 32, in <module>
    raise RuntimeError("Python version >= 3.5 required.")
RuntimeError: Python version >= 3.5 required.
}}

[mkoeppe]

comment:13

Setting milestone to 9.2, which is presumably the first py3-only release

[mkoeppe]

comment:47

An upgrade ticket like this really needs to be run on the GitHub test infrastructure...

[dimpase]

comment:48

running GH Actions test here https://github.com/dimpase/sage/pull/9

[dimpase]

comment:49

Replying to @mkoeppe:

An upgrade ticket like this really needs to be run on the GitHub test infrastructure...

Unfortunately there seem to be problems with runner setups, e.g. here
https://github.com/dimpase/sage/runs/740349791?check_suite_focus=true

• this is docker (archlinux-latest, standard-gcc_spkg)
  I see

2020-06-04T22:39:01.9688082Z Suggested packages:
2020-06-04T22:39:01.9688790Z   python3-pytest python-setuptools-doc

and a lot of

2020-06-05T03:08:41.2134392Z Failure: ModuleNotFoundError (No module named 'pytest') ... ERROR

which seems to suggest that python3-pytest is a requirement.

Do you know how to fix this?

(there are other errors pointing at internal problems of runners)

Can we easily extend setup to use our own runners?

[mkoeppe]

comment:50

Apparently, running the testsuite of networkx requires pytest. Let's discuss this on #29766.

I don't think this has anything to do with GitHub.

[mkoeppe]

comment:51

In the meantime, numpy 1.18.5 has appeared

[mkoeppe]

Changed branch from u/dimpase/trac/local_conda_forge_macos_standard__build_error_with_numpy to u/mkoeppe/trac/local_conda_forge_macos_standard__build_error_with_numpy

[mkoeppe]

Changed commit from c005562 to 56a6fe1

[mkoeppe]

comment:53

Tests (with #29766) run at https://github.com/mkoeppe/sage/pull/38/checks

---

New commits:

56a6fe1

build/pkgs/numpy: Update to 1.18.5

[mkoeppe]

comment:56

Can this ticket be tested successfully by itself or only as part of the networkx upgrade in #29766? If the latter, then let's close this ticket and merge it into #29766.

[dimpase]

comment:57

Replying to @mkoeppe:

Can this ticket be tested successfully by itself or only as part of the networkx upgrade in #29766? If the latter, then let's close this ticket and merge it into #29766.

no, this one does not need #29766

[mkoeppe]

comment:58

OK, running a new test without #29766 at https://github.com/mkoeppe/sage/pull/39/checks

[mkoeppe]

comment:59

Replying to @dimpase:

Replying to @mkoeppe:

Can this ticket be tested successfully by itself or only as part of the networkx upgrade in #29766? If the latter, then let's close this ticket and merge it into #29766.

no, this one does not need #29766

Actually, on ubuntu-xenial-standard (https://github.com/mkoeppe/sage/runs/745921419), I see lots of failures from the networkx testsuite such as this one:

File "/sage/local/var/tmp/sage/build/networkx-2.2/src/networkx/drawing/nx_pylab.py", line 249, in networkx.drawing.nx_pylab.draw_networkx
Failed example:
    nx.draw(G)
Exception raised:
    Traceback (most recent call last):
      File "/sage/local/lib/python3.7/doctest.py", line 1329, in __run
        compileflags, 1), test.globs)
      File "<doctest networkx.drawing.nx_pylab.draw_networkx[1]>", line 1, in <module>
        nx.draw(G)
      File "/sage/local/var/tmp/sage/build/networkx-2.2/src/networkx/drawing/nx_pylab.py", line 126, in draw
        draw_networkx(G, pos=pos, ax=ax, **kwds)
      File "/sage/local/var/tmp/sage/build/networkx-2.2/src/networkx/drawing/nx_pylab.py", line 278, in draw_networkx
        edge_collection = draw_networkx_edges(G, pos, arrows=arrows, **kwds)
      File "/sage/local/var/tmp/sage/build/networkx-2.2/src/networkx/drawing/nx_pylab.py", line 611, in draw_networkx_edges
        if cb.is_numlike(alpha):
    AttributeError: module 'matplotlib.cbook' has no attribute 'is_numlike'

[mkoeppe]

comment:60

Therefore, let's close this ticket and continue on #29766.