Still unable to install scipy, after switch to local toolchain

[ashesfall]

          The most recent version of graalpy doesn't use LLVM for C extension execution anymore by default, it would use your system toolchain, so the problem reported here shouldn't exist in graalpy 23.0. `scipy` would build with your system `gfortran` as it would for CPython. Please reopen the issue if you still observe problems.

Originally posted by @msimacek in #265 (comment)

After updating, I am still unable to install scipy. The ginstall command now says it isn't available, and pip also appears to be unavailable. If I use ginstall to install pip, I get a warning when using it:

 RuntimeWarning: You are using an untested version of pip. GraalPy provides patches and workarounds for a number of packages when used with compatible pip versions. We recommend to stick with the pip version that ships with this version of GraalPy.

But pip does not seem to ship with GraalPy.

Anyway, using the pip installed via ginstall to then install scipy still fails but does not seem to emit an entirely clear error message. You can see the full output in this gist:
https://gist.github.com/ashesfall/612c8ec7bef877f01e5b8e49df51ba76

Installing the failed requirement, numpy, by itself produces roughly similar behavior.

It seems odd to me that support for torch and scipy is mentioned as a long term goal in the readme, but there is no description of what an early experimenter should expect if they try and install either of these. I'd like to help out with the project in some way (at least through experimentation), but I don't know where to find a basic summary of the current expected behavior.

[qunaibit]

Could you please try to install numpy==1.23.5 before attempting to install scipy

[msimacek]

It's not a recommended practice to install dependencies directly into the installation directory. It's better to start with creating a virtualenv with graalpy -m venv my-virtualenv. When you activate it, you'll get working pip that should be able to install scipy into that virtualenv.

ginstall is deprecated and it will likely be removed at some point. It's a leftover from the times when pip didn't work on graalpy yet. Nowadays, pip should be used for all dependency manipulation.

If you really wanted to install things into the installation directory as opposed to installing into a virtualenv, run graalpy -m ensurepip --default-pip. It will install pip that works.

It seems odd to me that support for torch and scipy is mentioned as a long term goal in the readme, but there is no description of what an early experimenter should expect if they try and install either of these. I'd like to help out with the project in some way (at least through experimentation), but I don't know where to find a basic summary of the current expected behavior.

We periodically run tests of many popular python packages and we track the pass rate of their upstream tests. torch usually passes 97% of those tests (although there is a recent regression on master to 67% at the moment), so you can generally expect that nearly everything works. I should update the readme that torch is generally "supported" already. scipy usually passes around 60% of tests, so most things should work too.

[ashesfall]

@msimacek I took all this advice, and I'm in a little bit better shape. It didn't work, but by also installing numpy 1.23.5 specifically suggested by @qunaibit I got past the failure I was seeing.

Now it complains that it can't install ninja, another dependency. Is there maybe a known compatible version of ninja to specify as well?

[msimacek]

ninja should just work in any version, it's a C tool repackaged as python wheel for convenience, it doesn't even use python. What error do you get? Also, what GraalPy version are you using?

[ashesfall]

@msimacek okay actually the ninja failure was just a problem with cmake I guess, and I updating my system resolved it. scipy installation is still running, and has been for a while, we'll see what happens there...

[ashesfall]

oh, now it finished.

with the following error

patching file tools/cythonize.py
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... error
  error: subprocess-exited-with-error
  
  × Preparing metadata (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [16 lines of output]
      + meson setup --prefix=/home/michael-ubuntu/.jdks/graalvm-jdk-17.0.8+9.1/languages/python /tmp/pip-install-nx6xflt2/scipy_7dd434fd79524175bf20590c4ef3a7ed /tmp/pip-install-nx6xflt2/scipy_7dd434fd79524175bf20590c4ef3a7ed/.mesonpy-yzm3xsku/build --native-file=/tmp/pip-install-nx6xflt2/scipy_7dd434fd79524175bf20590c4ef3a7ed/.mesonpy-native-file.ini -Ddebug=false -Doptimization=2
      The Meson build system
      Version: 1.0.0
      Source dir: /tmp/pip-install-nx6xflt2/scipy_7dd434fd79524175bf20590c4ef3a7ed
      Build dir: /tmp/pip-install-nx6xflt2/scipy_7dd434fd79524175bf20590c4ef3a7ed/.mesonpy-yzm3xsku/build
      Build type: native build
      Project name: SciPy
      Project version: 1.10.1
      C compiler for the host machine: cc (gcc 11.4.0 "cc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0")
      C linker for the host machine: cc ld.bfd 2.38
      C++ compiler for the host machine: c++ (gcc 11.4.0 "c++ (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0")
      C++ linker for the host machine: c++ ld.bfd 2.38
      
      ../../meson.build:1:0: ERROR: Cython compiler 'cython' cannot compile programs
      
      A full log can be found at /tmp/pip-install-nx6xflt2/scipy_7dd434fd79524175bf20590c4ef3a7ed/.mesonpy-yzm3xsku/build/meson-logs/meson-log.txt
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

[msimacek]

Hmm, I've never seen that error. Could you please try with graalpy 23.1.0?

[ashesfall]

Switching from 23.0.1 to 23.1.0 and repeating the steps results in some compilation failures. A subset of that is below, I can capture and post the entire output in another github gist if it would be useful.

      Warning: Fortran 2018 deleted feature: DO termination statement which is not END DO or CONTINUE with label 20 at (1)
      [251/1524] Linking static target scipy/special/libspecfun.a
      [252/1524] Generating scipy/special/specfun_module with a custom command
      [253/1524] Compiling C++ object scipy/special/_ufuncs_cxx.graalpy231-310-native-x86_64-linux.so.p/Faddeeva.cc.o
      [254/1524] Compiling C++ object scipy/special/_ufuncs_cxx.graalpy231-310-native-x86_64-linux.so.p/sf_error.cc.o
      [255/1524] Compiling C object scipy/special/_specfun.graalpy231-310-native-x86_64-linux.so.p/meson-generated_..__specfunmodule.c.o
      [256/1524] Linking target scipy/special/_specfun.graalpy231-310-native-x86_64-linux.so
      [257/1524] Compiling C++ object scipy/special/_ufuncs_cxx.graalpy231-310-native-x86_64-linux.so.p/wright.cc.o
      [258/1524] Compiling C object scipy/special/_ellip_harm_2.graalpy231-310-native-x86_64-linux.so.p/sf_error.c.o
      [259/1524] Generating 'scipy/special/_ufuncs_cxx.graalpy231-310-native-x86_64-linux.so.p/_ufuncs_cxx.cpp'.
      WARNING: Overriding pythran description with argspec information for: numpy.random.bytes
      [260/1524] Compiling C++ object scipy/special/_ufuncs_cxx.graalpy231-310-native-x86_64-linux.so.p/meson-generated__ufuncs_cxx.cpp.o
      [261/1524] Linking target scipy/special/_ufuncs_cxx.graalpy231-310-native-x86_64-linux.so
      [262/1524] Compiling C object scipy/special/cython_special.graalpy231-310-native-x86_64-linux.so.p/_cosine.c.o
      [263/1524] Compiling C object scipy/special/cython_special.graalpy231-310-native-x86_64-linux.so.p/amos_wrappers.c.o
      [264/1524] Compiling C object scipy/special/cython_special.graalpy231-310-native-x86_64-linux.so.p/cdf_wrappers.c.o
      [265/1524] Compiling C object scipy/special/cython_special.graalpy231-310-native-x86_64-linux.so.p/specfun_wrappers.c.o
      [266/1524] Compiling C object scipy/special/cython_special.graalpy231-310-native-x86_64-linux.so.p/sf_error.c.o
      [267/1524] Generating 'scipy/special/_ellip_harm_2.graalpy231-310-native-x86_64-linux.so.p/_ellip_harm_2.c'.
      WARNING: Overriding pythran description with argspec information for: numpy.random.bytes
      [268/1524] Compiling C object scipy/special/_ellip_harm_2.graalpy231-310-native-x86_64-linux.so.p/meson-generated__ellip_harm_2.c.o
      [269/1524] Linking target scipy/special/_ellip_harm_2.graalpy231-310-native-x86_64-linux.so
      [270/1524] Generating 'scipy/special/_comb.graalpy231-310-native-x86_64-linux.so.p/_comb.c'.
      WARNING: Overriding pythran description with argspec information for: numpy.random.bytes
      [271/1524] Compiling C object scipy/special/_comb.graalpy231-310-native-x86_64-linux.so.p/meson-generated__comb.c.o
      [272/1524] Linking target scipy/special/_comb.graalpy231-310-native-x86_64-linux.so
      [273/1524] Compiling C object scipy/special/_test_internal.graalpy231-310-native-x86_64-linux.so.p/cephes_dd_real.c.o
      [274/1524] Generating scipy/linalg/__init__.py with a custom command
      [275/1524] Generating scipy/linalg/_cythonized_array_utils.pxd with a custom command
      [276/1524] Generating scipy/linalg/cython_linalg with a custom command
      [277/1524] Generating scipy/linalg/fblas_module with a custom command
      [278/1524] Compiling C object scipy/linalg/_fblas.graalpy231-310-native-x86_64-linux.so.p/meson-generated_..__fblasmodule.c.o
      [279/1524] Module scanner.
      [280/1524] Compiling Fortran object scipy/linalg/_fblas.graalpy231-310-native-x86_64-linux.so.p/meson-generated_..__fblas-f2pywrappers.f.o
      [281/1524] Compiling Fortran object scipy/linalg/_fblas.graalpy231-310-native-x86_64-linux.so.p/..__build_utils_src_wrap_dummy_g77_abi.f.o
      [282/1524] Linking target scipy/linalg/_fblas.graalpy231-310-native-x86_64-linux.so
      [283/1524] Generating 'scipy/special/_test_internal.graalpy231-310-native-x86_64-linux.so.p/_test_internal.c'.
      WARNING: Overriding pythran description with argspec information for: numpy.random.bytes
      [284/1524] Compiling C object scipy/special/_test_internal.graalpy231-310-native-x86_64-linux.so.p/meson-generated__test_internal.c.o
      [285/1524] Linking target scipy/special/_test_internal.graalpy231-310-native-x86_64-linux.so
      [286/1524] Generating scipy/linalg/flinalg_module with a custom command
      [287/1524] Compiling C object scipy/linalg/_flinalg.graalpy231-310-native-x86_64-linux.so.p/meson-generated_..__flinalgmodule.c.o
      [288/1524] Module scanner.
      [289/1524] Compiling Fortran object scipy/linalg/_flinalg.graalpy231-310-native-x86_64-linux.so.p/src_det.f.o
      [290/1524] Compiling Fortran object scipy/linalg/_flinalg.graalpy231-310-native-x86_64-linux.so.p/src_lu.f.o
      [291/1524] Linking target scipy/linalg/_flinalg.graalpy231-310-native-x86_64-linux.so
      [292/1524] Generating scipy/linalg/interpolative_module with a custom command
      [293/1524] Compiling C object scipy/linalg/_interpolative.graalpy231-310-native-x86_64-linux.so.p/meson-generated_..__interpolativemodule.c.o
      [294/1524] Module scanner.
      [295/1524] Compiling Fortran object scipy/linalg/_interpolative.graalpy231-310-native-x86_64-linux.so.p/src_id_dist_src_dfft.f.o
      FAILED: scipy/linalg/_interpolative.graalpy231-310-native-x86_64-linux.so.p/src_id_dist_src_dfft.f.o
      gfortran -Iscipy/linalg/_interpolative.graalpy231-310-native-x86_64-linux.so.p -Iscipy/linalg -I../scipy/linalg -I/tmp/pip-build-env-ywidgkh5/overlay/lib/python3.10/site-packages/numpy/core/include -I/tmp/pip-build-env-ywidgkh5/overlay/lib/python3.10/site-packages/numpy/f2py/src -Iscipy/lib_fortranobject.a.p -I/usr/include/x86_64-linux-gnu/openblas-pthread/ -I/home/michael-ubuntu/.pyenv/versions/graalpy-community-23.1.0/include/python3.10 -fvisibility=hidden -fdiagnostics-color=always -D_FILE_OFFSET_BITS=64 -Wall -O2 -Wno-conversion -fPIC -Wno-argument-mismatch -Wno-conversion -Wno-maybe-uninitialized -Wno-unused-dummy-argument -Wno-unused-label -Wno-unused-variable -Wno-tabs -w -Jscipy/linalg/_interpolative.graalpy231-310-native-x86_64-linux.so.p -o scipy/linalg/_interpolative.graalpy231-310-native-x86_64-linux.so.p/src_id_dist_src_dfft.f.o -c ../scipy/linalg/src/id_dist/src/dfft.f
      ../scipy/linalg/src/id_dist/src/dfft.f:2902:72:
      
       2902 |       CALL DFFTI1 (N,WSAVE(N+1),WSAVE(2*N+1))
            |                                                                        1
      Error: Type mismatch in argument ‘ifac’ at (1); passed REAL(8) to INTEGER(4)
      ../scipy/linalg/src/id_dist/src/dfft.f:2835:72:
      
       2835 |       CALL DFFTF1 (N,R,WSAVE,WSAVE(N+1),WSAVE(2*N+1))
            |                                                                        1
      Error: Type mismatch in argument ‘ifac’ at (1); passed REAL(8) to INTEGER(4)
      ../scipy/linalg/src/id_dist/src/dfft.f:2765:72:
      
       2765 |       CALL DFFTB1 (N,R,WSAVE,WSAVE(N+1),WSAVE(2*N+1))
            |                                                                        1
      Error: Type mismatch in argument ‘ifac’ at (1); passed REAL(8) to INTEGER(4)
      ../scipy/linalg/src/id_dist/src/dfft.f:939:72:
      
        939 |       CALL ZFFTB1 (N,C,WSAVE,WSAVE(IW1),WSAVE(IW2))
            |                                                                        1
      Error: Type mismatch in argument ‘ifac’ at (1); passed REAL(8) to INTEGER(4)
      ../scipy/linalg/src/id_dist/src/dfft.f:1013:72:
      
       1013 |       CALL ZFFTF1 (N,C,WSAVE,WSAVE(IW1),WSAVE(IW2))
            |                                                                        1
      Error: Type mismatch in argument ‘ifac’ at (1); passed REAL(8) to INTEGER(4)
      ../scipy/linalg/src/id_dist/src/dfft.f:1086:72:
      
       1086 |       CALL ZFFTI1 (N,WSAVE(IW1),WSAVE(IW2))
            |                                                                        1
      Error: Type mismatch in argument ‘ifac’ at (1); passed REAL(8) to INTEGER(4)
      ../scipy/linalg/src/id_dist/src/dfft.f:1365:72:
      
       1365 |       CALL DZFFT1 (N,WSAVE(2*N+1),WSAVE(3*N+1))
            |                                                                        1
      Error: Type mismatch in argument ‘ifac’ at (1); passed REAL(8) to INTEGER(4)
      ../scipy/linalg/src/id_dist/src/dfft.f:2997:72:
      
       2997 |       CALL DSINT1(N,X,WSAVE,WSAVE(IW1),WSAVE(IW2),WSAVE(IW3))
            |                                                                        1
      Error: Type mismatch in argument ‘ifac’ at (1); passed REAL(8) to INTEGER(4)
      [296/1524] Generating 'scipy/special/cython_special.graalpy231-310-native-x86_64-linux.so.p/cython_special.c'.
      WARNING: Overriding pythran description with argspec information for: numpy.random.bytes
      [297/1524] Generating scipy/linalg/flapack_module with a custom command
      [298/1524] Generating 'scipy/special/_ufuncs.graalpy231-310-native-x86_64-linux.so.p/_ufuncs.c'.
      WARNING: Overriding pythran description with argspec information for: numpy.random.bytes
      ninja: build stopped: subcommand failed.
      [end of output]

[timfel]

scipy's code is incompatible with newer gfortran releases. You will have to export FFLAGS=-fallow-argument-mismatch to build it with current versions of gfortran.

[msimacek]

I'll check if we could set that flag automatically in our patch (backport the check from current scipy).

[ashesfall]

@timfel this variable did in fact fix it

now its installed, and I'll do some experimentation with it and see how it goes.

@msimacek it would be great if there was (1) a document somewhere with this tribal knowledge useful for things to build/install and (2) a docker image with things like scipy, torch, etc that are targeted for support so that there's a quick way to replicate that environment without having to know any workarounds.

If either of these exist, maybe the readme can be updated to point at them. Otherwise though, I'm very excited to try this out. Thanks for your help.

[msimacek]

I finally got around to update the scipy patch so that meson should set the flag automatically when it's available, so this problem shouldn't happen anymore.