Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

BUG: optimize: COBYLA hangs on some CPUs #15527

Open
apixandru opened this issue Feb 4, 2022 · 6 comments
Open

BUG: optimize: COBYLA hangs on some CPUs #15527

apixandru opened this issue Feb 4, 2022 · 6 comments
Labels
defect A clear bug or issue that prevents SciPy from being installed or used as expected scipy.optimize

Comments

@apixandru
Copy link

apixandru commented Feb 4, 2022 •
edited
Loading

Describe your issue.

A couple of weeks back we have noticed that some of our workloads have been hanging.

I attached the code that reproduces the bug

Works with any consumer grade Intel CPUs that I tried. It only has problems on servers.
Works with any AMD machine that I tried (consumer or on servers)

Works with Intel Xeon Haswell CPUs E5-2676 v3 (Amazon AWS t2.micro instance or Google GCP N1 instance)

Does not work under any circumstances with Cascade Lake CPUs, for instance Xeon(R) Platinum 8252C CPU (Amazon AWS m5zn.large or Google CGP N2 / C2 with Cascade Lake)

Works (somewhat) with Intel Xeon Ice Lake Cpus, for instance Xeon(R) Platinum 8375C (Amazon AWS
m6i.large or Google GCP N2 with Ice Lake)

On Ice Lake it only works if you specify the numpy dependency as numpy==1.20.3 , if you let scipy download 1.22.2 then it will hang

On Cascade Lake it hangs regardless of the versions and on Haswell it works regardless of the versions.

Here's a sample docker file if you want deploy it as a docker

Reproduces with every version of scipy since 1.4.0

Sample Dockerfile (our original dockerfile does actually compile python from source but I used this as an example to keep it very simple, it reproduces this way too)

FROM ubuntu:20.04

ENV DEBIAN_FRONTEND=noninteractive

RUN apt-get update && apt-get install -y python3-pip

RUN mkdir /home/app
COPY main.py /home/app/
COPY requirements.txt /home/app/
RUN pip3 install -r /home/app/requirements.txt

CMD python3 /home/app/main.py

To reproduce on Ice Lake/Cascade Lake cpus
requirements.txt (pretty much any scipy version since 1.4.0)

scipy==1.7.0

To reproduce working on Ice Lake but hanging on Cascade Lake

numpy==1.20.3
scipy==1.7.0

I included the Dockerfile for convenience, the issue reproduces if i execute the python process in the VM without docker.

Reproducing Code Example

main.py

import numpy
import scipy.optimize

csx = [[0, 0, 1.0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0, 1.0, 0, 0, 0, 0, 0, 0, 0],
       [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1.0, 0], [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1.0, 0, 0],
       [0, 0, 0, 1.0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], [0, 1.0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
       [1.0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 1.0, 0, 0, 0, 0, 0, 0, 0, 0],
       [0, 0, 0, 0, 0, 0, 0, 0, 1.0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1.0],
       [0, 0, 0, 0, 1.0, 0, 0, 0, 0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0, 0, 0, 0, 1.0, 0, 0, 0, 0],
       [0, 0, 0, 0, 0, 0, 0, 1.0, 0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1.0, 0, 0, 0],
       [0, -1.0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, -1.0, 0, 0, 0, 0, 0, 0, 0, 0],
       [0, 0, 0, 0, 0, 0, 0, 0, -1.0, 0, 0, 0, 0, 0], [0, 0, 1.0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
       [0, 0, -1.0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0, 1.0, 0, 0, 0, 0, 0, 0, 0],
       [0, 0, 0, 0, 0, 0, -1.0, 0, 0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1.0, 0],
       [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, -1.0, 0], [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1.0, 0, 0],
       [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, -1.0, 0, 0], [0, 0, 0, 1.0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
       [0, 0, 0, -1.0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], [0, 1.0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
       [0, -1.0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], [1.0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
       [-1.0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], [0, 0, 1.0, -1.0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
       [0, 0, 1.0, 0, -1.0, 0, 0, 0, 0, 0, 0, 0, 0, 0], [0, 0, 1.0, 0, 0, -1.0, 0, 0, 0, 0, 0, 0, 0, 0],
       [0, 0, 1.0, 0, 0, 0, -1.0, 0, 0, 0, 0, 0, 0, 0], [0, 0, 1.0, 0, 0, 0, 0, -1.0, 0, 0, 0, 0, 0, 0],
       [0, 0, 1.0, 0, 0, 0, 0, 0, -1.0, 0, 0, 0, 0, 0], [0, 0, 1.0, 0, 0, 0, 0, 0, 0, -1.0, 0, 0, 0, 0],
       [0, 0, 1.0, 0, 0, 0, 0, 0, 0, 0, -1.0, 0, 0, 0], [0, 0, 1.0, 0, 0, 0, 0, 0, 0, 0, 0, -1.0, 0, 0],
       [0, 0, 1.0, 0, 0, 0, 0, 0, 0, 0, 0, 0, -1.0, 0], [0, 0, 1.0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, -1.0],
       [0, 0, 0, 0, 0, 0, -1.0, 0, 0, 0, 0, 0, 1.0, 0], [0, 0, 0, 0, 0, 0, -1.0, 0, 0, 0, 0, 1.0, 0, 0],
       [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1.0, -1.0, 0], [0, 0, 0, 1.0, 0, 0, -1.0, 0, 0, 0, 0, 0, 0, 0],
       [0, 0, 0, 1.0, 0, 0, 0, 0, 0, 0, 0, 0, -1.0, 0], [0, 1.0, 0, -1.0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
       [0, 1.0, 0, 0, -1.0, 0, 0, 0, 0, 0, 0, 0, 0, 0], [0, 1.0, 0, 0, 0, 0, -1.0, 0, 0, 0, 0, 0, 0, 0],
       [0, 1.0, 0, 0, 0, 0, 0, 0, 0, 0, 0, -1.0, 0, 0], [0, 1.0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, -1.0, 0],
       [0, 1.0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, -1.0], [1.0, 0, 0, 0, -1.0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
       [1.0, 0, 0, 0, 0, 0, -1.0, 0, 0, 0, 0, 0, 0, 0], [1.0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, -1.0, 0],
       [1.0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, -1.0], [0, 0, 0, -1.0, 0, 1.0, 0, 0, 0, 0, 0, 0, 0, 0],
       [0, 0, 0, 0, -1.0, 1.0, 0, 0, 0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 1.0, -1.0, 0, 0, 0, 0, 0, 0, 0],
       [0, 0, 0, 0, 0, 1.0, 0, 0, 0, 0, 0, -1.0, 0, 0], [0, 0, 0, 0, 0, 1.0, 0, 0, 0, 0, 0, 0, -1.0, 0],
       [0, 0, 0, 0, 0, 1.0, 0, 0, 0, 0, 0, 0, 0, -1.0], [0, 0, 0, -1.0, 0, 0, 0, 0, 1.0, 0, 0, 0, 0, 0],
       [0, 0, 0, 0, -1.0, 0, 0, 0, 1.0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0, -1.0, 0, 1.0, 0, 0, 0, 0, 0],
       [0, 0, 0, 0, 0, 0, 0, 0, 1.0, 0, 0, -1.0, 0, 0], [0, 0, 0, 0, 0, 0, 0, 0, 1.0, 0, 0, 0, -1.0, 0],
       [0, 0, 0, 0, 0, 0, 0, 0, 1.0, 0, 0, 0, 0, -1.0], [0, 0, 0, 0, 0, 0, -1.0, 0, 0, 0, 0, 0, 0, 1.0],
       [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, -1.0, 1.0], [0, 0, 0, 0, 1.0, 0, -1.0, 0, 0, 0, 0, 0, 0, 0],
       [0, 0, 0, 0, 1.0, 0, 0, 0, 0, 0, 0, 0, -1.0, 0], [0, 0, 0, -1.0, 0, 0, 0, 0, 0, 1.0, 0, 0, 0, 0],
       [0, 0, 0, 0, -1.0, 0, 0, 0, 0, 1.0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0, -1.0, 0, 0, 1.0, 0, 0, 0, 0],
       [0, 0, 0, 0, 0, 0, 0, 0, 0, 1.0, 0, -1.0, 0, 0], [0, 0, 0, 0, 0, 0, 0, 0, 0, 1.0, 0, 0, -1.0, 0],
       [0, 0, 0, 0, 0, 0, 0, 0, 0, 1.0, 0, 0, 0, -1.0], [0, 0, 0, -1.0, 0, 0, 0, 1.0, 0, 0, 0, 0, 0, 0],
       [0, 0, 0, 0, -1.0, 0, 0, 1.0, 0, 0, 0, 0, 0, 0], [0, 0, 0, 0, 0, 0, -1.0, 1.0, 0, 0, 0, 0, 0, 0],
       [0, 0, 0, 0, 0, 0, 0, 1.0, 0, 0, 0, -1.0, 0, 0], [0, 0, 0, 0, 0, 0, 0, 1.0, 0, 0, 0, 0, -1.0, 0],
       [0, 0, 0, 0, 0, 0, 0, 1.0, 0, 0, 0, 0, 0, -1.0], [0, 0, 0, -1.0, 0, 0, 0, 0, 0, 0, 1.0, 0, 0, 0],
       [0, 0, 0, 0, -1.0, 0, 0, 0, 0, 0, 1.0, 0, 0, 0], [0, 0, 0, 0, 0, 0, -1.0, 0, 0, 0, 1.0, 0, 0, 0],
       [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1.0, -1.0, 0, 0], [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1.0, 0, -1.0, 0],
       [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1.0, 0, 0, -1.0], [0, 0, -0.0625, 0.18181818, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
       [0, 0, -0.0625, 0, 0.18181818, 0, 0, 0, 0, 0, 0, 0, 0, 0],
       [0, 0.08333333, -0.0625, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
       [0, 0, -0.0625, 0, 0, 0.09090909, 0, 0, 0, 0, 0, 0, 0, 0],
       [0, 0, -0.0625, 0, 0, 0, 1.33333333, 0, 0, 0, 0, 0, 0, 0],
       [0, 0, -0.0625, 0, 0, 0, 0, 0.09090909, 0, 0, 0, 0, 0, 0],
       [0, 0, -0.0625, 0, 0, 0, 0, 0, 0.09090909, 0, 0, 0, 0, 0],
       [0, 0, -0.0625, 0, 0, 0, 0, 0, 0, 0.09090909, 0, 0, 0, 0],
       [0, 0, -0.0625, 0, 0, 0, 0, 0, 0, 0, 0.08333333, 0, 0, 0],
       [0, 0, -0.0625, 0, 0, 0, 0, 0, 0, 0, 0, 0.18181818, 0, 0], [0, 0, -0.0625, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1.0, 0],
       [0, 0, -0.0625, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.18181818],
       [0, 0, 0, 0, 0, 0, 1.33333333, 0, 0, 0, 0, 0, -1.0, 0],
       [0, 0, 0, 0, 0, 0, 1.33333333, 0, 0, 0, 0, -0.18181818, 0, 0],
       [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, -0.18181818, 1.0, 0],
       [0, 0, 0, -0.18181818, 0, 0, 1.33333333, 0, 0, 0, 0, 0, 0, 0],
       [0, 0, 0, -0.18181818, 0, 0, 0, 0, 0, 0, 0, 0, 1.0, 0],
       [0, -0.08333333, 0, 0.18181818, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
       [0, -0.08333333, 0, 0, 0, 0, 1.33333333, 0, 0, 0, 0, 0, 0, 0],
       [0, -0.08333333, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.18181818, 0, 0],
       [0, -0.08333333, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1.0, 0],
       [-0.07142857, 0, 0, 0.18181818, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
       [-0.07142857, 0, 0, 0, 0.18181818, 0, 0, 0, 0, 0, 0, 0, 0, 0],
       [-0.07142857, 0, 0, 0, 0, 0.09090909, 0, 0, 0, 0, 0, 0, 0, 0],
       [-0.07142857, 0, 0, 0, 0, 0, 1.33333333, 0, 0, 0, 0, 0, 0, 0],
       [-0.07142857, 0, 0, 0, 0, 0, 0, 0.09090909, 0, 0, 0, 0, 0, 0],
       [-0.07142857, 0, 0, 0, 0, 0, 0, 0, 0.09090909, 0, 0, 0, 0, 0],
       [-0.07142857, 0, 0, 0, 0, 0, 0, 0, 0, 0.09090909, 0, 0, 0, 0],
       [-0.07142857, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.18181818, 0, 0],
       [-0.07142857, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1.0, 0],
       [-0.07142857, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.18181818],
       [0, 0, 0, 0.18181818, 0, -0.09090909, 0, 0, 0, 0, 0, 0, 0, 0],
       [0, 0, 0, 0, 0.18181818, -0.09090909, 0, 0, 0, 0, 0, 0, 0, 0],
       [0, 0, 0, 0, 0, -0.09090909, 1.33333333, 0, 0, 0, 0, 0, 0, 0],
       [0, 0, 0, 0, 0, -0.09090909, 0, 0, 0, 0, 0, 0.18181818, 0, 0],
       [0, 0, 0, 0, 0, -0.09090909, 0, 0, 0, 0, 0, 0, 1.0, 0],
       [0, 0, 0, 0, 0, -0.09090909, 0, 0, 0, 0, 0, 0, 0, 0.18181818],
       [0, 0, 0, 0.18181818, 0, 0, 0, 0, -0.09090909, 0, 0, 0, 0, 0],
       [0, 0, 0, 0, 0.18181818, 0, 0, 0, -0.09090909, 0, 0, 0, 0, 0],
       [0, 0, 0, 0, 0, 0, 1.33333333, 0, -0.09090909, 0, 0, 0, 0, 0],
       [0, 0, 0, 0, 0, 0, 0, 0, -0.09090909, 0, 0, 0.18181818, 0, 0],
       [0, 0, 0, 0, 0, 0, 0, 0, -0.09090909, 0, 0, 0, 1.0, 0],
       [0, 0, 0, 0, 0, 0, 0, 0, -0.09090909, 0, 0, 0, 0, 0.18181818],
       [0, 0, 0, 0, 0, 0, 1.33333333, 0, 0, 0, 0, 0, 0, -0.18181818],
       [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1.0, -0.18181818],
       [0, 0, 0, 0, -0.18181818, 0, 1.33333333, 0, 0, 0, 0, 0, 0, 0],
       [0, 0, 0, 0, -0.18181818, 0, 0, 0, 0, 0, 0, 0, 1.0, 0],
       [0, 0, 0, 0.18181818, 0, 0, 0, 0, 0, -0.09090909, 0, 0, 0, 0],
       [0, 0, 0, 0, 0.18181818, 0, 0, 0, 0, -0.09090909, 0, 0, 0, 0],
       [0, 0, 0, 0, 0, 0, 1.33333333, 0, 0, -0.09090909, 0, 0, 0, 0],
       [0, 0, 0, 0, 0, 0, 0, 0, 0, -0.09090909, 0, 0.18181818, 0, 0],
       [0, 0, 0, 0, 0, 0, 0, 0, 0, -0.09090909, 0, 0, 1.0, 0],
       [0, 0, 0, 0, 0, 0, 0, 0, 0, -0.09090909, 0, 0, 0, 0.18181818],
       [0, 0, 0, 0.18181818, 0, 0, 0, -0.09090909, 0, 0, 0, 0, 0, 0],
       [0, 0, 0, 0, 0.18181818, 0, 0, -0.09090909, 0, 0, 0, 0, 0, 0],
       [0, 0, 0, 0, 0, 0, 1.33333333, -0.09090909, 0, 0, 0, 0, 0, 0],
       [0, 0, 0, 0, 0, 0, 0, -0.09090909, 0, 0, 0, 0.18181818, 0, 0],
       [0, 0, 0, 0, 0, 0, 0, -0.09090909, 0, 0, 0, 0, 1.0, 0],
       [0, 0, 0, 0, 0, 0, 0, -0.09090909, 0, 0, 0, 0, 0, 0.18181818],
       [0, 0, 0, 0.18181818, 0, 0, 0, 0, 0, 0, -0.08333333, 0, 0, 0],
       [0, 0, 0, 0, 0.18181818, 0, 0, 0, 0, 0, -0.08333333, 0, 0, 0],
       [0, 0, 0, 0, 0, 0, 1.33333333, 0, 0, 0, -0.08333333, 0, 0, 0],
       [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, -0.08333333, 0.18181818, 0, 0],
       [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, -0.08333333, 0, 1.0, 0],
       [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, -0.08333333, 0, 0, 0.18181818]]

csy = [5.72, 0.63, 0.49, 4.39, 4.36, 8.8, 2.86, 8.78, 8.78, 4.39, 4.39, 8.78, 8.78, 8.8, -14.91, -14.91, -14.91, 8.99,
       -8.99, 0.99, -0.99, 1.29, -1.29, 6.99, -6.99, 6.94, -6.94, 11.99, -11.99, 4.99, -4.99, 0.05, 0.05, 0.05, 0.05,
       0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05,
       0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05,
       0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.05, 0.0,
       0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,
       0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,
       0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]


def constraint2(ix, iy):
    return lambda x: numpy.dot(ix, x) - iy


def minimize():
    def fun(x):
        index_ = -(x[6] ** 2)
        print(index_)
        return index_

    constraints = []
    for ix, iy in zip(csx, csy):
        constraints.append({'type': 'ineq', 'fun': constraint2(ix, iy)})

    x0 = [4.99, 11.99, 8.99, 6.94, 6.94, 11.99, 0.99, 11.99, 11.99, 11.99, 11.99, 6.99, 1.29, 6.94]
    scipy.optimize.minimize(fun, x0, constraints=constraints, method='COBYLA')

if __name__ == '__main__':
    print('starting')
    minimize()
    print('finished')

Error message

There is no error, the process just hangs and the shell that started it becomes completely unusable, there appears to be some sort of access violation and the whole process is corrupt.

Ice Lake example (It hangs and no amount of Ctrl+C will you get you out of this one, that shell session is dead)
image

Haswell example (It completes successfully)
image

SciPy/NumPy/Python version information

1.7.3 1.22.2 sys.version_info(major=3, minor=8, micro=10, releaselevel='final', serial=0)
@apixandru apixandru added the defect A clear bug or issue that prevents SciPy from being installed or used as expected label Feb 4, 2022
@apixandru apixandru changed the title BUG: Scipy/COBYLA hangs on some CPUs BUG: Scipy.optimize/COBYLA hangs on some CPUs Feb 5, 2022
@apixandru apixandru changed the title BUG: Scipy.optimize/COBYLA hangs on some CPUs BUG: Scipy.optimize / COBYLA hangs on some CPUs Feb 5, 2022
@tupui
Copy link
Member

tupui commented Feb 24, 2022 •
edited
Loading

Hi @apixandru, thanks for reporting. Did you try the latest release of SciPy? The development version?

Also please remove the zip file, it's a security issue.

@apixandru
Copy link
Author

@tupui 1.8.0 still hangs but i'm not sure how to install The development version.

I'm getting
FileNotFoundError: [Errno 2] No such file or directory: 'scipy/_lib/highs/CMakeLists.txt'

The _lib/highs folder is empty
I tried a bunch of installation methods and haven't had any luck with any
I tried creating a venv from environment.yml and got the above error
I tried installing dependencies then running python setup install, same error and same erorr with pip install .

@tupui
Copy link
Member

tupui commented Feb 24, 2022

@tupui 1.8.0 still hangs but i'm not sure how to install The development version.

I'm getting FileNotFoundError: [Errno 2] No such file or directory: 'scipy/_lib/highs/CMakeLists.txt'

The _lib/highs folder is empty I tried a bunch of installation methods and haven't had any luck with any I tried creating a venv from environment.yml and got the above error I tried installing dependencies then running python setup install, same error and same erorr with pip install .

Installing a dev environment is more complicated than this. You have to follow a guide from our doc: https://scipy.github.io/devdocs/dev/contributor/contributor_toc.html#development-environment

Basically pip install . and setup.py do not work like this as we have some compilation steps to do.

@apixandru
Copy link
Author

@tupui yep, it's dead on dev too

1.9.0.dev0+1585.b529d4c 1.22.2 sys.version_info(major=3, minor=8, micro=10, releaselevel='final', serial=0)
starting
-0.9801
-0.9801
-0.9801
-0.9801
-0.9801
-0.9801
-0.9801
-3.9601
-3.9601
-3.9601
-3.9601
-3.9601
-3.9601
-3.9601
-3.9601
-3.9601
-3.9601000000000024
-3.9601000000000086
-3.3481877047226027
-2.6908098044504505
-2.13383000096217
-1.6413663265061476
-1.2134187810823855
-0.9800999999999964
-0.9800999999999966

@apixandru
Copy link
Author

I tried precomputing the results of all my numpy,.dot computations and having the constraints be just popping the value from the list and that worked

def constraint2(ix, iy): return lambda x: numpy.dot(ix, x) - iy

Then i tried running just the numpy.dot function in isolation for all the inputs that i'm sending here and that also worked

Seems like it has to be a combination of both for the issue to reproduce in certain cloud environments.

This was referenced Mar 9, 2023
Open
Open
Open
Closed
Open
This was referenced Sep 20, 2023
Open
Closed
@zaikunzhang zaikunzhang mentioned this issue Oct 18, 2023
Open
This was referenced Jan 7, 2024
Open
Open
@alesandru
Copy link

alesandru commented Jul 26, 2024 •
edited
Loading

The issue still occurs even in the latest available version

Collecting numpy
  Downloading numpy-2.0.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (19.5 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 19.5/19.5 MB 43.3 MB/s eta 0:00:00
Collecting scipy
  Downloading scipy-1.14.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (41.1 MB)

starting
-0.9801
-0.9801
-0.9801
-0.9801
-0.9801
-0.9801
-0.9801
-3.9601
-3.9601
-3.9601
-3.9601
-3.9601
-3.9601
-3.9601
-3.9601
-3.9601
-3.9601000000000024
-3.9601000000000086
-3.3481877047226027
-2.6908098044504505
-2.13383000096217
-1.6413663265061476
-1.2134187810823855
-0.9800999999999964
-0.9800999999999966
^C^C^C^C^C^C

@lucascolley lucascolley changed the title BUG: Scipy.optimize / COBYLA hangs on some CPUs BUG: optimize: COBYLA hangs on some CPUs Jul 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
defect A clear bug or issue that prevents SciPy from being installed or used as expected scipy.optimize
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@AtsushiSakai @apixandru @tupui @alesandru