Issue with cmake during installation of pyscf

[vandan-revanur]

Currently if cmake is not installed locally, the installation of pyscf does not work and gives the following error.

To reproduce:

pip install pyscf

Error

Processing /Users/vandanrevanur/personal/codes/chemistry/pyscf
  Preparing metadata (setup.py) ... done
Requirement already satisfied: numpy!=1.16,!=1.17,>=1.13 in /Users/vandanrevanur/.virtualenvs/pyscf/lib/python3.10/site-packages (from pyscf==2.2.0) (1.24.2)
Requirement already satisfied: scipy!=1.5.0,!=1.5.1 in /Users/vandanrevanur/.virtualenvs/pyscf/lib/python3.10/site-packages (from pyscf==2.2.0) (1.10.1)
Requirement already satisfied: h5py>=2.7 in /Users/vandanrevanur/.virtualenvs/pyscf/lib/python3.10/site-packages (from pyscf==2.2.0) (3.8.0)
Building wheels for collected packages: pyscf
  Building wheel for pyscf (setup.py) ... error
  error: subprocess-exited-with-error
  
  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> [7 lines of output]
      scipy>1.1.0 may crash when calling scipy.linalg.eigh. (Issues https://github.com/scipy/scipy/issues/15362 https://github.com/scipy/scipy/issues/16151)
      running bdist_wheel
      running build
      running build_ext
      Configuring extensions
      cmake -S/Users/vandanrevanur/personal/codes/chemistry/pyscf/pyscf/lib -Bbuild/temp.macosx-12-x86_64-cpython-310
      error: command 'cmake' failed: No such file or directory
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for pyscf
  Running setup.py clean for pyscf
Failed to build pyscf
Installing collected packages: pyscf
  Running setup.py install for pyscf ... error
  error: subprocess-exited-with-error
  
  × Running setup.py install for pyscf did not run successfully.
  │ exit code: 1
  ╰─> [9 lines of output]
      scipy>1.1.0 may crash when calling scipy.linalg.eigh. (Issues https://github.com/scipy/scipy/issues/15362 https://github.com/scipy/scipy/issues/16151)
      running install
      /Users/vandanrevanur/.virtualenvs/pyscf/lib/python3.10/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
        warnings.warn(
      running build
      running build_ext
      Configuring extensions
      cmake -S/Users/vandanrevanur/personal/codes/chemistry/pyscf/pyscf/lib -Bbuild/temp.macosx-12-x86_64-cpython-310
      error: command 'cmake' failed: No such file or directory
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: legacy-install-failure

× Encountered error while trying to install package.
╰─> pyscf

note: This is an issue with the package mentioned above, not pip.
hint: See above for output from the failure.

[sunqm]

Is it a MacOS running on M1 chips? There is a macos wheel for x86 chips so pip install pyscf should install the precompiled wheel. If it is MaOS M1, pip install cmake can install cmake on your system.

[vandan-revanur]

My Mac is running on an Intel chip.
2,6 GHz 6-Core Intel Core i7

[jamesETsmith]

@vandan-revanur, I'm not sure why it's defaulting to building from sources, it should default to downloading the wheel. Can you tell us if the following runs successfully?

pip install --prefer-binary pyscf==2.2.0

[vandan-revanur]

@jamesETsmith That command works fine and it install pyscf.

[jamesETsmith]

Great to hear @vandan-revanur! Are you ok if I close the issue?

[vandan-revanur]

No @jamesETsmith that doesn’t solve the original issue. The installation should work with just pip install pyscf. Which still does not work. So it’s not right to close the issue.

[jamesETsmith]

Fair enough, my guess is that your version of pip could be older. A couple followup questions:

1. What version of pip are you working with?
2. Could you try running the following (you'll have to uninstall pyscf in between each command):

pip install pyscf==2.2.0
pip install pyscf==2.1.1
pip install pyscf==2.0.1

3. Can you reproduce this problem on other systems you have access to?

[vandan-revanur]

@jamesETsmith

I am using pip version: 23.0.1

pip install pyscf==2.2.0 : Works fine
pip install pyscf==2.1.1: Works fine
pip install pyscf==2.0.1: cmake error

[jamesETsmith]

@vandan-revanur thanks for the extra info!

It looks like things are working fine from those results. Does pip install pyscf still fail with the error in the original post? pip install pyscf==2.0.1 should build with cmake because there are no wheels for Python@3.10 in that release so it will always build from source. The fact that it fails is just a problem with your local setup and not necessarily pyscf.

[ndattani]

@jamesETsmith The reason why @vandan-revanur raised this issue is because several members of my lab struggled to install PySCF in their first attempt using pip install pyscf, and in many cases this was fixed by running pip install cmake before running pip install pyscf. We have a "pyscf" channel in our group Discord server, and the channel is full of people complaining about this.

Therefore the answer to:

"Can you reproduce this problem on other systems you have access to?"

is yes 😄

However, I was skeptical about the solution in #1685, because my procedure for installing/upgrading CMake on Ubuntu is not to run pip install cmake. The procedure I use is found here: https://askubuntu.com/a/865294/676348

So I'm not sure what the universal solution is, but perhaps if the website's installation instructions told Mac users to run pip install cmake before running pip install pyscf, this would at least have fixed the problems for half of my students last month. As a comment on #1685, I think Qiming said that pip install cmake could cause more problems because the PyPi installation requires other dependencies that not everyone will have. However I don't recall any of these students having trouble with running pip install cmake followed by pip install pyscf.

[jamesETsmith]

@ndattani, thanks for elaborating. Were your students all working in a consistent environment (i.e. on the same cluster) or were they working on their personal machines with a variety of python/pip/OS/architecture combinations?

The main problem here is that pip is trying to install PySCF from sources. I'm not totally sure how pip makes this decision and why it's choosing this for you and your students. With that said, the decision is happening inside pip and not something we can control (as far as I know). I'm really at a loss for why we're suddenly seeing this behavior from pip, I've never seen it before when working with PySCF and can't reproduce it on any systems I have access to.

I agree with @sunqm that we shouldn't add cmake to the list of python package dependencies because it's unnecessary for people downloading wheels.

@sunqm do you have any ideas about this?

[ndattani]

@jamesETsmith These were on personal machines rather than on a cluster. I think only Mac users experienced this, and even then it was not universal across all Mac users. I can double check the names of the people who complained about this error in the Discord server.

I'm also skeptical about the solution proposed in #1685, but maybe the website's installation instructions could say that if the user gets this error on a Mac, they can fix it by running "pip install cmake" first.

[ndattani]

Yizhen had the cmake problem on Windows.
Mia, Jack, Ahmad, and Sukrit had the cmake problem on MacOS.

Mia fixed it by doing:

pip install wheel
pip install pyscf

Most of the others fixed it by doing:

pip install cmake
pip install pyscf

[jamesETsmith]

@ndattani, thanks again for the detailed info. Since most people are working on individual machines my theory that this bizarre behavior is coming from older pip versions doesn't make sense. I agree that we should update the documentation, but I don't think we should recommend installing cmake and building from source. I think it would be best if we triaged the installation instructions like this:

1. Easiest method intended for most users:

pip install --prefer-binary pyscf

2. Building from source with git+pip
   This requires cmake, BLAS, several python packages (numpy, scipy, and h5py), and a working C compiler

pip install git+https://github.com/pyscf/pyscf

3. Developer builds
   Same requirements at 2) above.

git clone https://github.com/pyscf/pyscf.git
cd pyscf/ilb
cmake -B build <other_cmake_options>
cmake --build build --parallel 

We have this information already on the install instructions page, but I think reorganizing as I suggested above could make things clearer. We could also include the cmake error message in a "gotchas" section on the install page similar to what we do for users who experience the Library not loaded error. Would these changes be helpful?

@ndattani, just as a heads up, PySCF isn't supported on Windows so you may want to suggest to your students with Windows machines that they set up WSL. Alternatively, if they are only running small problems they can use Google Colab notebooks to run PySCF without any installation on their local machine, see the examples here.

[ndattani]

@jamesETsmith Thanks very much.

Perhaps pip install --prefer-binary pyscf is the best way for beginners to get started. I'll see if future students have problems with this approach. I agree with you that they don't need to be building from source if they're just using PySCF for some small calculations.

Your proposal for improved documentation seems like it would indeed be helpful to new users. One thing I'll add is that a lot of people had errors due to trying pip install pyscf[all], even though I never asked them to install the extension modules (I just asked them to visit https://pyscf.org/install.html and follow the instructions). If the website is going to change, perhaps it can also be made even more clear what the purpose of [all] is. The vast majority of users will not need [all] extension modules, and these just introduce more possibilities for the installation process to result in an error message, which then discourages new users.

As for Windows, indeed I've got all Windows users to install an Ubuntu VM. For some reason, WSL doesn't seem to work as well as it did when I had used it from around 2015-2017. Multiple students couldn't get things working smoothly in WSL and have things working perfectly fine in an Ubuntu VM (which defeats the purpose of WSL). I can ask them if they were using Windows 10 or 11, because @erikkjellgren has had a good experience with WSL2 on Windows 11.

[Raghav-Bell]

I have faced same issue while installing environment for qiskit textbook which uses pyscf==2.0.1 on Windows 11,64-bit, Anaconda virtual env
so far i have tried majority of fixes but nothing worked , following are the fixes and error that i encoutered :
pip install --prefer-binary pyscf or pip install pyscf or pip install git+https://github.com/pyscf/pyscf
ERROR: Could not build wheels for pyscf, which is required to install pyproject.toml-based projects.
pip install cmake or pip install wheel
pip install pyscf
ERROR: Could not build wheels for pyscf, which is required to install pyproject.toml-based projects
pip install –prefer-binary pyscf
ERROR: Invalid requirement: '–prefer-binary'
conda install -c pyscf pyscf or conda install pyscf or conda install -c conda-forge pyscf
PackagesNotFoundError: The following packages are not available from current channels:
pyscf..
pipwin install pyscf
Package pyscf not found
Please help me out to install pyscf.

[hebrewsnabla]

I have faced same issue while installing environment for qiskit textbook which uses pyscf==2.0.1 on Windows 11,64-bit, Anaconda virtual env so far i have tried majority of fixes but nothing worked , following are the fixes and error that i encoutered : pip install --prefer-binary pyscf or pip install pyscf or pip install git+https://github.com/pyscf/pyscf ERROR: Could not build wheels for pyscf, which is required to install pyproject.toml-based projects. pip install cmake or pip install wheel pip install pyscf ERROR: Could not build wheels for pyscf, which is required to install pyproject.toml-based projects pip install –prefer-binary pyscf ERROR: Invalid requirement: '–prefer-binary' conda install -c pyscf pyscf or conda install pyscf or conda install -c conda-forge pyscf PackagesNotFoundError: The following packages are not available from current channels: pyscf.. pipwin install pyscf Package pyscf not found Please help me out to install pyscf.

PySCF isn't supported on Windows

[verena-neufeld]

Hi @vandan-revanur , has this been resolved?

[vandan-revanur]

@verena-neufeld The latest version works with just : pip install pyscf.
So I think this issue is resolved.

[ndattani]

@verena-neufeld The latest version works with just : pip install pyscf.
So I think this issue is resolved.

The original problem in your first post, was that if cmake isn't installed, then "pip install pyscf" didn't work as expected. Now it works when cmake isn't installed?

[vandan-revanur]

@ndattani Yes it worked without installing cmake. I tried “pip install pyscf” on a fresh 3.10 virtual env and it worked without installing cmake.

[ndattani]

It then seems that everything is solved, so this issue can be closed.

[Marsmmz]

I have faced same issue while installing environment for qiskit textbook which uses pyscf==2.0.1 on Windows 11,64-bit, Anaconda virtual env so far i have tried majority of fixes but nothing worked , following are the fixes and error that i encoutered : pip install --prefer-binary pyscf or pip install pyscf or pip install git+https://github.com/pyscf/pyscf ERROR: Could not build wheels for pyscf, which is required to install pyproject.toml-based projects. pip install cmake or pip install wheel pip install pyscf ERROR: Could not build wheels for pyscf, which is required to install pyproject.toml-based projects pip install –prefer-binary pyscf ERROR: Invalid requirement: '–prefer-binary' conda install -c pyscf pyscf or conda install pyscf or conda install -c conda-forge pyscf PackagesNotFoundError: The following packages are not available from current channels: pyscf.. pipwin install pyscf Package pyscf not found Please help me out to install pyscf.

It seems that I'm encountering exactly the same issues here, has the installation problem been solved on windows? How can I install it with pip?

[ndattani]

@Marsmmz As explained in the issue #1857 that you created (which is now closed), PySCF is not supported on Windows.