setup.py broke again

[MatthewRHermes]

See the thread in PR #18. I do not think the CI failures there have anything to do with the PR itself.

Experimentation suggests that the dependencies indicated by setup.py are, for some reason, now placed in a location that the sys.path inside of setup.py cannot see:

Requirement already satisfied: pyscf in /opt/hostedtoolcache/Python/3.7.16/x64/lib/python3.7/site-packages (from pyscf-forge==1.0.0) (2.2.1)
Requirement already satisfied: numpy in /opt/hostedtoolcache/Python/3.7.16/x64/lib/python3.7/site-packages (from pyscf-forge==1.0.0) (1.21.6)
Requirement already satisfied: scipy!=1.5.0,!=1.5.1 in /opt/hostedtoolcache/Python/3.7.16/x64/lib/python3.7/site-packages (from pyscf->pyscf-forge==1.0.0) (1.7.3)
Requirement already satisfied: h5py>=2.7 in /opt/hostedtoolcache/Python/3.7.16/x64/lib/python3.7/site-packages (from pyscf->pyscf-forge==1.0.0) (3.8.0)

whereas print (sys.path) inside of setup.py:CMakeBuildExt.build_cmake produces:

['/home/runner/work/pyscf-forge/pyscf-forge', '/opt/hostedtoolcache/Python/3.7.16/x64/lib/python3.7/site-packages/pip/_vendor/pyproject_hooks/_in_process', '/tmp/pip-build-env-ofuuo5v2/site', '/opt/hostedtoolcache/Python/3.7.16/x64/lib/python37.zip', '/opt/hostedtoolcache/Python/3.7.16/x64/lib/python3.7', '/opt/hostedtoolcache/Python/3.7.16/x64/lib/python3.7/lib-dynload', '/tmp/pip-build-env-ofuuo5v2/overlay/lib/python3.7/site-packages', '/tmp/pip-build-env-ofuuo5v2/normal/lib/python3.7/site-packages']

I can maybe think of some horrible hodge-podge way to "fix" this in run_ci.sh, but really I would appreciate some help figuring out the "right" way to fix it.

[MatthewRHermes]

The utterly bizarre thing to me is that it's so close:

'/opt/hostedtoolcache/Python/3.7.16/x64/lib/python3.7', '/opt/hostedtoolcache/Python/3.7.16/x64/lib/python3.7/lib-dynload'

and a subdirectory of the right path,

'/opt/hostedtoolcache/Python/3.7.16/x64/lib/python3.7/site-packages/pip/_vendor/pyproject_hooks/_in_process'

and two other paths that end in python3.7/site-packages. But just not the right one.

[matthew-hennefarth]

It probably is not the best practice to have the folder of source files in pyscf-forge called pyscf as well. Typically python packages follow the <pkgname>/<pkgname> like in the PySCF repo (as in pyscf/pyscf). So for this repo it should be pyscf-forge/pyscf-forge, but obviously this will break a lot of code here and other peoples code since they expect that they can import from pyscf only.

[MatthewRHermes]

By examining the last successful CI run I think I figured it out: pip version >= 23.1 is no longer compatible with "the legacy 'setup.py install'" method. Setting a lower pip version seems to fix it. However, for the sake of users, we should probably come up with a fix that works for pip version >= 23.1. Probably related to this.

[MatthewRHermes]

It probably is not the best practice to have the folder of source files in pyscf-forge called pyscf as well. Typically python packages follow the <pkgname>/<pkgname> like in the PySCF repo (as in pyscf/pyscf). So for this repo it should be pyscf-forge/pyscf-forge, but obviously this will break a lot of code here and other peoples code since they expect that they can import from pyscf only.

My understanding is that this was deliberate, so that at some future time modules can be moved to the PySCF core modules with a minimum of rewriting and fuss

[matthew-hennefarth]

My understanding is that this was deliberate, so that at some future time modules can be moved to the PySCF core modules with a minimum of rewriting and fuss

In general that makes sense for this package.

[sunqm]

Thanks for tracking down the problem! Separating build dependency and install dependency is indeed necessary. I'll close this issue as PR #20 has fixed this problem.