build helper, take 2

[loriab]

Description

closes #3016 (supersedes)
closes #3084

User API & Changelog headlines

• For users, replaces the psi4-dev conda package that brought psi4 build dependencies and a "psi4-path-adivsor.pyscript that produced cmake configure lines. Now, that script is in-repo atconda/psi4-path-advisor.py` and running the help menu provides the following. This should be more configurable for folks who like to choose all conda or byo.

=========================================
  (A) black-box usage (copy/paste-able)
=========================================
# (1) get code from GitHub
git clone https://github.com/psi4/psi4.git && cd psi4
# (2) generate env spec file from codedeps.yaml. "eval $(...)" creates and activates conda env.
eval $(conda/psi4-path-advisor.py env)
# (3) generate cmake cache file from conda env. "eval $(...)" configures and builds with cmake.
eval $(conda/psi4-path-advisor.py cmake)
shows up in p4dev
=========================================
  (B) flexible usage
=========================================
# (1) get code from GitHub
git clone https://github.com/psi4/psi4.git && cd psi4
# (2.0) consider dependency options
conda/psi4-path-advisor.py env -h
# (2.1) generate env spec file from codedeps.yaml.
conda/psi4-path-advisor.py env -n p4dev310 --python 3.10 --disable addons --lapack openblas
#> conda env create -n p4dev310 -f /home/psi4/env_p4dev310.yaml && conda activate p4dev310
# (2.2) edit env_p4dev310.yaml to customize software packages.
# (2.3) issue suggested or customized command to create and activate conda env.
conda env create -n p4dev310 -f /home/psi4/env_p4dev310.yaml && conda activate p4dev310

• For packagers or other high-level what-does-it-need folks, dependency info is programmatically available now through codedeps.yaml

Dev notes & details

• add codedeps.yaml file that makes dependency and ecosystem information programmatically accessible. Presently useful for (1) people and (2) the in-repo psi4-path-advisor.py script, but more uses can follow.
• add in-repo conda/psi4-path-advisor.py script to serve as build helper. this is the main point of the PR. will elaborate more in future.
• there's a tidy new way of defining flags that are essential for the compiler with the compiler in CMAKE_<LANG>_COMPILER instead of cluttering up the user choice var CMAKE_<LANG>_FLAGS. This is especially relevant for intel compilers that need gcc passed. There's something that's not set up right yet for Intel Clang. Thus, we need to pass CMAKE_<LANG>_COMPILER_ARG1 through to psi4 subproject and all other externals.
• let ctest tests run whether old or new d3 and gcp implementations present.
• don't expect a pylibefpConfig.cmake for libefp detection. this is the case with the c-f package -- I'll probably put it back later when libefp/pylibefp repos combine.
• I was using inconsistent naming for basis_set_exchange detection/import, so fix that.
• adapt docs build so that it can use Ninja (or presumably any non-Makefiles) generator. The special thing about Makefiles is that it works as expected for glob depends. So we can tell the basis table to regenerate if any of the gbs files change. Other generators won't understand this. But most docs builds are one-off, not continuous development anyways, so it's worth no globs and yes flexibility. ADDED: switched to ninja for docs and docs-pr (ok memory-wise at max parallelism b/c debug build).

ADDED 12 Nov

• update ecosystem and docs GHA and linux azure to use the path-advisor env/cache procedure. this helps keep the path-advisore procedure tested and keeps testing envs up-to-date. alternate testing can still be imposed by seding the envs.
• in azure linux, use the three testing lanes to go beyond quicktests. apart from some longer tests, we're now hitting all but stdsuite non-quick.
• fixed the windows nightly build through azure, which was many layers of bugs. the improved path handling in psi4 & psi4/init.py from May technically didn't play nicely with the build-and-copy-into-place of the azure file cmake_install_*-paths-wise, and the recent (Oct) trick to get a fast psi4 --version tripped the problem, hence the sed.
• Strengthened the default to Release so that it works for windows, too. Ninja was setting Debug as-was.
• Updated the buildrun.yaml files from path-advisor
• Update the docs build for py312 by fixing the non-raw strings having backspace chars.
• removed the pb11=2.10 pin for docs build by adding to nitpick list
• Avoid all the warnings about INSIST_FIND_PACKAGE_ and _DIR unused user-spec variable warnings that could be unnerving.
• Use a different way of detecting libdl for plugins that doesn't throw cmake warnings. lmk if there's any problems.
• Patch over a sapt+windows+ecp bug described at SAPT and ECP and Windows #3072 so can test complete ecosystem
• Add a plain psi4 --psiapi variation on psi4 --psiapi-path that can be used in eval $(psi4 --psiapi)

Checklist

• Tests added for any new features
• All or relevant fraction of full tests run

Status

• Ready for review
• Ready for merge

Many revisions since original approval