PsiEP 002

Building and Distributing Add-ons

Goals and Constraints

Binary must be as foolproof as possible. Techniques:
- static linking of MKL, esp. of non-Redistributables
- static linking of system libraries (libg++, libgcc, libc) where possible
- resolving symbols to reasonably old libc
- turning off processor-specific optimizations
Binary should be as full-featured, not lean-featured, wrt add-ons for users
Plugins should be accessible via the binary executable, to be helpful for new developers
Developers should have both efficient compile and full-featured build. (It does no good for developers to keep getting vexed at add-on build problems, disabling the add-on, and never seeing if it plays well with development code.)
Most of an add-on code should remain outside the Psi4 repo for the developer to maintain, enhance features, & interface elsewhere. Add-ons should have independent download counts.
As much as possible, developers and users should be using the same built code
When multiple approximately equal paths, follow the anaconda rut
Dynamically linking pieces we can ship with isn't nearly so bad as dynamically linking system/gcc/gxx/mkl libraries that may not be present or may conflict between native and shipped

psi4metapackage

contains proven helper packages for psi4

possible contents

  # vital- whole reason for anaconda binary in the first place
  - python 2.7*

  # documentation help
  - perl
  - sphinx

  # dependencies and add-ons
  - boost                 # pre-built core non-qc dependency
  - chemps2               # pre-built qc add-on (`.so` of interest)
  - pcmsolver             # pre-built qc add-on (`.so` of interest)

  # not sure how much raw building machinery want to bring in
  - gcc                   # compiling plugins c/o from binary in a predictable environment
  - cmake >3              # building psi4. handy if don't have recent enough version
  - openblas              # compiling plugins    

  # one of these two for MKL runtime libs for linking pre-built addon qc dependency
  - mkl-rt                # all mkl runtime libs. supplied by anaconda. we can't get in trouble for license agreements
  - iomp5                 # one mkl runtime lib we need. we control this package. have Y/n accept EULA agreement

  # gcc run-time libs. handy if we can suppress libgcc or libg++
  - libgcc                # gcc_s for linking pre-built addon qc dependency

  # these would be pulled in by the addons anyway, just being explicit here
  - zlib                  # non qc-dep for addons chemps2 & pcmsolver
  - gsl                   # non qc-dep for addon chemps2
  - hdf5                  # non qc-dep for addon chemps2

setup --meta-dir /path/to/conda/metapackage expands to --boost-include-dir, --boost-libs-dir, --zlib-dir, --chemps-dir, --pcmsolver-dir, SPHINX_ROOT, etc., as well as enabling all non-required packages
setup --meta-dir /path/to/conda/metapackage --chemps off expands to above while also disabling CheMPS2

Four Conda Use Cases

User
- one package supplies full-featured exe: conda install psi4
Core Developer
- supplies his own psi4 compilation (probably a git clone)
- one package supplies add-ons and dependencies (these are .so): conda install psi4metapackage
- own (or conda) compilers and blas, pre-built others: setup --cc icc ... --meta-dir /path/to/conda/metapackage
Add-on Developer
- writes a FindAddon.cmake and ConfigAddon.cmake in psi4
- implements both static and dynamic lib targets (easy in CMake)
- informs us so that add-on can be built from psinet (for static compatibility)
- either implements an orderable version system (so can build nightly) or informs us when time to rebuild package
Plugin Developer
- uses package psi4 and add-ons
- own compilers and blas (in which case only Makefile help is psi4-config) or conda compilers and blas (in which case Makefile is ready-to-go). see discussion at https://github.com/psi4/psi4public/issues/214 .

to Bake or Link

current (pre-1.0.0) psi4. no CheMPS2 or PCMSolver. glibc >= 2.7. minimal dependency status:

ldd bin/psi4 linux-vdso.so.1 => (0x00007fffc4dfe000) libutil.so.1 => /lib64/libutil.so.1 (0x00007f91a8400000) libm.so.6 => /lib64/libm.so.6 (0x00007f91a80fe000) librt.so.1 => /lib64/librt.so.1 (0x00007f91a7ef6000) libdl.so.2 => /lib64/libdl.so.2 (0x00007f91a7cf1000) libpython2.7.so.1.0 => /theoryfs2/ds/cdsgroup/psi4-install/miniconda/envs/p4bin/bin/../lib/libpython2.7.so.1.0 (0x00007f91a790e000) libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f91a76f2000) libc.so.6 => /lib64/libc.so.6 (0x00007f91a7330000) /lib64/ld-linux-x86-64.so.2 (0x00007f91a862a000) ```

State of chemps2 package
- iomp5 comes about b/c Intel doesn't want OpenMP linked in statically more than once (i.e., at executable formation time)
- gcc_s is dragged in b/c iomp5 present. I think it's inevitably present on a Linux distro, but just in case, can ship with libgcc package that provides old-libc gcc/g++ runtime libs

ldd -v lib/libchemps2.so.1 linux-vdso.so.1 => (0x00007fffb329a000) libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f1582018000) libc.so.6 => /lib64/libc.so.6 (0x00007f1581c57000) libiomp5.so => /theoryfs2/ds/loriab/miniconda3/envs/addons/lib/./libiomp5.so (0x00007f1581914000) libm.so.6 => /lib64/libm.so.6 (0x00007f1581611000) libgsl.so.0 => /theoryfs2/ds/loriab/miniconda3/envs/addons/lib/./libgsl.so.0 (0x00007f15811cf000) libgslcblas.so.0 => /theoryfs2/ds/loriab/miniconda3/envs/addons/lib/./libgslcblas.so.0 (0x00007f1580f90000) libhdf5.so.10 => /theoryfs2/ds/loriab/miniconda3/envs/addons/lib/./libhdf5.so.10 (0x00007f1580ad8000) libhdf5_hl.so.10 => /theoryfs2/ds/loriab/miniconda3/envs/addons/lib/./libhdf5_hl.so.10 (0x00007f15808ba000) libz.so.1 => /theoryfs2/ds/loriab/miniconda3/envs/addons/lib/./libz.so.1 (0x00007f15806a4000) librt.so.1 => /lib64/librt.so.1 (0x00007f158049b000) libdl.so.2 => /lib64/libdl.so.2 (0x00007f1580297000) /lib64/ld-linux-x86-64.so.2 (0x00007f1582da3000) libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f1580081000)


* State of `pcmsolver` package
  * couldn't ask for better dependency profile than this
  ```bash
>>> ldd -v lib/libpcm.so.1 
	linux-vdso.so.1 =>  (0x00007fffa4172000)
	libz.so.1 => /theoryfs2/ds/loriab/miniconda3/envs/addons/lib/./libz.so.1 (0x00007fe5c9f69000)
	libpthread.so.0 => /lib64/libpthread.so.0 (0x00007fe5c9d27000)
	libc.so.6 => /lib64/libc.so.6 (0x00007fe5c9966000)
	/lib64/ld-linux-x86-64.so.2 (0x00007fe5cbc72000)
	libdl.so.2 => /lib64/libdl.so.2 (0x00007fe5c9761000)

with chemps2 and pcmsolver baked in
- psi4 linking to libchemps2.a, etc.

cdsgroup@bash:psinet:/theoryfs2/ds/cdsgroup/psi4-compile/psi4rdr/psi4public/objdir3: ldd -v bin/psi4 linux-vdso.so.1 => (0x00007fff6ff52000) libutil.so.1 => /lib64/libutil.so.1 (0x00007f8ea0f94000) libm.so.6 => /lib64/libm.so.6 (0x00007f8ea0c92000) librt.so.1 => /lib64/librt.so.1 (0x00007f8ea0a8a000) libdl.so.2 => /lib64/libdl.so.2 (0x00007f8ea0885000) libpython2.7.so.1.0 => /theoryfs2/ds/cdsgroup/psi4-install/miniconda/envs/boostenv/lib/libpython2.7.so.1.0 (0x00007f8ea04a2000) libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f8ea0286000) libgsl.so.0 => /theoryfs2/ds/loriab/miniconda3/envs/addons/lib/libgsl.so.0 (0x00007f8e9fe43000) libgslcblas.so.0 => /theoryfs2/ds/loriab/miniconda3/envs/addons/lib/libgslcblas.so.0 (0x00007f8e9fc04000) libhdf5.so.10 => /theoryfs2/ds/loriab/miniconda3/envs/addons/lib/libhdf5.so.10 (0x00007f8e9f74d000) libhdf5_hl.so.10 => /theoryfs2/ds/loriab/miniconda3/envs/addons/lib/libhdf5_hl.so.10 (0x00007f8e9f52e000) libz.so.1 => /theoryfs2/ds/loriab/miniconda3/envs/addons/lib/libz.so.1 (0x00007f8e9f318000) libc.so.6 => /lib64/libc.so.6 (0x00007f8e9ef57000) /lib64/ld-linux-x86-64.so.2 (0x00007f8ea11be000)


* with `chemps2` and `pcmsolver` _linked_ in:
* psi4 linking to `libchemps2.so`, etc.
* `iomp5` and `gcc_s` mkl/system dependencies inevitably incurred
```bash
  linux-vdso.so.1 =>  (0x00007fff44723000)
  libutil.so.1 => /lib64/libutil.so.1 (0x00007f0a2cf65000)
  libm.so.6 => /lib64/libm.so.6 (0x00007f0a2cc63000)
  librt.so.1 => /lib64/librt.so.1 (0x00007f0a2ca5b000)
  libdl.so.2 => /lib64/libdl.so.2 (0x00007f0a2c856000)
  libpython2.7.so.1.0 => /theoryfs2/ds/cdsgroup/psi4-install/miniconda/envs/boostenv/lib/libpython2.7.so.1.0 (0x00007f0a2c473000)
  libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f0a2c257000)
  libchemps2.so.1 => /theoryfs2/ds/loriab/miniconda3/envs/addons/lib/libchemps2.so.1 (0x00007f0a2b70f000)
  libpcm.so.1 => /theoryfs2/ds/loriab/miniconda3/envs/addons/lib/libpcm.so.1 (0x00007f0a29c1e000)
  libgsl.so.0 => /theoryfs2/ds/loriab/miniconda3/envs/addons/lib/libgsl.so.0 (0x00007f0a297dc000)
  libgslcblas.so.0 => /theoryfs2/ds/loriab/miniconda3/envs/addons/lib/libgslcblas.so.0 (0x00007f0a2959c000)
  libhdf5.so.10 => /theoryfs2/ds/loriab/miniconda3/envs/addons/lib/libhdf5.so.10 (0x00007f0a290e5000)
  libhdf5_hl.so.10 => /theoryfs2/ds/loriab/miniconda3/envs/addons/lib/libhdf5_hl.so.10 (0x00007f0a28ec7000)
  libz.so.1 => /theoryfs2/ds/loriab/miniconda3/envs/addons/lib/libz.so.1 (0x00007f0a28cb0000)
  libc.so.6 => /lib64/libc.so.6 (0x00007f0a288ef000)
  /lib64/ld-linux-x86-64.so.2 (0x00007f0a2d18f000)
  libiomp5.so => /theoryfs2/ds/loriab/miniconda3/envs/addons/lib/./libiomp5.so (0x00007f0a285ab000)
  libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f0a28395000)

baked-in advantages
- except for ship-along dependencies like hdf5, no more dependent libraries than longstanding binary
- we get to declare ready-to-use add-ons
- we tie together compatible versions of add-on and psi4
linked-in advantages
- download count is true
- developer could distribute add-on w/o releasing source
- everyone (users and developers) running same library (rather than a vs so)
Misc. Generally, .a libraries aren't too portable. We can use them:
- w/i a compilation, b/c all same conditions
- when obtained from yum, apt-get, etc. b/c distros have compatilibity rules for this sort of thing
- when they shouldn't necessarily work (b/c from different conda packages) but do (b/c they're all from psinet)

Provide feedback

Saved searches