build fail : missing probe directory, missing files in exe_dev

[iqphan]

On Ubuntu 22.04 in fresh directory:

svn --quiet --non-interactive --trust-server-cert export https://github.com/rlabduke/MolProbity.git/trunk/install_via_bootstrap.sh

Then:

./install_via_bootstrap.sh 2

Throws:
Processing: "/home/ubuntu/Downloads/molprobity_last_attempt/modules/probe/command_line"
Ignored: [Errno 2] No such file or directory: '/home/ubuntu/Downloads/molprobity_last_attempt/build/probe/exe/probe'
Ignored: [Errno 2] No such file or directory: '/home/ubuntu/Downloads/molprobity_last_attempt/build/probe/exe/probe'

then later:

BOOST_VERSION: 107000
libtbx.scons: implicit dependency scan disabled for directory /home/ubuntu/Downloads/molprobity_last_attempt/modules/boost
Ignored: [Errno 2] No such file or directory: '/home/ubuntu/Downloads/molprobity_last_attempt/build/exe_dev/cctbx.getting_started'
Ignored: [Errno 2] No such file or directory: '/home/ubuntu/Downloads/molprobity_last_attempt/build/exe_dev/cctbx.sym_equiv_sites'
Ignored: [Errno 2] No such file or directory: '/home/ubuntu/Downloads/molprobity_last_attempt/build/exe_dev/cctbx.convert_ccp4_symop_lib'
Ignored: [Errno 2] No such file or directory: '/home/ubuntu/Downloads/molprobity_last_attempt/build/exe_dev/cctbx.lattice_symmetry'
Ignored: [Errno 2] No such file or directory: '/home/ubuntu/Downloads/molprobity_last_attempt/build/exe_dev/cctbx.find_distances'
Ignored: [Errno 2] No such file or directory: '/home/ubuntu/Downloads/molprobity_last_attempt/build/exe_dev/cctbx.sys_abs_equiv_space_groups'
Ignored: [Errno 2] No such file or directory: '/home/ubuntu/Downloads/molprobity_last_attempt/build/exe_dev/cbflib.tst_memory'
scons: done reading SConscript files.

Installation then fails with error: ImportError: import("boost_python_meta_ext"): No module named boost_python_meta_ext

[chrissciwilliams]

Sorry for this difficulty. It looks like some of the (many) checkouts are failing. That's the best explanation I have at the moment for missing Reduce and/or Boost. Depending on your institution, this could be a firewall issue. Most of our checkouts invoke svn, which seems to be falling out of favor. Check with your IT folks if this seems like a likely problem. You can run the components of the bootstrap install separately to better monitor the checkouts. bootstrap.py --builder=molprobity hot bootstrap.py --builder=molprobity update bootstrap.py --builder=molprobity base bootstrap.py --builder=molprobity build Most of the checkouts occur in the update step. This includes Reduce. The base step installs most of the dependencies, including Python. I'm not sure which step the Boost installation occurs during currently. update and build can be rerun without using --force. To rerun base, you may have to use --force or delete the conda_base dir. I have never had to rerun hot. I will keep looking to see if I can better understand the Boost issue. Good luck, -Christopher Williams ---Richardson Lab, Duke University

…

On Mon, Nov 7, 2022 at 6:57 PM iqphan ***@***.***> wrote: On Ubuntu 22.04 in fresh directory: svn --quiet --non-interactive --trust-server-cert export https://github.com/rlabduke/MolProbity.git/trunk/install_via_bootstrap.sh Then: ./install_via_bootstrap.sh 2 Throws: Processing: "/home/ubuntu/Downloads/molprobity_last_attempt/modules/probe/command_line" Ignored: [Errno 2] No such file or directory: '/home/ubuntu/Downloads/molprobity_last_attempt/build/probe/exe/probe' Ignored: [Errno 2] No such file or directory: '/home/ubuntu/Downloads/molprobity_last_attempt/build/probe/exe/probe' then later: BOOST_VERSION: 107000 libtbx.scons: implicit dependency scan disabled for directory /home/ubuntu/Downloads/molprobity_last_attempt/modules/boost Ignored: [Errno 2] No such file or directory: '/home/ubuntu/Downloads/molprobity_last_attempt/build/exe_dev/cctbx.getting_started' Ignored: [Errno 2] No such file or directory: '/home/ubuntu/Downloads/molprobity_last_attempt/build/exe_dev/cctbx.sym_equiv_sites' Ignored: [Errno 2] No such file or directory: '/home/ubuntu/Downloads/molprobity_last_attempt/build/exe_dev/cctbx.convert_ccp4_symop_lib' Ignored: [Errno 2] No such file or directory: '/home/ubuntu/Downloads/molprobity_last_attempt/build/exe_dev/cctbx.lattice_symmetry' Ignored: [Errno 2] No such file or directory: '/home/ubuntu/Downloads/molprobity_last_attempt/build/exe_dev/cctbx.find_distances' Ignored: [Errno 2] No such file or directory: '/home/ubuntu/Downloads/molprobity_last_attempt/build/exe_dev/cctbx.sys_abs_equiv_space_groups' Ignored: [Errno 2] No such file or directory: '/home/ubuntu/Downloads/molprobity_last_attempt/build/exe_dev/cbflib.tst_memory' scons: done reading SConscript files. Installation then fails with error: ImportError: *import*("boost_python_meta_ext"): No module named boost_python_meta_ext — Reply to this email directly, view it on GitHub <#25>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACLEREEGUTXJZQRSVO7ZEALWHGJPXANCNFSM6AAAAAARZXJTIQ> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[iqphan]

I'm installing on an AWS instance and svn does not appear to be the issue.

hot and updated complete fine.

base fails on h5py:

• stack trace:

Installing h5py 2.10.0...
Read and write HDF5 files from Python
log file is /home/ubuntu/Downloads/MolProbity/base_tmp/h5py_install_log
Traceback (most recent call last):
File "/home/ubuntu/Downloads/MolProbity/modules/cctbx_project/libtbx/auto_build/install_base_packages.py", line 1678, in
installer(args=sys.argv, log=sys.stdout)
File "/home/ubuntu/Downloads/MolProbity/modules/cctbx_project/libtbx/auto_build/install_base_packages.py", line 257, in init
self.build_dependencies(packages=packages)
File "/home/ubuntu/Downloads/MolProbity/modules/cctbx_project/libtbx/auto_build/install_base_packages.py", line 778, in build_dependencies
getattr(self, 'build_%s'%i)()
File "/home/ubuntu/Downloads/MolProbity/modules/cctbx_project/libtbx/auto_build/install_base_packages.py", line 1214, in build_h5py
self.build_python_module_pip(
File "/home/ubuntu/Downloads/MolProbity/modules/cctbx_project/libtbx/auto_build/install_base_packages.py", line 688, in build_python_module_pip
self.call(pkg_info['python'] + ' -m pip install ' + pkg_info['debug'] +
File "/home/ubuntu/Downloads/MolProbity/modules/cctbx_project/libtbx/auto_build/install_base_packages.py", line 360, in call
return call(args, log=log, verbose=self.verbose, **kwargs)
File "/home/ubuntu/Downloads/MolProbity/modules/cctbx_project/libtbx/auto_build/installer_utils.py", line 55, in call
raise RuntimeError("Call to '%s' failed with exit code %d" % (args, rc))
RuntimeError: Call to '/home/ubuntu/Downloads/MolProbity/base/bin/python -m pip install "h5py==2.10.0" --no-binary=h5py' failed with exit code 1
Performing actions: base
Installing base packages using:
/home/ubuntu/Downloads/MolProbity/mc3/bin/python modules/cctbx_project/libtbx/auto_build/install_base_packages.py --python-shared --molprobity --nproc=1

removing .pyc files in /home/ubuntu/Downloads/MolProbity/modules, walk? True
removed 3 files
===== Running in .: base
Process failed with return code 1

• h5py_install_log :

Installing collected packages: h5py
Running setup.py install for h5py: started
Running setup.py install for h5py: finished with status 'error'
ERROR: Command errored out with exit status 1:
command: /home/ubuntu/Downloads/MolProbity/base/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-7YJjTU/h5py/setup.py'"'"'; file='"'"'/tmp/pip-install-7YJjTU/h5py/setup.
py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' install --record /tmp/pip-record-zrONXc/install-
record.txt --single-version-externally-managed --compile
cwd: /tmp/pip-install-7YJjTU/h5py/
Complete output (107 lines):
File "build/bdist.linux-x86_64/egg/pkgconfig/pkgconfig.py", line 281
flags = _query(packages, *os_opts, *_build_options(option, static=static))
^
SyntaxError: invalid syntax

zip_safe flag not set; analyzing archive contents...

[iqphan]

It looks like the h5py error is similar to the boost error: incompatible versions with Ubuntu 22.04.

This actually does not work :(
apt install python-h5py

... but I'm sure I can figure out how to install h5py on Ubuntu 22.04.

How do I modify the MolProbity installation scripts to pick up the OS-installed modules, instead of re-installing broken versions of those modules?

[iqphan]

lz4_plugin and tiff just won't install, I will try to build MolProbity without those 2 libs. Sorry for the multiple posts, I have no IT sysadmin support for this.

If anybody can point me to ubuntu binaries that would be immensely helpful. I only need the command line tools:

• reduce-build
• oneline-analysis

Alternatively, an API to query an existing MolProbity installation would work. We only need to run molprobity once a week with < 10 PDBs, the only requirement is that the process has to be automated.

[chrissciwilliams]

Sorry that I haven't been able to provide more direct help. And that we don't have binaries available.

I have a possible solution, if you are at an academic institution.

1. Get a build of Phenix from https://phenix-online.org/download and install it. That should contain all the dependencies and get the conda environment set up correctly. (The Phenix license is free to academics.)
2. cd to the level of the Phenix installation that contains build, modules, and conda_base.
3. "git clone https://github.com/rlabduke/MolProbity.git molprobity" at this location
4. cd molprobity; then ./setup.sh

And that's it. This method skips the whole "install_via_bootstrap" mess. It installs more than is necessary for MolProbity, but it works very reliably.

[iqphan]

Yes, we have a license for Phenix, this looks like a great solution, thanks!!

[iqphan]

In case it helps others: there's no need to fill your disk with Phenix. To compile MolProbity on Ubuntu 22.04 and at least get reduce-build and oneline-analysis to work, the steps below worked for me.

Note the switch between python3 and python2, this happened through trial and error.

# install dependencies listed in git repo REQUIREMENTS
cd Molprobity
# download installer
svn --quiet --non-interactive --trust-server-cert export https://github.com/rlabduke/MolProbity.git/trunk/install_via_bootstrap.sh

bash ./install_via_bootstrap.sh
# fails with error: modules/boost/boost/thread/pthread/thread_data.hpp:60:5: error: missing binary operator before token "("
# and: ImportError: import("boost_python_meta_ext"): No module named boost_python_meta_ext
# to get round that error:
### EDIT modules/boost/boost/thread/pthread/thread_data.hpp
### add these 2 lines after: #include <boost/config/abi_prefix.hpp>
#undef PTHREAD_STACK_MIN
#define PTHREAD_STACK_MIN 16384

# activate conda env: python3.9!
./mc3/bin/conda init bash
python bootstrap.py --builder=molprobity hot
python bootstrap.py --builder=molprobity update
# work around broken HDF5 install
# now calling python2.7! 
base/bin/python -m pip install  install versioned-hdf5
base/bin/python -m pip install  install h5py
# other pip install: docutils, psutil, mrcfile, pango, fonts
# FAILED pip install, found no work around : lz4_plugin, tiff, GTK2 (tried and failed: gtk-2.0 , gtk==2.0, gtk2.0, python-gtk2 )

# install
base/bin/python bootstrap.py --builder=molprobity base
base/bin/python bootstrap.py --builder=molprobity build

# command-line tools won't work if this is not called:
bash ./setup.sh 

# install java JDK: not listed in REQUIREMENTS, but still required!
sudo apt install default-jre
sudo apt install default-jdk

The build still complains about missing pango, gtk and fonts :( but command-line tools seem unaffected!
/home/ubuntu/Downloads/MolProbity/base/etc/pango not present, could not regenerate pango files
/home/ubuntu/Downloads/MolProbity/base/etc/gtk-2.0 not present, could not regenerate gdk-pixbuf.loaders
/home/ubuntu/Downloads/MolProbity/base/etc/fonts not present, could not rebuild fonts

The Phenix route is probably a better solution, if filling the disk with un-needed is not an issue ;-)

[raziel81]

I've been running into the same issue as well, tried pretty much everything written in the documentation and in this thread but the compilation won't go through. Tried it on an ubuntu 20.04 lts machine and an ubuntu 22.04 lts machine and i keep running back into this issue. Using php5.6 as recommended in previous posts (although this issue seems to appear also with php 8). Please advise

[amorehead]

It seems like I'm able to run install_via_bootstrap.sh successfully with https://anaconda.org/conda-forge/php at version 8.1.13. Not sure if anything has changed in the code recently that would allow the install script to be compatible with the latest version of PHP...