octopus-0.5.2-beta does not compile, Centos 6.9

[jbsysadmin]

Greetings. We can't get this build process to work. Version octopus-0.5.2-beta. At milestone "[ 26%]", the output includes many of these errors:

/usr/local/boost_1_68_0/include/boost/container/detail/flat_tree.hpp:924:25: error: dereferencing type-punned pointer will break strict-aliasing rules [-Werror=strict-aliasing]
value_type &val = *static_cast<value_type *>(static_cast<void *>(v.data));

The environment includes source builds gcc-6.4.0, binutils 2.29.1, htslib 1.8, python 3.6.1.
The build call resembled: ./scripts/install.py --cxx_compiler /usr/local/gcc-6.4.0/bin/gcc --prefix /usr/local/octopus-0.5.2-beta/bin

Do you have any leads?

[dancooke]

Hi, thanks for your interest in Octopus. Please could you confirm how Boost was installed? It would also be helpful if you could post the entire output from your installation attempt.

[jbsysadmin]

Thanks for writing. I tried against after building gcc 8.2. I got further than the snippet I sent you before. Prior progress stopped at 26%. This time: [ 41%] Building CXX object src/CMakeFiles/octopus.dir/core/types/calls/denovo_reference_reversion_call.cpp.o /usr/local/installs/octopus/src/utils/genotype_reader.cpp: In function ‘octopus::Genotype<octopus::Haplotype> octopus::{anonymous}::extract_genotype(const std::vector<octopus::{anonymous}::CallWrapper>&, const octopus::GenomicRegion&, const SampleName&, const octopus::ReferenceGenome&)’: /usr/local/installs/octopus/src/utils/genotype_reader.cpp:144:26: error: ‘*((void*)& max_ref_pad +4)’ may be used uninitialized in this function [-Werror=maybe-uninitialized] boost::optional<int> max_ref_pad {}; Boost 1.68 was installed with just the items available in a vanilla Centos 6.9 install: ./bootstrap.sh --prefix=/path_to_boost_1_68_0 ./b2 ./b2 install ( I did try a boost compilation using gcc 8.2. However, I abandonned that when the octopus build stopped at 30%, a step back. ) From: Daniel Cooke <notifications@github.com> Sent: Thursday, December 13, 2018 10:13 AM To: luntergroup/octopus <octopus@noreply.github.com> Cc: Burns, W. J. <wjb2130@cumc.columbia.edu>; Author <author@noreply.github.com> Subject: Re: [luntergroup/octopus] octopus-0.5.2-beta does not compile, Centos 6.9 (#43) Hi, thanks for your interest in Octopus. Please could you confirm how Boost was installed? It would also be helpful if you could post the entire output from your installation attempt.

[jbsysadmin]

Hello again. I've continued the build attempts. But no matter what, given boost 1.68 and gcc 8.2, Octopus fails in the exact same place: 41%, just after "Building CXX object src/CMakeFiles/octopus.dir/core/types/calls/denovo_reference_reversion_call.cpp.o" See enclosed snapshot. If there was anything special that needed to be done during boost compile, that should be documented and added to your install instructions. This time, I did build boost with gcc 8.2 as well. Please let us know what you find.

…

-John From: Burns, W. J. Sent: Thursday, December 13, 2018 12:09 PM To: 'luntergroup/octopus' <reply@reply.github.com>; luntergroup/octopus <octopus@noreply.github.com> Cc: Author <author@noreply.github.com> Subject: RE: [luntergroup/octopus] octopus-0.5.2-beta does not compile, Centos 6.9 (#43) Thanks for writing. I tried against after building gcc 8.2. I got further than the snippet I sent you before. Prior progress stopped at 26%. This time: [ 41%] Building CXX object src/CMakeFiles/octopus.dir/core/types/calls/denovo_reference_reversion_call.cpp.o /usr/local/installs/octopus/src/utils/genotype_reader.cpp: In function ‘octopus::Genotype<octopus::Haplotype> octopus::{anonymous}::extract_genotype(const std::vector<octopus::{anonymous}::CallWrapper>&, const octopus::GenomicRegion&, const SampleName&, const octopus::ReferenceGenome&)’: /usr/local/installs/octopus/src/utils/genotype_reader.cpp:144:26: error: ‘*((void*)& max_ref_pad +4)’ may be used uninitialized in this function [-Werror=maybe-uninitialized] boost::optional<int> max_ref_pad {}; Boost 1.68 was installed with just the items available in a vanilla Centos 6.9 install: ./bootstrap.sh --prefix=/path_to_boost_1_68_0 ./b2 ./b2 install

[dancooke]

Hi,

I've successfully build Octopus v0.5.2-beta with GCC 8.2 and Boost 1.68, so the problem is certainly with your GCC or Boost installation.

It's strange that your GNU C++ 6.4 compiler installation is /usr/local/gcc-6.4.0/bin/gcc, as gcc usually refers to the C compiler rather than the C++ compiler (g++ is normally used for the C++ compiler).

Assuming that you installed the GCC 8.2 C++ compiler in /usr/local/gcc-8.2.0/bin/g++, could you please post the output from

/usr/local/gcc-8.2.0/bin/g++ -v

Please also try to install Boost with the following (after completely removing your previous installs):

cd /tmp
wget -O boost_1_68_0.tar.gz http://sourceforge.net/projects/boost/files/boost/1.68.0/boost_1_68_0.tar.gz/download
tar xzvf boost_1_68_0.tar.gz
cd boost_1_68_0
echo "using gcc : 8.2 : /usr/local/gcc-8.2.0/bin/g++ ; " >> tools/build/src/user-config.jam
./bootstrap.sh --prefix=/usr/local --without-libraries=python,mpi
./b2 -j2 toolset=gcc-8.2 cxxflags="-std=c++14"
./b2 install

Note that you will need to change the g++ path if I guessed incorrectly.

[jbsysadmin]

--- You posted: From: Daniel Cooke <notifications@github.com> Sent: Friday, December 14, 2018 5:44 AM To: luntergroup/octopus <octopus@noreply.github.com> Cc: Burns, W. J. <wjb2130@cumc.columbia.edu>; Author <author@noreply.github.com> Subject: Re: [luntergroup/octopus] octopus-0.5.2-beta does not compile, Centos 6.9 (#43) I've successfully build Octopus v0.5.2-beta with GCC 8.2 and Boost 1.68, so the problem is certainly with your GCC or Boost installation. It's strange that your GNU C++ 6.4 compiler installation is /usr/local/gcc-6.4.0/bin/gcc, as gcc usually refers to the C compiler rather than the C++ compiler (g++ is normally used for the C++ compiler). Assuming that you installed the GCC 8.2 C++ compiler in /usr/local/gcc-8.2.0/bin/g++, could you please post the output from /usr/local/gcc-8.2.0/bin/g++ -v Please also try to install Boost with the following (after completely removing your previous installs): cd /tmp wget -O boost_1_68_0.tar.gz http://sourceforge.net/projects/boost/files/boost/1.68.0/boost_1_68_0.tar.gz/download tar xzvf boost_1_68_0.tar.gz cd boost_1_68_0 echo "using gcc : 8.2 : /usr/local/gcc-8.2.0/bin/g++ ; " >> tools/build/src/user-config.jam ./bootstrap.sh --prefix=/usr/local --without-libraries=python,mpi ./b2 -j2 toolset=gcc-8.2 cxxflags="-std=c++14" ./b2 install Note that you will need to change the g++ path if I guessed incorrect

…

---- end of post ---- Thank you for this instruction. I built the Boost following your steps. During the ensuing Octpopus build, I see it took much longer per object than before. Same end result at bottom, however. The output of g++ -v is below. All source builds were conducted to an NFS share with a long name; I converted that to /usr/local/ For presentation in support tickets. Using built-in specs. COLLECT_GCC=/usr/local/gcc-8.2.0-x86_64/bin/g++ COLLECT_LTO_WRAPPER=/usr/local/gcc-8.2.0-x86_64/libexec/gcc/x86_64-pc-linux-gnu/8.2.0/lto-wrapper Target: x86_64-pc-linux-gnu Configured with: ../configure --prefix=/usr/local/gcc-8.2.0-x86_64 --enable-multilib : (reconfigured) ../configure --prefix=/usr/local/gcc-8.2.0-x86_64 --enable-bootstrap --enable-languages=c,c++,fortran,lto --enable-shared --enable-threads=posix --enable-checking=release --disable-multilib --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-gnu-unique-object --enable-linker-build-id --with-gcc-major-version-only --enable-plugin --with-linker-hash-style=gnu --enable-initfini-array --enable-libmpx --enable-gnu-indirect-function --with-tune=generic : (reconfigured) ../configure --prefix=/usr/local/gcc-8.2.0-x86_64 --enable-multilib Thread model: posix gcc version 8.2.0 (GCC

-John

---- Error: /usr/local/installs/octopus/src/utils/genotype_reader.cpp: In function ‘octopus::Genotype<octopus::Haplotype> octopus::{anonymous}::extract_genotype(const std::vector<octopus::{anonymous}::CallWrapper>&, const octopus::GenomicRegion&, const SampleName&, const octopus::ReferenceGenome&)’: /usr/local/installs/octopus/src/utils/genotype_reader.cpp:144:26: error: ‘*((void*)& max_ref_pad +4)’ may be used uninitialized in this function [-Werror=maybe-uninitialized] boost::optional<int> max_ref_pad {}; ^~~~~~~~~~~ cc1plus: all warnings being treated as errors make[2]: *** [src/CMakeFiles/octopus.dir/utils/genotype_reader.cpp.o] Error 1

[dancooke]

I've been trying to replicate your error in a CentOS 6.9 Docker container but have not been able to. I did however run into other problems with compilation due to the compiler emitting unrecognised instructions, suggesting that GCC was not installed correctly. I did not encounter any of these problems with CentOS 7.

I finally managed to install Octopus in a fresh CentOS 6.9 Docker container using Linuxbrew to install dependencies. For example, the following works:

docker run -it --rm centos:6.9

## Initial installs
yum -y update
yum install -y which yum-utils gcc gcc-c++ make wget git autoconf openssl bzip2.x86_64 bzip2 ca-certificates curl file fonts-dejavu-core locales openssh-client patch sudo uuid-runtime

## Add and switch to non-root sudo user (for Linuxbrew install)
localedef -i en_US -f UTF-8 en_US.UTF-8 && useradd -m -s /bin/bash linuxbrew && echo 'linuxbrew ALL=(ALL) NOPASSWD:ALL' >>/etc/sudoers
sudo su - linuxbrew

## Install Linuxbrew
git clone https://github.com/Linuxbrew/brew.git ~/.linuxbrew
PATH="$HOME/.linuxbrew/bin:$PATH"

## Install Octopus dependencies with Linuxbrew
brew install git python gcc@8 cmake boost htslib

## Install Octopus
git clone -b master https://github.com/luntergroup/octopus.git
cd octopus
export HTSLIB_ROOT=~/.linuxbrew
./scripts/install.py --threads 3 -c ~/.linuxbrew/Cellar/gcc@8/8.2.0/bin/gcc-8 -cxx ~/.linuxbrew/Cellar/gcc@8/8.2.0/bin/g++-8 --boost ~/.linuxbrew

I would recommend that you do the same.

[dancooke]

Hi, I've improved the installation script so that it can optionally install all of Octopus' dependencies into the build directory using Homebrew (as shown above). This is available on the develop branch (I'll add to next release). You should just be able to use

$ git clone https://github.com/luntergroup/octopus.git
$ octopus/scripts/install.py --install-dependencies

to cleanly install Octopus. Let me know if you have any issues.

[dancooke]

I'm closing this as I believe the new installation procedure resolves the issue. Please re-open if not.

[jbsysadmin]

Sorry for delay. I had some ultra priority items, and did not get to consider this again for the rest of the year. Greetings. I don't think we'll be able to use linuxbrew in the environment here. * When building from source in Centos 6.9, we need output targets that can be shared across the board: a working NFS-shared Python, gcc, boost, etc. * When building from Centos 7.4 (a future option) we need to be able to build an octopus RPM, with dependency RPMS, using resources present in the vanilla OS. How was your gcc 8.2 compiled? What flags were used? Regards, -John From: Daniel Cooke <notifications@github.com> Sent: Monday, December 17, 2018 8:48 AM To: luntergroup/octopus <octopus@noreply.github.com> Cc: Burns, W. J. <wjb2130@cumc.columbia.edu>; Author <author@noreply.github.com> Subject: Re: [luntergroup/octopus] octopus-0.5.2-beta does not compile, Centos 6.9 (#43) Hi, I've improved the installation script so that it can optionally install all of Octopus' dependencies into the build directory using Homebrew (as shown above). This is available on the develop branch (I'll add to next release). You should just be able to use $ git clone https://github.com/luntergroup/octopus.git $ octopus/scripts/install.py --install-dependencies to cleanly install Octopus. Let me know if you have any issues.

[jbsysadmin]

Hello again. Additionally, BOOST_ROOT, HTSLIB_ROOT, HTSLIB_INCLUDE_DIR, & HTSLIB_LIBRARY were exported. Were there any other environment variables to consider? From: Burns, W. J. Sent: Wednesday, January 02, 2019 5:34 PM To: 'luntergroup/octopus' <reply@reply.github.com>; luntergroup/octopus <octopus@noreply.github.com> Cc: Author <author@noreply.github.com> Subject: RE: [luntergroup/octopus] octopus-0.5.2-beta does not compile, Centos 6.9 (#43) Sorry for delay. I had some ultra priority items, and did not get to consider this again for the rest of the year. Greetings. I don't think we'll be able to use linuxbrew in the environment here. * When building from source in Centos 6.9, we need output targets that can be shared across the board: a working NFS-shared Python, gcc, boost, etc. * When building from Centos 7.4 (a future option) we need to be able to build an octopus RPM, with dependency RPMS, using resources present in the vanilla OS. How was your gcc 8.2 compiled? What flags were used? Regards, -John From: Daniel Cooke <notifications@github.com> Sent: Monday, December 17, 2018 8:48 AM To: luntergroup/octopus <octopus@noreply.github.com> Cc: Burns, W. J. <wjb2130@cumc.columbia.edu>; Author <author@noreply.github.com> Subject: Re: [luntergroup/octopus] octopus-0.5.2-beta does not compile, Centos 6.9 (#43) Hi, I've improved the installation script so that it can optionally install all of Octopus' dependencies into the build directory using Homebrew (as shown above). This is available on the develop branch (I'll add to next release). You should just be able to use $ git clone https://github.com/luntergroup/octopus.git $ octopus/scripts/install.py --install-dependencies to cleanly install Octopus. Let me know if you have any issues.