Source installation (Linux)

Mario Emmenlauer edited this page Jun 27, 2018 · 14 revisions

CentOS 6

  • Install the Java Development Kit (JDK)
  • Install Python
  • Install Cython, matplotlib, NumPy, and SciPy with pip
  • Clone the CellProfiler repository
  • Run pip install -e . from the CellProfiler directory

Ubuntu 16.04 LTS

https://github.com/CellProfiler/CellProfiler/wiki/Source-installation-(Ubuntu-16.04-LTS)

Ubuntu 14.04 LTS

https://github.com/CellProfiler/CellProfiler/wiki/Source-Installation-(Ubuntu-14.04-LTS)

Ubuntu 9.10

https://github.com/CellProfiler/CellProfiler/wiki/Source-Installation-(Ubuntu-9.10)

CellProfiler 2 for Batch processing

There is a discussion in the forum about using CP2.0 for batch processing, from where much of the following section originates. A similar approach using pip/virtualenv is probably the easiest way to get many of the required python libraries.

Note that with CellProfiler 2, there is no need for a separate CPCluster anymore. See the command line options for CellProfiler.py to invoke it in (headless) batch mode for cluster operation.

Option 1: Using the Makefile

These instructions will compile and install CellProfiler into a specific directory. If you are looking for an installation on your Linux desktop computer, you might prefer the instructions in the next section.

To fully automatically download, compile and install all prerequisites from scratch, you can use the supplied Makefile.CP2 in the root directory. It will install almost all prerequisites that are not to be expected in Linux Standard Base installation, except for Java (but look at https://github.com/CellProfiler/CellProfiler-build for the complete list of dependencies).

In the most simple case, the following invocation will download all required packages to this directory, and compile and install CellProfiler 2 trunk version into the directory specified with PREFIX:

export PREFIX="${HOME}/usr/cp2"
export GITHOME=$PREFIX/src/CellProfiler
mkdir -p $PREFIX/src
git clone https://github.com/CellProfiler/CellProfiler $GITHOME
cd $GITHOME
make -f Makefile.CP2 PREFIX="${PREFIX}"

More options are available and are documented within the header of the makefile itself.

After installation, you will want to export the paths to CellProfiler, its library dependencies, and to Java in your environment. For an installation PREFIX="${HOME}/usr/cp2", the following should be entered into your shells startup scripts (i.e. into $HOME/.bashrc):

# python and other CP2 prerequisites
export PATH="$HOME/usr/cp2/bin:${PATH}"
export LD_LIBRARY_PATH="$HOME/usr/cp2/lib:$HOME/usr/cp2/lib/mysql:$HOME/usr/cp2/lib64:${LD_LIBRARY_PATH}"

# OpenJDK 1.6 for 64bit (this will depend on your Java installation!)
export JAVA_HOME="/usr/lib/jvm/java-6-openjdk"
export PATH="$JAVA_HOME/bin:${PATH}"
export LD_LIBRARY_PATH="${JAVA_HOME}/jre/lib/amd64:${JAVA_HOME}/jre/lib/amd64/server:${LD_LIBRARY_PATH}"

You can then start CellProfiler by invoking python in the PREFIX directory:

cd "$HOME/usr/cp2/CellProfiler/"
python CellProfiler.py --do-not-build --do-not-fetch

Ilastik is a pixel-based classifier that is integrated into CellProfiler through the ClassifyPixels module. You can run Ilastik itself using the following command line:

python -m ilastik.ilastikMain

Option 2: Install all the individual libraries and dependencies

CellProfiler 2 requires that you install the following:

  1. MatplotLib: A Python plotting library for Numpy.
  2. WxPython: A wrapper for the cross-platform GUI API wxWidgets (which is written in C++) for Python
  3. Java Development Kit
  4. Cython: Generates C extensions to Python.
  5. MySQLdb: A Python interface to MySQL
  6. NumPy: A Python extension which provides convenient and fast N-dimensional array manipulation. Important note: You will need to download the NumPy MKL version of the package for compatibility with Scipy.
  7. SciPy: A library of algorithms and mathematical tools for Python; depends on Numpy.
  8. ZMQ Inter-process communication for multiprocessing mode.
  9. H5py: For reading and writing HDF5 files

Potential Pitfalls

The supplied Makefile for CP2 prerequisites tries to check for many known error conditions upon invocation. However, compilation on Unix is a complex thing, and certain errors can occur. Here is a list of potential pitfalls and a workaround (if known):

  • The ATLAS library is a beast to compile. It benefits from a recent c/c++ compiler, best is gcc-4.4 or better. It also requires a fortran compiler, with gfortran preferred. gfortran should be in the same version as gcc, so again gfortran-4.4 or better. If g77 is also installed on the same machine, strange linker problems can arise. Ideally, remove g77. For more details, see http://math-atlas.sourceforge.net/errata.html and http://www.scipy.org/Installing_SciPy/BuildingGeneral
  • during ATLAS library compilation, ATLAS performs code efficiency checks. These checks can only provide optimal results, if "frequency scaling" is disabled on the CPU, and no other load-intense processes are running. Ideally, you should compile on an empty cluster node with CPU frequency scaling disabled (see "cpufreq-selector").
  • CentOS 5 has a gcc-compiler that is known to be buggy, and that leads to a linker error. Use a more recent gcc compiler on CentOS 5.
  • When using a newer GCC compiler, it is not sufficient to set the CC and CPP variables. Instead, add the GCC "bin" directory to your PATH variable, and GCC's "lib" directory to your LD_LIBRARY_PATH variable. Make sure, when you call gcc --version from the command line, the correct gcc version is used. We can provide a Makefile for a very recent gcc installation as well, if required!
  • For wxWidgets (the GUI toolkit), a GTK installation is highly preferred over the native X11 toolkit. Without GTK, wxWidgets will look very bad and might have limited functionality. If the Unix you are compiling on does not provide GTK, you can compile it from the supplied Makefile using make gtk. This, again, requires several X11 development packages to be available - but is out of the scope of this document (sorry).
  • CellProfiler 2 has an excellent Java bridge. To make this Java bridge available, a recent Java is required (OpenJDK6 or Sun JDK6 both seem to work). It is recommended to set the JAVA_HOME variable, in order for python to select the correct (newest) java. Also, add JAVA_HOME to your PATH and LD_LIBRARY_PATH variable.
  • Java is not downloaded or installed automatically. Please download and unpack it yourself, and adjust your JAVA_HOME variable accordingly.