Pre-processing software for serial electron diffraction data using Python. See https://doi.org/10.1101/682575 for example results.
Diffractem is intended for usage within Jupyter notebooks - get a set of examples here: https://github.com/robertbuecker/serialed-examples.
Critical dependencies are CrystFEL 0.10.0 or higher (http://www.desy.de/~twhite/crystfel/), available at https://gitlab.desy.de/thomas.white/crystfel, and pinkIndexer, available at https://stash.desy.de/users/gevorkov/repos/pinkindexer.
To use most features you will also need an installation of CrystFEL and pinkIndexer.
Create conda enivronment
We strongly suggest to use the Anaconda3 Python distribution/package manager, and create a dedicated environment within it for diffractem (and CrystFEL).
If you do not have Anaconda installed, it is sufficient to obtain the minimal Miniconda of the
conda package manager at https://docs.conda.io/en/latest/miniconda.
Once installed, please create a new anaconda environment for diffractem, without installing any packages, and activate it:
conda create -n diffractem -c conda-forge conda activate diffractem
Install CrystFEL and pinkIndexer
It is probably worth checking if you already have an installation of CrystFEL (>=0.10.0) with pinkIndexer in your computing environment (e.g. via SBgrid). If so, you can skip this section and proceed with the installation of diffractem below. Otherwise, as you might not want the new CrystFEL version to interfere with a potentially existing (or future) installation of standard CrystFEL (or are not allowed to), or don't have sudo rights on your computer, we suggest to install them directly into the location of our anaconda environment.
Please first make sure that all dependencies for compiling CrystFEL are installed: https://gitlab.desy.de/thomas.white/crystfel/-/blob/master/INSTALL.md
[N.B. If you're having trouble fulfilling CrystFELs dependencies e.g. because you do not have root access to install them or they are not included in your distribution's package repository, also consider installing them using
As a common example, if you are using Ubuntu 16.04, which comes with a too old version of cmake, you may want to run
conda install cmake.]
Here is a complete sequence of steps to install pinkIndexer and CrystFEL, assuming you're installing CrystFEL directly from the repository at DESY (alternatively, you can download the latest release at http://www.desy.de/~twhite/crystfel/):
git clone https://stash.desy.de/scm/~gevorkov/pinkindexer.git git clone https://gitlab.desy.de/thomas.white/crystfel.git cd pinkindexer mkdir build cd build cmake .. -DCMAKE_INSTALL_PREFIX=$CONDA_PREFIX make -j `nproc` make install export PKG_CONFIG_PATH="$CONDA_PREFIX/lib64/pkgconfig:$PKG_CONFIG_PATH" cd ../../crystfel mkdir build cd build cmake .. -DCMAKE_INSTALL_PREFIX=$CONDA_PREFIX make -j `nproc` make install cd ../..
If you prefer to install CrystFEL globally, just omit the
-DCMAKE_INSTALL_PREFIX parameters and skip the
export PKG_CONFIG_PATH line.
After that, running
indexamajig --version should return something similar to
CrystFEL: 0.10.0+<some commit hash>.
Install Python and Jupyter
Now, install a basic Python with Jupyter into your environment.
We recommend using
jupyterlab for interaction with diffractem.
If you prefer the classic
jupyter, you can use it instead in the command below.
conda install -c conda-forge python=3.8 numpy jupyterlab ipywidgets ipympl
Finally install diffractem itself, either from PyPi:
pip install diffractem
or, if you want to play/develop a bit more and stay up-to-date, you can clone this git repository and install diffractem in developer mode:
git clone https://github.com/robertbuecker/diffractem cd diffractem pip install -e .
Now you should be ready to go! To get started, why don't you download the example notebooks:
git clone https://github.com/robertbuecker/serialed-examples
And get example raw data at MPDL Edmond: https://edmond.mpdl.mpg.de/imeji/collection/32lI6YJ7DZaF5L_K.
And when you're ready to go: just make your own branches of the notebooks for your own projects, and have fun!
diffractem, (C) 2019-2021 Robert Bücker, email@example.com
peakfinder8, (C) 2014-2019 Deutsches Elektronen-Synchrotron DESY