A memory efficient implementation can be found here.
Tim-Oliver Buchholz1, Mareike Jordan, Gaia Pigino, Florian Jug
1tibuch@mpi-cbg.de
Multiple approaches to use deep learning for image restoration have recently been proposed. Training such approaches requires well registered pairs of high and low quality images. While this is easily achievable for many imaging modalities, e.g. fluorescence light microscopy, for others it is not. Cryo-transmission electron microscopy (cryo-TEM) could profoundly benefit from improved denoising methods, unfortunately it is one of the latter. Here we show how recent advances in network training for image restoration tasks, i.e. denoising, can be applied to cryo-TEM data. We describe our proposed method and show how it can be applied to single cryo-TEM projections and whole cryo-tomographic image volumes. Our proposed restoration method dramatically increases contrast in cryo-TEM images, which improves the interpretability of the acquired data. Furthermore we show that automated downstream processing on restored image data, demonstrated on a dense segmentation task, leads to improved results.
Publications:
This is the implementation of the Tomo2Tomo denoising scheme for direct-detector movie acquisitions.
- NVIDIA GPU with driver >= 430.40
First step is to clone this github repository.
Then the simplest way to run the examples is to use the Singularity container. Singularity allows us to provied you with a fully configured environment, which has all needed packages and dependecies installed. But it requires you to install Singularity on your system:
Once you have installed Singularity you can either build the container yourself with the following commands:
- Change direcotry into the cloned repository:
$ cd cryoCARE_T2T - Run the Singularity build:
$ sudo singularity build cryoCARE_v0.1.1.simg cryoCARE.Singularity
If you don't have sudo rights you can also download the built Singularity container from here.
Due to licensing we can not ship MotionCor2 directly with the singularity container. You have to download MotionCor2 v1.3.0 CUDA-10.1 from here and place it in cryoCARE_T2T/example.
Note: It is important to get the CUDA-10.1 version, since we specifically installed CUDA-10.1 in the singularity container.
We would like to thank Mareike Jordan from the Pigino Lab at MPI-CBG for the example data.
The data can be downloaded here.
Once the data is downloaded unpack it into cryoCARE_T2T/example/data/.
You should now have the following directory tree:
cryo_CARE_T2T
| └─── example
| └─── data
| | └─── Tomo110
| | └─── frames
| | └─── imod
| |--- MotionCor2 v1.3.0 CUDA-10.1
To start a jupyter notebook from the singularity image run this command:
$ singularity run --nv -B user:/run/user -B example/:/notebooks -B example/data/:/data cryoCARE_v0.1.1.simg
This will display a link to the running jupyter server. Open this link in a browser and you will be able to run the five example notebooks in 'cryoCARE_simg/example':
01_Split_Frames.ipynb: This notebook aligns and splits the movie frames into even/odd halves.02_Tomogram_Reconstruction.ipynb: This notebook reconstructs the two (even/odd) tomograms based on a previous IMOD-reconstruction.03_Training_Data_Generation.ipynb: This notebook is used to extract training and validation data.04_Train_cryoCARE_Network.ipynb: This notebook shows how a cryo-CARE network is trained.05_Predict_cryoCARE.ipynb: The last notebook is used to apply a trained model to tomographic data.
You can also install the packages on your own:
- MotionCor2 v1.3.0 CUDA-10.1
- IMOD v4.10.16 (beta-release)
- CUDA-10.1 for MotionCor2
- CUDA-10.0 for Tensorflow
- Tensorflow 1.15
$ pip install tensorflow-gpu==1.15 - csbdeep
$ pip install csbdeep - mrcfile
$ pip install mrcfile - jupyter
$ pip install jupyter
Note: If you run it without the Singularity container you probably have to adjust the paths inside the jupyter notebooks.
@inproceedings{buchholz2019cryo,
title={Cryo-CARE: content-aware image restoration for cryo-transmission electron microscopy data},
author={Buchholz, Tim-Oliver and Jordan, Mareike and Pigino, Gaia and Jug, Florian},
booktitle={2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019)},
pages={502--506},
year={2019},
organization={IEEE}
}
@article{buchholz2019content,
title={Content-aware image restoration for electron microscopy.},
author={Buchholz, Tim-Oliver and Krull, Alexander and Shahidi, R{\'e}za and Pigino, Gaia and J{\'e}kely, G{\'a}sp{\'a}r and Jug, Florian},
journal={Methods in cell biology},
volume={152},
pages={277--289},
year={2019}
}
