Skip to content


Switch branches/tags

Name already in use

A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch?

Latest commit


Git stats


Failed to load latest commit information.
Latest commit message
Commit time

Open In Colab

ERnet Transformer

Vision Transformer applied to segmentation of endoplasmic reticulum (ER) image sequences

Meng Lu1, Charles N. Christensen1,*, Jana M. Weber1,*, Tasuku Konno, Nino F. Läubli1, Katharina M. Scherer1, Edward Avezov, Pietro Lio, Alexei A. Lapkin, Gabriele S. Kaminski Schierle1, Clemens F. Kaminski1

1University of Cambridge, Department of Chemical Engineering and Biotechnology
*Author of this repository - GitHub username: charlesnchr - Email address:



ERnet is a vision transformer-based processing pipeline for segmentation and analysis of temporal image sequences of the endoplasmic reticulum. The main component is the vision transformer model used for segmentation, inspired by Swin. Other steps in the pipeline that facilitates quantitative analysis are shown in the figure above.

The method follows our previous work, in which we used a residual neural network for two-class image segmentation, ERnet — Residual network.

Try it

Get started right away with our Colab notebooks. If you prefer running the code locally, we provide Jupyter notebooks and Python scripts in this repository. Another option for running ERnet is to use it as a plugin in the graphical front-end, Mambio, based on an integration of the Electron UI framework and a Python environment. Mambio is also included in this repository.

Repository structure

The graphical user interface, Mambio, is provided in the folder Graphical-App. Instructions and further information can be found in the corresponding README.

Code for training models based on the ERnet architecture can be found in the folder Training, and further information is also in the README file of that folder. A script that generates synthetic ER images can be found in Training/Simulate_ER, see the Synthetic data generation pipeline section.


To run inference with ERnet locally, we provide the graphical user interface Mambio: Multi-purpose Advanced ML-based Batch Image Operations. Mambio is a more general software that extends beyond image segmentation — there is also a plugin for ML-SIM.

Synthetic data generation pipeline for testing

We propose a model for synthesising ground truth ER images with a simple algorithm. First a set of uniformly distributed coordinates are generated. Delaunay triangulation is performed on the coordinates providing a set of triangles. The circumcentre of each triangle becomes a vertex in a Voronoi diagram. The diagram is formed by connecting vertices between adjacent triangles. This procedure is called Voronoi tessellation. Each edge in the undirected graph corresponding to the Voronoi diagram is the perturbed in the following way: the midpoint of an edge is moved in a random direction by a random distance that is smaller than the length of the edge. The displaced midpoint and the two ends of the edge are then used the produce a smooth curve using cubic spline interpolation of the three points. Finally, the network of vertices connected with interpolated curves are processed with a image formation model that applies blur (diffraction) and noise (Poisson noise and Gaussian read-out noise).

The steps of this algorithm are illustrated on the figure above. The implementation can be found in Training/Simulate_ER, and the script can be used to generate a collection of test images with


Parameters are given in the file.

Preceding work

Github repository

ERNet — Residual network
Segmentation of Endoplasmic Reticulum microscopy images using modified CNN-based image restoration models.

Science Advances publication

Meng Lu1, Francesca W. van Tartwijk1, Julie Qiaojin Lin1, Wilco Nijenhuis, Pierre Parutto, Marcus Fantham1, Charles N. Christensen1,*, Edward Avezov, Christine E. Holt, Alan Tunnacliffe, David Holcman, Lukas C. Kapitein, Gabriele Kaminski Schierle1, Clemens F. Kaminski1

1University of Cambridge, Department of Chemical Engineering and Biotechnology
*Author of this repository - GitHub username: charlesnchr



Segmentation and analysis of the endoplasmic reticulum facilitated by a vision transformer segmentation model







No packages published