Addressing Data Scarcity in Nanomaterial Segmentation Networks with Differentiable Rendering
This repository contains the implementation of DiffRenderGAN, a framework developed to synthesize annotated microscopy images of nanoparticles using differentiable rendering integrated into a generative adversarial network.
The method and experiments are described in the study accepted in npj Computational Materials:
Title: Addressing Data Scarcity in Nanomaterial Segmentation Networks with Differentiable Rendering
Journal: npj Computational Materials
Authors: [Dennis Possart, Leonid Mill, Florian Vollnhals, Tor Hildebrand, Peter Suter, Mathis Hoffmann, Jonas Utz, Daniel Augsburger, Mareike Thies, Mingxuan Gu, Fabian Wagner, George Sarau, Silke Christiansen, Katharina Breininger]
DOI: [Pending]
This implementation was tested on:
- OS: AlmaLinux 9.2 (Turquoise Kodkod)
- GPU: NVIDIA A40, Driver 545.23.08, CUDA 12.3
.
├── CycleGAN/ # CycleGAN components
├── datasets/ # Dataset definitions and utilities
├── experiments/ # Pretrained weights and experimental outputs
├── model/ # Model architecture
├── scripts/ # Scripts for training and testing
├── utils/ # Helper functions and utilities
├── train_gan.py # Entry point for training
├── test_gan.py # Entry point for evaluation
├── environment.yml # Conda environment definition
├── .gitmodules
├── .gitignore
├── LICENSE # MIT License
└── README.md
We recommend using Conda to manage dependencies:
conda env create -f environment.yml
conda activate diffrendergan
Prior to training, microscopy images should be preprocessed into greyscale 256 × 256 pixel patches, each containing at least one nanoparticle instance.
For details regarding dataset sources and preprocessing pipelines, please refer to the publication. Raw data used in the experiments can be obtained from the studies cited in the paper.
To train DiffRenderGAN, use train_gan.py.
Example training scripts illustrating parameter settings are provided in scripts/train. See the example scripts and adapt the parameter configuration.
For a complete list of configurable arguments, run:
python train_gan.py --help
After training, models can be evaluated using test_gan.py.
Example testing scripts illustrating are provided in scripts/test.
For details on all configurable arguments, run:
python test_gan.py --help
We are preparing additional materials that will be released in upcoming updates:
- Example Jupyter notebooks demonstrating training, evaluation, and synthetic data generation workflows
- nnU-Net segmentation results and configuration files corresponding to the experiments in the paper
This project is released under the MIT License. See the LICENSE file for details.
If you use this code or pretrained models in your research, please cite our publication:
[Pending]
For questions regarding the code or experiments, please contact dennis.possart@gmail.com.