Deep-Phase

ResNet classification of condensate morphologies

Setup

Installing this package

# clone the repo to your project directory
git clone https://github.com/SoftLivingMatter/Deep-Phase.git
cd Deep-Phase

conda create --name deepPhase pytorch torchvision pytorch-cuda=12.1 -c conda-forge -c pytorch -c nvidia
conda activate deepPhase

pip install bioio bioio-nd2 bioio-tifffile  # or your preferred file type
pip install -e .

# cache model on login node
python -c 'from deep_phase.models.modified_resnet import build_network ;
build_network(resnet="resnet50")'

Analysis

Create plate layout

Each image set needs a mapping between wells and names. Add a plate_layout.csv to the directory containing images. Wells are signified in the image filename as Well{WELL}. Alternatively, you can retain the filename and match it to experimental conditions later.

,,DMSO,DMSO,DMSO,DMSO
,,CX-5461,CX-5461,CX-5461,CX-5461
,,FVP,FVP,FVP,FVP

Note that headers should not be included, empty rows above or columns to the left must be included. If the first row was empty it’d be:

,,,,,,
,,DMSO,DMSO,DMSO,DMSO
,,CX-5461,CX-5461,CX-5461,CX-5461
,,FVP,FVP,FVP,FVP

All images in the directory will be analyzed, ensure there are no extra files.

Find cells

Prior to running the network, cell locations need be recorded. There is a cellprofiler pipeline for this step. To submit it to the cluster, run

sbatch findcells.sbatch /path/to/images

in the cell_finding directory.

Once CellProfiler has run, check that you have multiple nuclei per frame or in your dataset (makes sure your cells are called correctly):

Check for the presence of these files in the image directory. Make sure they aren’t empty.

less EnlargedObjects.csv
less Image.csv

In particular, the Image.csv file includes a count of the number of cells per image.

Run Deep Phase

Configuration

All hyperparameters for a given network are encoded in a configuration yaml file. Generally, you should keep the same configuration between training and inference. A yaml with default values can be generated by:

deepPhase --print-defaults > config.yaml

Here are the results with added comments explaining each setting. Any settings which are not changed can be removed to use default values.

# config.yaml
# how to augment training images.  Options are randaug or simple
augmentation: randaug
# number of images to include in a batch of training or inference images
batch_size: 128
# how to parse cell positions from input csv.  Options are cellprofiler or processed
cell_position_format: cellprofiler
# name of csv with cell locations.  Must be in root directory of each input folder
cell_positions: EnlargedObjects.csv
# number of channels in input image
channels: 4
# size to crop from center position.  Final pixel size will be twice this value
crop_size: 192
# number of epochs for training
epochs: 50
# freeze initial resnet section during training
freeze: false
# number of latent dimensions in activation space
latent: 2
# limit the number of training examples per class.  Options are null, min or a number
limit: null
# how to map filenames to categories.  Options are plate, plate-loose, or file
# if plate or plate-loose, must have a plate_layout.csv in root input directories
mapper: plate
# where to save the network, relative to output_dir
network_name: network.pth
# type of network
network_type: flat
# amount of gaussian noise to include in simple augmentation
noise: 0.001
# where to store outputs, including network, configuration copy, logs and evals
output_dir: .
# size of resent base.  Options are resnet18, resnet34 or resnet50
resnet: resnet34
# mapping of channels to rgb of network input.  Here channels 2,3,4 are mapped
# to red green and blue.  Channel 1 is added to all channels with 25% intensity
rgb_map: 2,3,4:1[25]
# magnitude of rotation to apply with simple augmentation
rotation: 5
# if set to an existing network, use those weights for the resnet base
starting_network: null
# how to split the test, train and validation sets from the input data
test_train_split:
- 0.8
- 0.1
- 0.1
# name of classes to predict
training_classes:
- class1
- class2

The cellprofiler format must contain the columns Location_Center_X, Metadata_SizeX, (and for Y) Metadata_FileLocation, and Metadata_Series. The processed format requires columns for center_x, size_x (and y), series (will set to 0 if missing), category, and local_path.

In the rgb_map, channels set to 0 will be left black. An input channel can be used multiple times. 1,2,0 will use channel 1 in red, 2 in green and leave blue unused. 1,1,1 will be a gray scale image of only channel 1.

Training

Every deepPhase command requires the config and base name for all outputs and the input folders. Additionally you can control what portion of the analysis is run. Generally, for training you will want to run:

deepPhase \
  --config your_config.yaml \
  --base-name training \
  /path/to/your/inputs

The base name can be anything you'd like and you can include multiple input folders. This will run 50 epochs of training, save the best test set performing model, then evaluate the network on all images. At the end, your output folder will contain the following files:

• network.pth: by default the network weights.
• training_config.yaml: a copy of the configuration along with input files
• training_log.csv: record of the training and test loss over epochs. The header (lines with #) also contains information on the number of cells and images.
• training_eval.csv: results of evaluation. Each row has information on the cell location, filename, and series. For training outputs, the data_usage is reported, which tracks if a cell was used for training, testing, validation, or evaluation. The act_x and act_y indicate the activation value of the latent neurons. For each training class, there is a cls_{class} column with the softmax probability of that class. The called class contains the training class with the highest probability. It is set to no_call if no class is above 0.8.

If you want to skip the evaluation step, you may pass in the --no-eval option.

Training will use a GPU if available. Data is streamed to the device but memory usage can be large due to the resnet size. For resnet50 and 128 batch size, we find memory usage to be around 13 GB and training takes from several minutes to hours depending on the number of cells provided. Data is cached into CPU memory prior to training so sufficient RAM is also required to hold all cells.

Evaluation

To use an existing network on new images, you supply a configuration file and a new base name. To skip training, using the --no-train option. If you forget this step, deepPhase will try to train a new network. However, it will stop with an error message about the network already existing. The same message is displayed if any of the output files already exists to prevent accidental deletion. To ignore this warning and continue, use the --overwrite flag.

An example evaluation would be:

deepPhase \
  --config your_config.yaml \
  --no-train \
  --base-name experiment \
  /path/to/your/experiment_inputs

The config, log and eval files will all be generated using the existing network specified by the configuration file. It is possible to alter the number of channels, rgb map and other settings between training and inference, but the network architecture must match (e.g. resnet size, latent layers, etc). We have also found the performance generally suffers with such adjustments but can still provide sufficient accuracy for some settings. Ideally, train as close to the inference setting as possible.

GPU utilization is generally lower for inference, taking only a few minutes but large GPU memory is needed to fit the network parameters.

Data analysis

The resulting csv file can be analyzed in your software of choice. A few helper functions are provided in the utils package to simplify common plots. Sample notebooks can be found HERE