Skip to content

v0.6.0 — PyTorch migration

Latest

Choose a tag to compare

@zsylvester zsylvester released this 01 Nov 09:12

v0.6.0 Release Notes

The deep-learning backend has moved from TensorFlow/Keras to PyTorch. The U-Net, its training code, and the CNN feature extractors used for grain clustering are now implemented with torch and torchvision; SAM 2.1 already used PyTorch. Installation no longer requires TensorFlow or Keras, and a single framework serves all models. Segmentation results are unchanged: the shipped PyTorch U-Net reproduces the Keras model's predictions to within 1e-5.

Breaking Changes

  • The U-Net model file is now a PyTorch checkpoint. The shipped model is models/seg_model_smooth_labels.pt (9 MB, replacing the 26 MB .keras file) and is loaded with seg.load_unet(path) instead of keras.saving.load_model(path). The notebooks have been updated accordingly. seg.Unet() returns the PyTorch UNet module.
  • Custom or fine-tuned Keras models must be converted once, in an environment where Keras and TensorFlow are still installed: seg.convert_keras_model("my_model.keras", "my_model.pt"). The converter checks that the two models agree before writing the file. Models from any previous version convert, including v0.3.0-era models with the softmax built in. Passing a Keras model object to the segmentation functions still works for now but emits a FutureWarning; this fallback will be removed in a later release.
  • The training API is built on PyTorch and lives in the new training submodule (still re-exported at the package level): create_train_val_test_data returns PyTorch DataLoaders (with batch_size and num_workers arguments) instead of tf.data datasets, with the same train/validation/test split; create_and_train_model accepts learning_rate and device arguments and a .pt (or, with Keras installed, .keras) starting model; weighted_crossentropy takes logits of shape (N, 3, H, W) and targets as class indices (N, H, W) or one-hot (N, 3, H, W), with unchanged class weights and label smoothing; patchify_training_data is implemented with numpy and returns the image and label directories; evaluate_model returns a dictionary with loss, accuracy, mean_iou, and per_class_iou. Trained models are saved with seg.save_unet(model, "model.pt").
  • grain_utils.extract_vgg16_features uses torchvision's ImageNet models (VGG16, ResNet50, InceptionV3) with the same preprocessing as before. Features are not numerically identical to the Keras ones because the pretrained weights differ, so previously computed feature vectors and clusterings are not directly comparable to new ones. The layer_name argument keeps the Keras names (fc1, fc2, flatten, avg_pool, ...). The weights are downloaded by torchvision on first use.
  • tensorflow and keras are no longer dependencies; torchvision is a new one. load_and_preprocess was removed (its work is done by GrainPatchDataset).
  • Environment files are split by hardware instead of operating system: environment.yml (Apple Silicon via MPS, or CPU) and environment_cuda.yml (NVIDIA GPUs via the conda-forge pytorch-gpu package). environment_macos.yml is removed.

New Features

  • UNet module, load_unet, save_unet, predict_batch, get_device (CUDA, MPS or CPU auto-detection), and convert_keras_model in the new unet submodule.
  • train_model, plot_training_history, GrainPatchDataset, and augment_pair in the new training submodule.
  • predict_image accepts a batch_size argument; tiles are batched through the model instead of being predicted one by one, and the result is returned as float32 probabilities.
  • sam_segmentation raises an informative ValueError when called with an empty prompt list, and predict_large_image emits a warning when no grains are detected. Previously a failed U-Net prediction could surface as a cryptic downstream error such as "attempt to get argmax of an empty sequence" (#10).

Bug Fixes

  • GrainPlot and plot_image_w_colorful_grains work with matplotlib 3.11, where matplotlib.cm.get_cmap was removed.
  • Region properties are requested under their current scikit-image names (axis_major_length, intensity_mean, ...), which silences the deprecation warnings of scikit-image 0.26. Grain summary columns keep their existing names (major_axis_length, mean_intensity-0, ...).
  • On macOS, importing the package no longer risks the duplicate OpenMP runtime abort (OMP: Error #15) caused by loading TensorFlow and PyTorch in the same process.

Improvements

  • PyTorch is installed from conda-forge in the environment files so that it shares the environment's OpenMP runtime with numpy/scipy.
  • interactions.load_image and seg.load_img use Pillow instead of keras.utils.load_img; interactions.save_mask writes the same 8-bit grayscale PNG as before.
  • U-Net inference is substantially faster on Apple Silicon: a 256x256 tile takes about 4 ms on the MPS backend, compared with about 33 ms with TensorFlow Metal.
  • CI tests on Python 3.10 and 3.12 with CPU-only PyTorch wheels; the documentation build mocks torch, torchvision and sam2.
  • Added tests for the PyTorch U-Net, the Keras conversion, and the training pipeline (124 tests total, up from 80).

Deprecations

  • Passing a Keras model object to predict_image, predict_large_image and the other segmentation functions still works but emits a FutureWarning. Convert the model with seg.convert_keras_model and load the resulting .pt file with seg.load_unet.

Full Changelog: v0.5.0...v0.6.0