semantic_segmentation.rst

Semantic Segmentation

The Task

Semantic Segmentation, or image segmentation, is the task of performing classification at a pixel-level, meaning each pixel will associated to a given class. See more: https://paperswithcode.com/task/semantic-segmentation

---

Example

Let's look at an example using a data set generated with the CARLA driving simulator. The data was generated as part of the Kaggle Lyft Udacity Challenge. The data contains one folder of images and another folder with the corresponding segmentation masks. Here's the structure:

data
├── CameraRGB
│   ├── F61-1.png
│   ├── F61-2.png
│       ...
└── CameraSeg
    ├── F61-1.png
    ├── F61-2.png
        ...

Once we've downloaded the data using ~flash.core.data.download_data, we create the ~flash.image.segmentation.data.SemanticSegmentationData. We select a pre-trained mobilenet_v3_large backbone with an fpn head to use for our ~flash.image.segmentation.model.SemanticSegmentation task and fine-tune on the CARLA data. We then use the trained ~flash.image.segmentation.model.SemanticSegmentation for inference. You can check the available pretrained weights for the backbones like this SemanticSegmentation.available_pretrained_weights("resnet18"). Finally, we save the model. Here's the full example:

../../../flash_examples/semantic_segmentation.py

To learn how to view the available backbones / heads for this task, see backbones_heads.

---

Flash Zero

The semantic segmentation task can be used directly from the command line with zero code using flash_zero. You can run the above example with:

flash semantic_segmentation

To view configuration options and options for running the semantic segmentation task with your own data, use:

flash semantic_segmentation --help

---

Serving

The ~flash.image.segmentation.model.SemanticSegmentation task is servable. This means you can call .serve to serve your ~flash.core.model.Task. Here's an example:

../../../flash_examples/serve/semantic_segmentation/inference_server.py

You can now perform inference from your client like this:

../../../flash_examples/serve/semantic_segmentation/client.py