Skip to content
This repository has been archived by the owner on Oct 9, 2023. It is now read-only.

Latest commit

 

History

History
118 lines (92 loc) · 4.69 KB

fiftyone.rst

File metadata and controls

118 lines (92 loc) · 4.69 KB
Raw

FiftyOne

We have collaborated with the team at Voxel51 to integrate their tool, FiftyOne, into Lightning Flash.

FiftyOne is an open-source tool for building high-quality datasets and computer vision models. The FiftyOne API and App enable you to visualize datasets and interpret models faster and more effectively.

This integration allows you to view predictions generated by your tasks in the FiftyOne App <fiftyone:fiftyone-app>, as well as easily incorporate FiftyOne Datasets <fiftyone:fiftyone-basics> into your tasks. All image and video tasks are supported!

Installation

In order to utilize this integration, you will need to install FiftyOne <fiftyone:installing-fiftyone>:

pip install fiftyone

Visualizing Flash predictions

This section shows you how to augment your existing Lightning Flash workflows with a couple of lines of code that let you visualize predictions in FiftyOne. You can visualize predictions for classification, object detection, and semantic segmentation tasks. Doing so is as easy as updating your model to use one of the following outputs:

  • FiftyOneLabelsOutput(return_filepath=True)<flash.core.classification.FiftyOneLabelsOutput>
  • FiftyOneSegmentationLabelsOutput(return_filepath=True)<flash.image.segmentation.output.FiftyOneSegmentationLabelsOutput>
  • FiftyOneDetectionLabelsOutput(return_filepath=True)<flash.image.detection.output.FiftyOneDetectionLabelsOutput>

The ~flash.core.integrations.fiftyone.visualize function then lets you visualize your predictions in the FiftyOne App <fiftyone:fiftyone-app>. This function accepts a list of dictionaries containing FiftyOne Label <fiftyone:using-labels> objects and filepaths, which is exactly the output of the FiftyOne outputs when the return_filepath=True option is specified.

../../../flash_examples/integrations/fiftyone/image_classification.py

The ~flash.core.integrations.fiftyone.visualize function can be used in all of the following environments:

  • Local Python shell: The App will launch in a new tab in your default web browser
  • Remote Python shell: Pass the remote=True option and then follow the instructions printed to your remote shell to open the App in your browser on your local machine
  • Jupyter notebook: The App will launch in the output of your current cell
  • Google Colab: The App will launch in the output of your current cell
  • Python script: Pass the wait=True option to block execution of your script until the App is closed

See this page <fiftyone:environments> for more information about using the FiftyOne App in different environments.

Using FiftyOne datasets

The above workflow is great for visualizing model predictions. However, if you store your data in a FiftyOne Dataset initially, then you can also visualize ground truth annotations. This allows you to perform more complex analysis with views <fiftyone:using-views> into your data and evaluation <fiftyone:evaluating-models> of your model results.

The ~flash.core.data.data_module.DataModule.from_fiftyone method allows you to load your FiftyOne datasets directly into a ~flash.core.data.data_module.DataModule to be used for training, testing, or inference.

../../../flash_examples/integrations/fiftyone/image_classification_fiftyone_datasets.py

Visualizing embeddings

FiftyOne provides the methods for dimensionality reduction<fiftyone:brain-embeddings-visualization> and interactive plotting<fiftyone:embeddings-plots>. When combined with embedding tasks <image_embedder> in Flash, you can accomplish powerful workflows like clustering, similarity search, pre-annotation, and more in only a few lines of code.

../../../flash_examples/integrations/fiftyone/image_embedding.py

embeddings_example