Pigeon annotator

docs/book/stacks-and-components/component-guide/annotators/annotators.md

@@ -55,11 +55,12 @@ The core parts of the annotation workflow include:
 ### List of available annotators
 
 For production use cases, some more flavors can be found in specific `integrations` modules. In terms of annotators,
-ZenML features an integration with `label_studio`.
+ZenML features integrations with `label_studio` and `pigeon`.
 
 | Annotator                               | Flavor         | Integration    | Notes                                                                |
 |-----------------------------------------|----------------|----------------|----------------------------------------------------------------------|
 | [LabelStudioAnnotator](label-studio.md) | `label_studio` | `label_studio` | Connect ZenML with Label Studio                                      |
+| [PigeonAnnotator](pigeon.md) | `pigeon` | `pigeon` | Connect ZenML with Pigeon. Notebook only & for image and text classification tasks.      |
 | [ProdigyAnnotator](prodigy.md)           | `prodigy`       | `prodigy`       | Connect ZenML with [Prodigy](https://prodi.gy/)                                             |
 | [Custom Implementation](custom.md)      | _custom_       |                | Extend the annotator abstraction and provide your own implementation |
 
@@ -71,9 +72,11 @@ zenml annotator flavor list
 
 ### How to use it
 
-The available implementation of the annotator is built on top of the Label Studio integration, which means that using an
-annotator currently is no different from what's described on
-the [Label Studio page: How to use it?](label-studio.md#how-do-you-use-it).
+The available implementation of the annotator is built on top of the Label
+Studio integration, which means that using an annotator currently is no
+different from what's described on the [Label Studio page: How to use
+it?](label-studio.md#how-do-you-use-it). ([Pigeon](pigeon.md) is also supported, but has a
+very limited functionality and only works within Jupyter notebooks.)
 
 ### A note on names
 

docs/book/stacks-and-components/component-guide/annotators/pigeon.md

@@ -0,0 +1,113 @@
+---
+description: Annotating data using Pigeon.
+---
+
+# Pigeon
+
+Pigeon is a lightweight, open-source annotation tool designed for quick and easy labeling of data directly within Jupyter notebooks. It provides a simple and intuitive interface for annotating various types of data, including:
+
+* Text Classification
+* Image Classification
+* Text Captioning
+
+### When would you want to use it?
+
+![Pigeon annotator interface](../../../.gitbook/assets/pigeon.png)
+
+If you need to label a small to medium-sized dataset as part of your ML workflow and prefer the convenience of doing it directly within your Jupyter notebook, Pigeon is a great choice. It is particularly useful for:
+
+* Quick labeling tasks that don't require a full-fledged annotation platform
+* Iterative labeling during the exploratory phase of your ML project
+* Collaborative labeling within a Jupyter notebook environment
+
+### How to deploy it?
+
+To use the Pigeon annotator, you first need to install the ZenML Pigeon integration:
+
+```shell
+zenml integration install pigeon
+```
+
+Next, register the Pigeon annotator with ZenML, specifying the output directory where the annotation files will be stored:
+
+```shell
+zenml annotator register pigeon --flavor pigeon --output_dir="path/to/dir"
+```
+
+Note that the `output_dir` is relative to the repository or notebook root.
+
+Finally, add the Pigeon annotator to your stack and set it as the active stack:
+
+```shell
+zenml stack update <YOUR_STACK_NAME> --annotator pigeon
+```
+
+Now you're ready to use the Pigeon annotator in your ML workflow!
+
+### How do you use it?
+
+With the Pigeon annotator registered and added to your active stack, you can easily access it using the ZenML client within your Jupyter notebook.
+
+For text classification tasks, you can launch the Pigeon annotator as follows:
+
+````python
+from zenml.client import Client
+
+annotator = Client().active_stack.annotator
+
+annotations = annotator.launch(
+    data=[
+        'I love this movie',
+        'I was really disappointed by the book'
+    ],
+    options=[
+        'positive',
+        'negative'
+    ]
+)
+````
+
+For image classification tasks, you can provide a custom display function to render the images:
+
+````python
+from zenml.client import Client
+from IPython.display import display, Image
+
+annotator = Client().active_stack.annotator
+
+annotations = annotator.launch(
+    data=[
+        '/path/to/image1.png',
+        '/path/to/image2.png'
+    ],
+    options=[
+        'cat',
+        'dog'
+    ],
+    display_fn=lambda filename: display(Image(filename))
+)
+````
+
+The `launch` method returns the annotations as a list of tuples, where each tuple contains the data item and its corresponding label.
+
+You can also use the `zenml annotator dataset` commands to manage your datasets:
+
+* `zenml annotator dataset list` - List all available datasets
+* `zenml annotator dataset delete <dataset_name>` - Delete a specific dataset
+* `zenml annotator dataset stats <dataset_name>` - Get statistics for a specific dataset
+
+Annotation files are saved as JSON files in the specified output directory. Each
+annotation file represents a dataset, with the filename serving as the dataset
+name.
+
+## Acknowledgements
+
+Pigeon was created by [Anastasis Germanidis](https://github.com/agermanidis) and
+released as a [Python package](https://pypi.org/project/pigeon-jupyter/) and
+[Github repository](https://github.com/agermanidis/pigeon). It is licensed under
+the Apache License. It has been updated to work with more recent `ipywidgets`
+versions and some small UI improvements were added. We are grateful to Anastasis
+for creating this tool and making it available to the community.
+
+<!-- For scarf -->
+<figure><img alt="ZenML Scarf" referrerpolicy="no-referrer-when-downgrade" src="https://static.scarf.sh/a.png?x-pxid=f0b4f458-0a54-4fcd-aa95-d5ee424815bc" /></figure>

docs/book/toc.md

@@ -164,6 +164,7 @@
     * [Develop a Custom Feature Store](stacks-and-components/component-guide/feature-stores/custom.md)
   * [Annotators](stacks-and-components/component-guide/annotators/annotators.md)
     * [Label Studio](stacks-and-components/component-guide/annotators/label-studio.md)
+    * [Pigeon](stacks-and-components/component-guide/annotators/pigeon.md)
     * [Prodigy](stacks-and-components/component-guide/annotators/prodigy.md)
     * [Develop a Custom Annotator](stacks-and-components/component-guide/annotators/custom.md)
   * [Image Builders](stacks-and-components/component-guide/image-builders/image-builders.md)

scripts/install-zenml-dev.sh

@@ -32,7 +32,8 @@ install_integrations() {
     # figure out the python version
     python_version=$(python -c "import sys; print('.'.join(map(str, sys.version_info[:2])))")
 
-    ignore_integrations="feast label_studio bentoml seldon pycaret skypilot_aws skypilot_gcp skypilot_azure prodigy"
+    ignore_integrations="feast label_studio bentoml seldon pycaret skypilot_aws skypilot_gcp skypilot_azure pigeon prodigy"
+
     # if python version is 3.11, exclude all integrations depending on kfp
     # because they are not yet compatible with python 3.11
     if [ "$python_version" = "3.11" ]; then

src/zenml/integrations/__init__.py

@@ -17,8 +17,6 @@
 support. This includes orchestrators like Apache Airflow, visualization tools
 like the ``facets`` library, as well as deep learning libraries like PyTorch.
 """
-import sys
-
 from zenml.integrations.airflow import AirflowIntegration  # noqa
 from zenml.integrations.aws import AWSIntegration  # noqa
 from zenml.integrations.azure import AzureIntegration  # noqa
@@ -49,8 +47,9 @@
 from zenml.integrations.neptune import NeptuneIntegration  # noqa
 from zenml.integrations.neural_prophet import NeuralProphetIntegration  # noqa
 from zenml.integrations.openai import OpenAIIntegration  # noqa
+from zenml.integrations.pigeon import PigeonIntegration # noqa
 from zenml.integrations.pillow import PillowIntegration  # noqa
-from zenml.integrations.polars import PolarsIntegration
+from zenml.integrations.polars import PolarsIntegration  # noqa
 from zenml.integrations.prodigy import ProdigyIntegration  # noqa
 from zenml.integrations.pycaret import PyCaretIntegration  # noqa
 from zenml.integrations.pytorch import PytorchIntegration  # noqa

src/zenml/integrations/constants.py

@@ -44,6 +44,7 @@
 NEPTUNE = "neptune"
 NEURAL_PROPHET = "neural_prophet"
 OPEN_AI = "openai"
+PIGEON = "pigeon"
 PILLOW = "pillow"
 PLOTLY = "plotly"
 POLARS = "polars"

src/zenml/integrations/pigeon/__init__.py

@@ -0,0 +1,44 @@
+#  Copyright (c) ZenML GmbH 2024. All Rights Reserved.
+#
+#  Licensed under the Apache License, Version 2.0 (the "License");
+#  you may not use this file except in compliance with the License.
+#  You may obtain a copy of the License at:
+#
+#       https://www.apache.org/licenses/LICENSE-2.0
+#
+#  Unless required by applicable law or agreed to in writing, software
+#  distributed under the License is distributed on an "AS IS" BASIS,
+#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
+#  or implied. See the License for the specific language governing
+#  permissions and limitations under the License.
+"""Initialization of the Pigeon integration."""
+from typing import List, Type
+
+from zenml.integrations.constants import PIGEON
+from zenml.integrations.integration import Integration
+from zenml.stack import Flavor
+
+PIGEON_ANNOTATOR_FLAVOR = "pigeon"
+
+
+class PigeonIntegration(Integration):
+    """Definition of Pigeon integration for ZenML."""
+
+    NAME = PIGEON
+    REQUIREMENTS = ["ipywidgets>=8.0.0"]
+
+    @classmethod
+    def flavors(cls) -> List[Type[Flavor]]:
+        """Declare the stack component flavors for the Pigeon integration.
+
+        Returns:
+            List of stack component flavors for this integration.
+        """
+        from zenml.integrations.pigeon.flavors import (
+            PigeonAnnotatorFlavor,
+        )
+
+        return [PigeonAnnotatorFlavor]
+
+
+PigeonIntegration.check_installation()

src/zenml/integrations/pigeon/annotators/__init__.py

@@ -0,0 +1,20 @@
+#  Copyright (c) ZenML GmbH 2024. All Rights Reserved.
+#
+#  Licensed under the Apache License, Version 2.0 (the "License");
+#  you may not use this file except in compliance with the License.
+#  You may obtain a copy of the License at:
+#
+#       https://www.apache.org/licenses/LICENSE-2.0
+#
+#  Unless required by applicable law or agreed to in writing, software
+#  distributed under the License is distributed on an "AS IS" BASIS,
+#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
+#  or implied. See the License for the specific language governing
+#  permissions and limitations under the License.
+"""Initialization of the Pigeon annotators submodule."""
+
+from zenml.integrations.pigeon.annotators.pigeon_annotator import (
+    PigeonAnnotator,
+)
+
+__all__ = ["PigeonAnnotator"]