feat(docs): Update documentation for 0.5.0 release (#144)

* feat(docs): New screenshots and updated user guide * fix(docs): Fix incorrect Pose docstrings * fix(docs): Fix outdated module name * fix(docs): Lint API references generator * fix(docs): Remove outdated comment * feat(docs): Update S3 documentation * feat(docs): Add information on pre-annotation * feat(docs): Add information on auto-completion
pixano · Feb 15, 2024 · f41a738 · f41a738
1 parent e0f719a
commit f41a738
Show file tree

Hide file tree

Showing 19 changed files with 101 additions and 74 deletions.
diff --git a/docs/assets/user/annotation_bounding_box.png b/docs/assets/user/annotation_bounding_box.png
diff --git a/docs/assets/user/annotation_polygon.png b/docs/assets/user/annotation_polygon.png
diff --git a/docs/assets/user/annotation_sam_points.png b/docs/assets/user/annotation_sam_points.png
diff --git a/docs/assets/user/annotation_sam_rectangle.png b/docs/assets/user/annotation_sam_rectangle.png
diff --git a/docs/assets/user/app_annotation.png b/docs/assets/user/app_annotation.png
diff --git a/docs/assets/user/app_dashboard.png b/docs/assets/user/app_dashboard.png
diff --git a/docs/assets/user/app_dataset_1.png b/docs/assets/user/app_dataset_1.png
diff --git a/docs/assets/user/app_dataset_2.png b/docs/assets/user/app_dataset_2.png
diff --git a/docs/assets/user/app_elementview.png b/docs/assets/user/app_elementview.png
diff --git a/docs/assets/user/app_home.png b/docs/assets/user/app_home.png
diff --git a/docs/assets/user/exploration_object_edition.png b/docs/assets/user/exploration_object_edition.png
diff --git a/docs/assets/user/exploration_objects.png b/docs/assets/user/exploration_objects.png
diff --git a/docs/assets/user/exploration_scene.png b/docs/assets/user/exploration_scene.png
diff --git a/docs/code/index.md b/docs/code/index.md
@@ -7,7 +7,7 @@ title: Pixano API reference
 Here you will find the documentation for all of our Python API.
 
 - The **_analytics_** module contains useful functions for computing statistics on a dataset.
-- The **_apps_** module contains the Pixano app and its API.
+- The **_app_** module contains the Pixano app and its API.
 - The **_core_** module contains the Pixano custom data types for storing data in the PyArrow tabular format.
 - The **_data_** module contains classes like for interfacing data with the API, as well as the classes for importing and exporting datasets.
 - The **_models_** module contains the base class for inference generation and embedding precomputing models.

diff --git a/docs/gen_ref_pages.py b/docs/gen_ref_pages.py
@@ -30,7 +30,9 @@
         doc_path = doc_path.with_name("index.md")
         full_doc_path = full_doc_path.with_name("index.md")
         nav["index"] = doc_path.as_posix()
-        with open(f"../pixano/docs/{REF_PATH}/index.md", "r") as index_file:
+        with open(
+            f"../pixano/docs/{REF_PATH}/index.md", "r", encoding="utf-8"
+        ) as index_file:
             lines = index_file.readlines()
             with mkdocs_gen_files.open(full_doc_path, "w") as fd:
                 fd.writelines(lines)
@@ -40,8 +42,7 @@
     elif not any(ignored in path.name for ignored in IGNORED_FILES):
         nav[parts] = doc_path.as_posix()
         with mkdocs_gen_files.open(full_doc_path, "w") as fd:
-            identifier = ".".join(parts)
-            print("::: pixano." + identifier, file=fd)
+            print("::: pixano." + ".".join(parts), file=fd)
         mkdocs_gen_files.set_edit_path(full_doc_path, path)
 
 

diff --git a/docs/user/app.md b/docs/user/app.md
@@ -1,97 +1,93 @@
-[comment]: <> (TODO: Update screenshots and descriptions for new Pixano app)
-
 # Using the Pixano app
 
 ## Home page
 
-![Pixano App Home Page](../assets/user/app_home.png)
+![Pixano home page](../assets/user/app_home.png)
 
 From the app home page, you will be greeted with a list of all the Pixano format datasets found in the directory you provided.
 
-For each dataset, you can see its name, the number of items inside it, and a thumbnail composed from six sample items.
+## Dataset page
+
+![Pixano dataset page - top](../assets/user/app_dataset_1.png)
+
+On the dataset page, you will see a list of all the items it contains, organized in pages.
+
+![Pixano dataset page - bottom](../assets/user/app_dataset_2.png)
+
+When scrolling to the bottom of the page, you will see navigation buttons to move through the pages of your dataset.
 
-You can hover over a dataset name to check the dataset description if it has one.
+## Dashboard page
 
-You can click on a dataset to open its annotation page on its first item.
+![Pixano dataset dashboard](../assets/user/app_dashboard.png)
+
+From the dataset page, you can go to the dashboard page, which contains more information about your datasets and also displays all the computed statistics available.
 
 ## Item page
 
-### Item view
+When opening an item, the item media will be displayed in the center on the screen (in case of multi-view datasets, the images will be tiled).
+
+On the left, a toolbar is available. On the right, two panels will display information on the item objects and scene.
+
+### Scene panel
+
+![Pixano item view - scene](../assets/user/exploration_scene.png)
 
-![Pixano App Item View](../assets/user/app_elementview.png)
+The scene panel will display all the scene features, like the item label, or any other feature created when importing your dataset, as well as metadata information on all the images in the item.
 
-The selected element (image or images for multi-view datasets) is displayed.
+You can edit the scene features and then click the save changes button to write them to the dataset.
 
-You can zoom in and out with the mouse wheel.
+### Object panel
 
-You can grab and move images with the middle click or with the **_Pan_** tool available in the left-hand _toolbar_.
+![Pixano item view - objects](../assets/user/exploration_objects.png)
 
-You can double click on an image to move it above other images.
+The objects panel will display all the item objects.
 
-Annotations, in form of segmentation mask, are displayed.
-Each object category is given a color.
+On the top, you will see the ground truth objects, which are the objects you imported with your dataset, and objects you create within the app. On the bottom, a dropdown menu will let you go through all the objects created by models like the ones available in Pixano Inference.
 
-On the top of the image, when you have an input Tool selected, a panel is displayed that allows to choose a category.
+You have visibility toggles for objects and object group, and when hovering on an object, you will have access to an edit tool and a delete tool.
 
-### Left toolbar
+If you have used an inference model for pre-annotating the dataset, a "Pre-annotation" toggle will also appear above the ground truth section. Activating this toggle will let you go through each object and accept or reject them individually. You will also be able to edit the object features before accepting it.
 
-A _toolbar_ is available on the left-hand side of the page with the following tools:
+![Pixano item view - object edition](../assets/user/exploration_object_edition.png)
 
-- **_Pan_**: Allows you to grab and move an image
-- **_Points_**: Allows you to place input points to interactively segment your image
-  - You can place positive points with the **+** tool (points shown in green) to indicate what must be included in the segmentation
-  - You can place negative points with **-** tool (points shown in red) to indicate what must not be included in the segmentation
-  - You can hover over any point and press the _Del_ key to remove it
-  - You can click and hold on any point to relocate it
-- **_Rectangle_**: Allows you to draw rectangles approximatively around the objects of interest to interactively segment them
-  - You can click and drag on the image to draw a rectangle
-  - There can only be one rectangle at a time, so drawing a new rectangle will discard the previous one. You have to validate the obtained segmentation, if satisfactory, before drawing a new rectangle
+The edit tool will allow you to edit the object features, for example its category and category ID, and also allow you to edit the object bounding box and mask on the image. For text features, auto-completion based on existing feature values in the dataset is available.
 
-The **_Points_** and **_Rectangle_** tools depend on an ONNX segmentation model you have to provide. Please look at the interactive annotation documentation and notebook for more information.
+To create new objects, you have multiple tools at your disposal on the left toolbar.
 
-_More tools will be coming soon._
+### Toolbar
 
-### Center toolbar
+#### Pan tool
 
-When an annotation tool is selected, a _toolbar_ is available at the center on the page.
+With the pan tool selected, you can move the image around. This is especially useful for multi-view datasets for organizing multiple images.
 
-Enter the label name for your annotation in the text box or select the label from the list of existing labels.
+Moving the images is still possible while any other tools is selected by using your mouse middle click. You can also zoom in and out of an image with the mouse wheel, and double click an image to bring it in front of the others.
 
-If the **_Points_** tool is selected, a **+** icon and a **-** icon allow to quickly switch between positive and negative points
+#### Bounding box tool
 
-Validate your annotation with the entered label by cliking on the Validate icon, or press _Enter_ key.
+![Pixano tools - bounding box](../assets/user/annotation_bounding_box.png)
 
-### Right toolbar
+With the bounding box tool, you can create a bounding box object by click and dragging a rectangle over the image. Once you are done with your selection, you will be prompted to enter values for your object features depending on your dataset (in this case category_id and category), and to confirm the object.
 
-A _toolbar_ is available on the right side of the page with the following tabs:
+Then, click save changes in the object panels to save the created object to your dataset.
 
-- **_Labels_**: This tab displays your annotations grouped by views and by labels
-  - Each annotation group can be opened or closed by clicking on it
-  - Each annotation and annotation group can be shown or hidden on the relevant image by clicking on the _Visibility_ icon
-  - Each annotation can be deleted by clicking on the _Delete_ icon
-  - Each annotation is represented by its unique ID.
-- **_Dataset_**: This tab allows you to navigate through the dataset
-  - Each element of the dataset is displayed with its ID and its thumbnail
-  - The list will automatically expand as you scroll down
-  - You can click on any element to change the current element
+#### Polygon tool
 
-### Annotating
+![Pixano tools - polygon](../assets/user/annotation_polygon.png)
 
-![Pixano App Annotation](../assets/user/app_annotation.png)
+With the polygon tool, you can create a segmentation mask manually by adding points with the granularity of your choice.
 
-You can currently annotate with the **_Points_** (**+** and **-**), and **_Rectangle_** tools available in the left toolbar as described above.
+Once you save this mask, a matching bounding box will automatically be created.
 
-When using these tools, the generated annotation will be displayed in green. You can use both tools together to refine your annotation.
+#### Smart segmentation tool
 
-You can press the _Enter_ key or click the Validate icon of the center toolbar to validate your annotation.
-You can press the _Esc_ key to reset all your **_Points_** and **_Rectangle_** inputs.
+With Pixano, you can segment with smart segmentation tool like SAM (Segment Anything Model). Please follow our documentation on how to precompute the embeddings required by SAM and export its ONNX model to be able to use it.
 
-### Saving
+![Pixano tools - SAM points](../assets/user/annotation_sam_points.png)
 
-To save your annotations, a _Save_ icon is available in the top right-hand corner. It will be highlighted if there are unsaved changes.
+With the positive and negative points, you can inform SAM on which part of the image you are trying to segment, and SAM will generate the mask for you.
 
-If you try to go back to the home page or change element with unsaved changes, you will see a confirmation window. You can choose "OK" to discard your changes, or cancel to be able to go back save them.
+![Pixano tools - SAM rectangle](../assets/user/annotation_sam_rectangle.png)
 
-### Going home
+When relevant, you can also use the rectangle tool to select the thing you want SAM to segment.
 
-To go back to the home page, click on the Pixano logo in the top left corner.
+When saving the mask created by SAM, like with the polygon tool, a matching bounding box will automatically be created.
diff --git a/docs/user/launch.md b/docs/user/launch.md
@@ -2,33 +2,42 @@
 
 ## From a terminal
 
-You can start the Pixano app with the following commands:
+You can start the Pixano app with the following command:
 
 ```shell
-pixano <path/to/your/datasets>
+pixano your_dataset_directory/
 ```
 
 You will then be provided with a URL to open in your browser to use the app.
 
-### With S3 compatible storage
+Note that you can also connect to an S3 compatible storage by providing an S3 path instead of a local path to your datasets.
 
-You can connect to a S3 compatible storage, by providing a S3 path instead of local path to your datasets
+The following arguments have to be passed:
 
-The followings environ variables have to be set:
+- `--aws_endpoint`: S3 endpoint URL, use 'AWS' if not provided.
+- `--aws_region`: S3 region name, not always required for private storages.
+- `--aws_access_key`: S3 AWS access key.
+- `--aws_secret_key`: S3 AWS secret key.
+- `--local_model_dir`: Local path to your models.
 
-- AWS_ENDPOINT : S3 Compatible Storage endpoint
-- AWS_ACCESS_KEY_ID : access key credentials
-- AWS_SECRET_ACCESS_KEY : secret access credentials
-- AWS_REGION (optionnal): S3 region if relevant
-- LOCAL_MODEL_DIR: local path to model directory
+So the command becomes:
+
+```shell
+pixano s3://your_dataset_directory/ \
+--aws_endpoint="https://your-aws-endpoint.com" \
+--aws_region="" \
+--aws_access_key="your_access_key" \
+--aws_secret_key="your_secret_key" \
+--local_model_dir="your_local_onnx_models/"
+```
 
 ## From a notebook
 
 If you are in a Jupyter or Google Colab notebook, you can start the app by running the following cells:
 
 ```python
 from pixano.app import App
-app = App(<path/to/your/datasets>)
+app = App("your_dataset_directory/")
 ```
 
 You can then use the apps directly from the notebook in another cell with:

diff --git a/pixano/app/README.md b/pixano/app/README.md
@@ -18,18 +18,39 @@
 You can start the Pixano app with the following command:
 
 ```shell
-pixano <path/to/your/datasets>
+pixano your_dataset_directory/
 ```
 
 You will then be provided with a URL to open in your browser to use the app.
 
+Note that you can also connect to an S3 compatible storage by providing an S3 path instead of a local path to your datasets.
+
+The following arguments have to be passed:
+
+- `--aws_endpoint`: S3 endpoint URL, use 'AWS' if not provided.
+- `--aws_region`: S3 region name, not always required for private storages.
+- `--aws_access_key`: S3 AWS access key.
+- `--aws_secret_key`: S3 AWS secret key.
+- `--local_model_dir`: Local path to your models.
+
+So the command becomes:
+
+```shell
+pixano s3://your_dataset_directory/ \
+--aws_endpoint="https://your-aws-endpoint.com" \
+--aws_region="" \
+--aws_access_key="your_access_key" \
+--aws_secret_key="your_secret_key" \
+--local_model_dir="your_local_onnx_models/"
+```
+
 ### From a notebook
 
 If you are in a Jupyter or Google Colab notebook, you can start the app by running a cell with:
 
 ```python
 from pixano.app import App
-app = App(<path/to/your/datasets>)
+app = App("your_dataset_directory/")
 ```
 
 You can then use the app directly from the notebook in another cell with:

diff --git a/pixano/core/pose.py b/pixano/core/pose.py
@@ -21,7 +21,7 @@ class Pose(PixanoType, BaseModel):
     """Pose type using orientation and translation matrices
 
     Attributes:
-        _cam_r_w2c (list[float]): 3*3 orientation matrix
+        _cam_r_m2c (list[float]): 3*3 orientation matrix
         _cam_t_m2c (list[float]): 3*1 translation matrix
     """
 
@@ -32,7 +32,7 @@ def __init__(self, cam_r_m2c: list[float], cam_t_m2c: list[float]):
         """Initialize pose from orientation and translation matrices
 
         Args:
-            cam_r_w2c (list[float]): 3*3 orientation matrix
+            cam_r_m2c (list[float]): 3*3 orientation matrix
             cam_t_m2c (list[float]): 3*1 translation matrix
         """