Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
151 changes: 77 additions & 74 deletions docs/Annotation-QuickStart.md
Original file line number Diff line number Diff line change
@@ -1,131 +1,134 @@
# Annotation Quickstart

[Single Frame Detections](#single-frame-detections) - creating multiple detections on a single image frame quickly

[Track Annotations](#track-annotations) - How to quickly create track annotations for a video or image sequence

* [Interpolation Mode](#interpolation-mode) - fastest and easiest way to generate track annotations
* [Advance Frame Mode](#advance-frame) - This mode keeps you editing the same track while automatically advancing the frame each time a detection is drawn. In most cases the interpolation mode will be preferred.

[Polygon Annotations](#polygon-annotations)

[Head/Tail Annotations](#head-tail-annotations)

[Important Keyboard Shortcuts](#important-keyboard-and-mouse-shortcuts)
Before following the quickstart, it could be helpful to skim the [User Interface Guide](Annotation-User-Interface-Overview.md)

## Single Frame Detections

1. Click the Settings wheel next to the '+Track' Icon
1. ![Add Track Button](images/CreationMode/AddTrackButton.png)
1. From the dropdown choose “Detection” mode.
1. ![Creation Mode Settings](images/CreationMode/CreationModeDetectionSettings.png)
1. Choose a type from the dropdown or type a name that you want all detections to inherit. If the type doesn't exist just type the name to create a new one. You should see the "+Detection" button change color to match the type. Now if you hover over the button it will display the default type.
1. ![Detection Settings](images/CreationMode/CreationModeTrackType.png)
1. Turn on continuous mode if you would like to keep dragging the mouse to create detections.
1. Create your first detection by clicking and dragging to generate a rectangle.
1. If you are in continuous mode just click and drag again to create the next detection
1. Otherwise edit the detection and then hit the “N” key on the keyboard or click on detection to create another detection.
How to quickly create multiple detections on a single image frame.

1. Click ==:material-cog:== (creation settings menu) in the [Track List](UI-Track-List.md) area.
1. From the **Mode** dropdown, choose **Detection**.
1. From the **Type** dropdown, choose or enter a default name that all new detections will have. If the type doesn't exist yet, enter a name to create a new one.
1. Turn on the **Continuous Mode** switch if you would like to automatically re-enter the creation state so you can click-and-drag repeatedly to quickly create many detections.
1. Enter the annotation creation state by clicking ==:material-plus: Detection== or pressing the ++n++ key.
1. Create your first detection by clicking and dragging to draw a rectangle.
1. If you are in continuous mode, click and drag again to create the next detection.
1. Press ++escape++ to exit continuous creation mode.

### Single Detection Mode Demo

The Demo below shows how to use Detection mode to quickly create numerous detections of the same type.
The demo below shows how to use Detection mode to quickly create numerous detections of the same type.

![Detection Mode Demo](videos/CreationModes/CreationModeDetection.gif)

## Track Annotations

How to quickly create track annotations for a video or image sequence.

### Interpolation Mode

1. Next to the "+Track" button click the Settings Icon to open the Creation Settings
1. From the dropdown make sure that “Track” mode is selected.
1. Also ensure that interpolation is turned on.
1. ![Interpolate Settings](images/CreationMode/CreationModeInterpolate.png)
1. Either click “+Track” or hit “N” on the keyboard to create the initial track.
1. You can now advance through frames and an outline of the previous annotation will remain. You can advance either by using Left Arrow/Right Arrow or clicking on the timeline to a specific position.
1. To set another frame you only need to move or resize the transparent annotation.
1. There are controls on for the currently selected track to add/remove keyframes. The “Star” icon will allow you to add and remove keyframes. The "interpolate icon" (two overlapped dashed rectangles) will turn on/off interpolation for the current region you are in.
Linear interpolation is a kind of spatio-temporal annotation that allows the inference of bounding boxes between keyframes. Interpolation mode is the fastest and easiest way to generate track annotations.

Interpolation editing for existing tracks will only be enabled on tracks that span more than one frame. It is enabled on new tracks by default.

1. Click ==:material-cog:== (creation settings menu) in the [Track List](UI-Track-List.md) area.
1. From the **Mode** dropdown, choose **Track**.
1. Also ensure that the **Interpolate** switch is turned **on**.
1. Enter the annotation creation state by clicking ==:material-plus: Track== or pressing the ++n++ key.
1. Create your first detection by clicking and dragging to draw a rectangle around the object you want to track.
1. You can now go forward one or more frames by pressing ++f++ or ++arrow-right++ or by using the [Timeline controls](UI-Timeline.md) and an outline of the previous annotation will remain.
1. ![Interpolated Editing](images/InterpolatedEditing.png)
1. To set another keyframe, either move or resize the transparent annotation or press ++k++. There are also controls on for the currently selected track to add/remove keyframes.
1. ==:material-star:== and ==:material-star-outline:== will allow you to add and remove the current keyframe.
1. ==:material-vector-selection:== and ==:material-selection-off:== will turn on or off interpolation for the current keyframe interval region you are in.
1. ![Interpolate Track Settings](images/CreationMode/InterpolateTrackSettings.png)
1. During editing there is a view within the event viewer which allows you to see where interpolation occurs and where the keyframes are located. If a line is not connecting two keyframes in a track then interpolation is disabled for that section.
1. ![Interpolate Event Viewer](images/Timeline/EventViewerInterpolatedTrack.png)

### Interpolation Mode Demo
### Visualizing interpolated tracks

The Demo below shows how to use Interpolation mode to quickly crate tracks for objects through a video/image-sequence.
Click ==Events== in the [Timeline controls](UI-Timeline.md) to see where interpolation occurs and where the keyframes are located.

![Detection Mode Demo](videos/CreationModes/CreationModeInterpolation.gif)
* **Keyframes** are indicated by solid rectangular blue tick marks in the highlighted track.
* **Interpolated regions** are indicated by a thin yellow line between keyframes.
* **Gap regions** are indicated by areas with neither interpolated frames nor keyframes. Typically means that a track is off-camera or occluded.

![Interpolated EventViewer](images/Timeline/EventViewerInterpolatedTrack.png)

### Advance Frame
### Interpolation Mode Demo

![Detection Mode Demo](videos/CreationModes/CreationModeInterpolation.gif)

1. Next to the "+Track" button click the Settings Icon to open the Creation Settings
1. From the dropdown make sure that “Track” mode is selected.
### Advance Frame Mode

1. ![Interpolate Settings](images/CreationMode/CreationModeAdvanceFrame.png)
This mode keeps you editing the same track while automatically advancing the frame each time a detection is drawn. In most cases **interpolation mode will be easier**.

1. Turn off interpolation and turn on "AdvanceFrame"
1. Now each time an individual detection is drawn the frame will automatically advance to the next frame.
1. Click ==:material-cog:== (creation settings menu) in the [Track List](UI-Track-List.md) area.
1. From the **Mode** dropdown, choose **Track**.
1. Also ensure that the **Interpolate** switch is turned **off**.
1. Enter the annotation creation state by clicking ==:material-plus: Track== or pressing the ++n++ key.
1. Create your first detection by clicking and dragging to draw a rectangle around the object you want to track.
1. Now each time an individual detection is drawn the frame will automatically advance to the next frame. Press ++escape++ to end creation of the track.

### Advance Frame Mode Demo

The Demo below shows how to use AdvanceFrame mode to travel through the video while creating annotations.
The demo below shows how to use AdvanceFrame mode to travel through the video while creating annotations.

![Detection Mode Demo](videos/CreationModes/CreationModeAdvanceFrame.gif)

## Head Tail Annotations

**Head/Tail creation on an existing Track:**

1. Either create a new detection/track or select an existing one with “Left Click”
1. When a track is selected Head/Tail annotations can be added by either going to the edit menu and selecting the line tool at the top of screen or by:
### Adding Head/Tail points to existing annotations

* H Key - Create Head annotation
* T Key - Create Tail annotation
1. **Right-click** an existing detection to enter edit mode.
1. Enter head/tail creation mode
1. In the [Edit bar](UI-Navigation-Editing-Bar.md), click ==:material-vector-line:==
1. Or Press ++h++ to create a head point.
1. Or press ++t++ to create a tail point.
1. The mouse cursor will become a crosshair. Click in the annotator to place each point.
1. Once the first marker is placed it automatically transitions to the second marker. If you start with head, the second one will be the tail and vice versa.

1. After entering head/tail annotation mode the cursor becomes a crosshair and the the head or tail can be placed.
1. Once the first marker is placed it automatically transitions to the second marker. (If you start with head, the second one will be the tail and vice versa if you start with tail)
### Creating new annotations using Head/Tail points

**Creating New Track from Head/Tail Annotation:**
You can create a track by starting with a head/tail annotation or just a single point.

1. After clicking "+Track" you can switch the editing mode into the line tool.
1. ![Line Tool](images/CreationMode/LineTool.png)
1. This will allow you place a single point initially by clicking
1. You can then either hit escape and a bounding box will appear around that point
1. Or you can place the second point and a new bounding box will be created which can be edited.
1. Enter the annotation creation state by clicking ==:material-plus: Track== or pressing the ++n++ key.
1. In the [Edit bar](UI-Navigation-Editing-Bar.md), click ==:material-vector-line:== to switch to head/tail creation mode or press ++h++, ++t++, or ++3++.
1. The mouse cursor will become a crosshair. Click in the annotator to place each point.
1. Press ++escape++ to finish creation after one or both points have been placed.

Notes:
### Other notes on Head/Tail

* The head point is denoted by a filled circle, while the tail point is denoted by a hollow circle.

![Tail Mode](images/CreationMode/HeadTail.png)

* You can also start with the tail: tap `t`
* ![Tail Mode](images/CreationMode/TailMode.png) - Indicates you are in tail mode and are going to place the tail point.
* You don't have to place both markers. Press `Escape` on your keyboard at anytime to exit out of the line creation mode.. Your cursor will change back to a pointer.
* You can modify an existing head/tail marker by placing the annotation into 'Edit Mode' and then selecting the line tool from the editing options.
* **To Delete a head/tail pair**, select a detection with existing markers and have it in line edit mode. This will allow you click the Delete button at the top to delete both points.
* You don't have to place both markers. Press ++escape++ on your keyboard at anytime to exit out of the line creation mode.
* You can **modify** an existing head/tail marker by placing the annotation into 'Edit Mode' and then selecting the line tool from the editing options.
* You can **delete** a head/tail pair by selecting a detection with existing markers, entering edit mode, and clicking ==Delete Linestring :material-delete:=={ .error }

## Fish Head Tail Demo
### Fish Head Tail Demo

![Head Tail Annotation Demo](videos/HeadTailAnnotation.gif)

## Polygon Annotations

While every track is required to have a bounding box a polygon can be added as well for more detailed annotations of objects. When a polygon is created or edited it will adjust the bounding box to fit the size of the polygon.
Every track is required to have a bounding box, but a polygon region may be added. When a polygon is created or edited it will generate or adjust the bounding box to fit the size of the polygon.

### Polygon Creation

1. Click on "+Track" or hit the 'N' key then select the polygon tool - ![Polygon Mode](images/CreationMode/PolygonTool.png)
1. Start by placing the first point and then start placing additional points for the polygon
1. **NOTE:** You can hold down the mouse to draw points quickly
1. You can either close the polygon manually or "Right-Click" to automatically close the polygon
1. Enter the annotation creation state by clicking ==:material-plus: Track== or pressing the ++n++ key.
1. In the [Edit bar](UI-Navigation-Editing-Bar.md), click ==:material-vector-polygon:== or press ++2++ to enter polygon creation mode.
1. Place each point on the polygon by clicking.
1. **Right-Click** to automatically close the polygon or press ++escape++ to cancel creation.

### Polygon Editing

1. In edit mode for a polygon you can select any large circle to move it. This will move the point to a new position and recalculate the bounding box.
1. The smaller circles allow you you to create new points which can be used to adjust the polygon and make it appear smoother.
1. A single click on a point will select a point which can then be deleted by using the delete button
1. ![Delete Point](images/EditBar/deletePointMenu.png)
1. **Right click** an annotation to enter edit mode.
1. In the [Edit bar](UI-Navigation-Editing-Bar.md), click ==:material-vector-polygon:== or press ++2++ to enter polygon edit mode.
1. Click and drag any large circle handle to move it. This will move the point to a new position and recalculate the bounding box.
1. Click and drag any small circle handle to create new points. This can be used to adjust the polygon and make it appear smoother.
1. To **delete the whole polygon**, in the [Edit bar](UI-Navigation-Editing-Bar.md), click ==Del polygon :material-delete:=={ .error }
1. To **delete a single keypoint**, click its handle then click ==Del Point N :material-delete:=={ .error }

## Polygon Demo
### Polygon Demo

![Polygon Demo](videos/CreationModes/PolygonAnnotation.gif)
24 changes: 4 additions & 20 deletions docs/Annotation-User-Interface-Overview.md
Original file line number Diff line number Diff line change
@@ -1,28 +1,12 @@
# Annotation User Interface Overview
# User Interface Guide Introduction

The User Interface for the Annotation Editor is broken into a few main areas:
This documentation section provides a reference guide to the annotation interface organized by screen region.

![UI Full View Highlighted](images/UIView/UIFullViewHighlight.png)

* **[Navigation Bar](UI-Navigation-Bar.md)** - Controls to return back to browser as well as perform higher level functions such as running pipelines. Also the ability to save annotations to the server.
* **[Edit Bar](UI-Edit-Bar.md)** - Controls the viewing of annotations on screen and allows for the editing/creation of annotations.
* **[Navigation and Editing Bar](UI-Navigation-Editing-Bar.md)** - Controls to return back to browser as well as perform higher level functions such as running pipelines. Save Button. Controls the viewing of annotations on screen and allows for the editing/creation of annotations.
* **[Annotation View](UI-Annotation-View.md)** - where the image/video is displayed as well as all annotations
* **[Type List](UI-Type-List.md)** - A list of all the types of tracks/detections on the page that can be used to filter the current view.
* **[Track List](UI-Track-List.md)** - List of all the tracks as well as providing a way to perform editing functions on those tracks.
* **[Timeline](UI-Timeline.md)** - timeline view of tracks and detections, as well as an interface to control the current frame along the video/image-sequence
* **[Attributes](Attributes.md)** - Attributes panel used to assign attributes to individual tracks or detections.

## Concepts and Definitions

**Detection** - A single slice of a track at a point in time. Unusually the point of time is a frame of a video or a sequence of images.

**Features** - Bounding Rectangle, Head/Tail or other visible elements of a detection.

**Track** - A collection of detections spanned over multiple frames in a video or image sequence. Tracks include a start and end time and can have periods in which no detections exist.

**Types** - a set of tracks that share specific display properties including color, line thickness, opacity.
Frame - a single image or point in time for a video or image sequence

**KeyFrame** - While using interpolation for tracks these are indicated as a locked position. Interpolation is calculated linearly between keyframes to draw the bounding rects.

**Interpolation** - linearly moving the bounding rect of a detection over time between two keyframes.
* **[Attributes](UI-Attributes.md)** - Attributes panel used to assign attributes to individual tracks or detections.
Loading