Implementation of a serverless tracking function with support for tracking by masks and polygons.

[DaniilKrich]

Actions before raising this issue

• I searched the existing issues and did not find anything similar.
• I read/searched the docs

Is your feature request related to a problem? Please describe.

I am interested in implementing the Track-Anything tool in CVAT for ai tracking, but I've encountered a limitation where CVAT does not support tracking for masks or polygons. Currently, the input and output data for AI tracking are confined to bounding boxes (bboxes) only.

Additionally, there's an issue with the tracking in the serverless function setup, where only the image is passed without any additional contextual information, such as frame numbers. This lack of information restricts potential functionalities in implementation.
example:
image: /9j/4AAQSkZJRgABAgAAAQABA...UAAAaZURVZaBGW2AQAV//2Q==
shapes: [
[
737,
686,
742,
702
]
]...
state:[...]

And, I have found that it is not possible to track a pre-defined area, whether it has been prepared manually, or via a detector or interactor. This is a significant limitation in the context of our work.

Could we consider enhancing the system to support tracking masks and polygons, and maybe provide more contextual data in the serverless function call? These changes could greatly improve the capabilities and versatility of CVAT

[bsekachev]

Polygon tracks are supported.
Mask tracks are not supported and we are not planning to implement it currently.