Allow tracker to run on more frames automatically

[LucaBallan96]

My actions before raising this issue

• Read/searched the docs
• Searched past issues

Steps to Reproduce (for bugs)

1. Pull repo and start up local CVAT instance
2. Set up SiamMask or TransT tracker as serverless function with Nuclio
3. Load video and annotate first frame

Expected Behaviour

We want a tracker to automatically annotate a number of consecutive frames, once the first one is annotated. Automatic annotation with a detector, for example, works like that.

Current Behaviour

Need to go manually frame by frame to run tracker. That means, click 'next frame' 1 by 1, till the end of the video.
Issue #2949 mentioned this already, but the solution is not clear. Can 'tracker frames' be set with an attribute in function.yaml, or somewhere else?

[dschoerk]

I've been working on the TransT implementation in CVAT. At the time of implementation, the serverless tracking functions were designed to only track one frame at a time. As a result, the UI is sending a web request for each tracked frame and tracking multiple frames would be as efficient as stepping forward manually by holding the 'f' key. A low hanging fruit to improve usability a little bit would be to implement tracking multiple frame when pressing 'v' to step forward x frames at a time. However, this wouldn't improve the efficiency of the tracker.

[laisimiao]

@dschoerk Thank you for your contribution. But after i build and deploy transt, it gives a error when use AI tools:

Tracker initialization error Error:
 Request failed with status code 500. 
"500 Server Error: Internal Server Error for url: http://host.docker.internal:49153/".

Here are my return:

[dschoerk]

@dschoerk Thank you for your contribution. But after i build and deploy transt, it gives a error when use AI tools:

Tracker initialization error Error:
 Request failed with status code 500. 
"500 Server Error: Internal Server Error for url: http://host.docker.internal:49153/".

Here are my return:

Check the docker logs.

[leosteets]

I've been working on the TransT implementation in CVAT. At the time of implementation, the serverless tracking functions were designed to only track one frame at a time. As a result, the UI is sending a web request for each tracked frame and tracking multiple frames would be as efficient as stepping forward manually by holding the 'f' key. A low hanging fruit to improve usability a little bit would be to implement tracking multiple frame when pressing 'v' to step forward x frames at a time. However, this wouldn't improve the efficiency of the tracker.

Hi, thanks for the update there, I was wondering why the 'v' doesn't work like the Tracker with TrackerMIL? When I press 'v' and step 10 frames, the current new frame is not processed with TransT, I don't think it has to process all the 10 frames since the interpolation in my case solves most of the cases.

Do you know how can I make it work?

[dschoerk]

I've been working on the TransT implementation in CVAT. At the time of implementation, the serverless tracking functions were designed to only track one frame at a time. As a result, the UI is sending a web request for each tracked frame and tracking multiple frames would be as efficient as stepping forward manually by holding the 'f' key. A low hanging fruit to improve usability a little bit would be to implement tracking multiple frame when pressing 'v' to step forward x frames at a time. However, this wouldn't improve the efficiency of the tracker.

Hi, thanks for the update there, I was wondering why the 'v' doesn't work like the Tracker with TrackerMIL? When I press 'v' and step 10 frames, the current new frame is not processed with TransT, I don't think it has to process all the 10 frames since the interpolation in my case solves most of the cases.

Do you know how can I make it work?

I believe this is handled in the CVAT UI code, the integration for OpenCV based trackers (like TrackerMIL) is a little different from the Nuclio based AI tools (interactor, tracker, detector).

[hoangdung3498]

@dschoerk Thank you for your contribution. But after i build and deploy transt, it gives a error when use AI tools:

Tracker initialization error Error:
 Request failed with status code 500. 
"500 Server Error: Internal Server Error for url: http://host.docker.internal:49153/".

Here are my return:

Do you resolve it ?

[rohit901]

@dschoerk thank you for the integration. However it seems to take a long time for processing each frame, and also running it frame by frame seems slow. Can you tell me how is this better than the standard tracking with interpolation?

[dschoerk]

@dschoerk thank you for the integration. However it seems to take a long time for processing each frame, and also running it frame by frame seems slow. Can you tell me how is this better than the standard tracking with interpolation?

It's vastly more accurate than interpolation.