Recently I had a task which required tracking detections along with it's mask. I searched the internet for days to find anything that solves my problem but found no tutorial, articles, repository that solved my problem.
Then I found this repository: https://github.com/theAIGuysCode/yolov4-deepsort Actually for object detecion there were many resources for tracking, so I chose the above one and decided to change it to tracking objects along with masks.
This repository will guide you through tracking objects with masks produced by default MaskRCNN in Detectron2 framework. You can train your custom MaskRCNN using Detectron2 easily following this tutorial:
Video 1: https://www.youtube.com/watch?v=ffTURA0JM1Q
Video2: https://www.youtube.com/watch?v=GoItxr16ae8&t=357s
Also here is another helpful repo: https://github.com/joheras/CLoDSA which will help with augmenting images with masks, in case you have limited training data, thats what I used too.
Open the Notebook given in this repo in Google Colab and run it following the instructions there, it's preety straightforward. The file mask_tracker.py is the one that takes care of everything and the details of the codes are properly documented in the code, please follow it for clarification.
Input videos are there in /data/video/
This repo guides you with tracking masked objects using Detectron2 MaskRCNN in detail. This same method can be used to track other Object Detection or Segmentation models built in different frameworks too, some tweaks would be needed, try to figure it out and if some help is needed can raise an Issue or connect with me on linkedIn https://www.linkedin.com/in/rehan-zafar-48797b193/
If this repository helped you in anyway please give it a star.
Thanks,
REHAN ZAFAR