How to get MOT metrics on custom test set?

[MKiremitci]

Hi everyone and @cheind,

Thank you for this great jobs. I am newbie and I trained an object detector to detect people from top-view. Now, I want to deal with Multiple Object Tracking, but I don't know how can I get the MOT results. I think lots of people are using the MOT Challange's datasets to evaluate their Multiple Object tracking methods. However, when I look to MOT Challanges, there is no top-view people datasets to evaluate my method. Also, I have already gather top-view human videos to use with tracking.

Therefore, I want to know how should I prepare the test set? How should I annotate? Should test set contain images or videos?

If you answer these questions with basic language to me, I will be too happy. Thanks for your help.

[khalidw]

@MKiremitci Hi! I am also working on a similar problem. I have a custom made dataset of boats for which I am trying to calculate MOT metrics using two different tracking algorithm.

I have annotated the dataset myself, as I understand for custom dataset you need to manually create a ground truth file as per MOT16 or equivalent format.

I am generating a text file of all the tracks in MOT16 format for each tracker.

Once I have both ground truth and tracker files, I use them to generate the metric. I hope this is helpful to you

[cheind]

Hey, @MKiremitci you might want to look at the documentation of

https://github.com/cheind/py-motmetrics/blob/develop/motmetrics/apps/eval_motchallenge.py

which gives you an idea of how to structure files so that you can use the script for evaluation.

[MKiremitci]

@khalidw and @cheind Thanks for your answers. :)

[ImSuMyatNoe]

@MKiremitci Hi! I am also working on a similar problem. I have a custom made dataset of boats for which I am trying to calculate MOT metrics using two different tracking algorithm.

I have annotated the dataset myself, as I understand for custom dataset you need to manually create a ground truth file as per MOT16 or equivalent format.

I am generating a text file of all the tracks in MOT16 format for each tracker.

Once I have both ground truth and tracker files, I use them to generate the metric. I hope this is helpful to you

@khalidw
Hello! I would like to know how to measure with MOT metric on custom dataset! If you have finished the project, could you please let me know how to measure it?

[khalidw]

@ImSuMyatNoe of course. I can share sample code if you can connect on my email address.

[ImSuMyatNoe]

Hello Khalid! Greetings to you. I am Su Myat Noe who commented on your github repo related with "[cheind/py-motmetrics] How to get MOT metrics on a custom test set? (#131)". Sorry for taking a long time to reply to your email. If you have time, could you please share about the " motometrics library" to my email . ***@***.*** Thank you very much for your time and consideration. Have a nice day! Best regards, Su Myat Noe

…

On Sun, Mar 20, 2022 at 2:16 AM Khalid Waleed ***@***.***> wrote: @ImSuMyatNoe <https://github.com/ImSuMyatNoe> of course. I have used motmetrics library for that. I can share sample code if you can connect on my email address. — Reply to this email directly, view it on GitHub <#131 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AEWYCEAPFHSVITF2LAPC2ULVAYDXLANCNFSM4UZLI5CA> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[khalidw]

@ImSuMyatNoe Please see your gmail inbox. I have forwarded you the ipynb notebook. If you have any queries do let me know over email

[myuvarajmp]

@khalidw could you please share the notebook to me as well. mail id:m.yuvarajmp@gmail.com

[cheind]

@khalidw if that notebook is of potential interest to many people, would you mind making it publicly available? I'd be happy to put a link to it online

[khalidw]

@cheind Yes I would. Will share in a short while

[cheind]

@khalidw, nice, maybe add PR of readme.md as well? (make sure to base it on develop)

[myuvarajmp]

@khalidw Please share the notebook asap. sorry for the inconvenience.

[cheind]

@myuvarajmp, I've seen @khalidw has already forked the project and is probably already working on a solution. Please be patient.

[khalidw]

@cheind Hi. I have committed a section titled "For custom dataset" on the readme.md. This section has the code and guide on how to use pymotmetrics for custom dataset. Please review and let me know if any changes are required

[cheind]

@myuvarajmp , @everyone would you mind following @khalidw docs on how to use it on custom datasets and check if you run into any problems? Please see #167 for the PR and report there if possible. thanks