-
Notifications
You must be signed in to change notification settings - Fork 112
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
The MOTP metrics is showing a low score. #118
Comments
Have you tried optimizing the tracking thresholds, for example, |
I will do that and check if there is any improvement. Is there any other way to find the optimal tracking thresholds besides manually changing it with presumption? |
You could write a script to find the optimal hyperparameters. But there is no analytic way to find them. At first, I would visualize your outputs to understand whats happening. This could give you an idea what parameters and how to change them. |
Hi, I tried to use Visdom to visualize the training and evaluation metrics as suggested in the documents, but the Visdom server is showing a blank blue screen. I tried to visualize the bounding box location during validation, and the prediction worked quite well. But for the test data, the prediction is random. During the training phase, I only used validation data for tracking and didn't use any test data. The validation data itself is in a sequential format. I wanted the test data to be unseen during training.
|
I managed to load results from visdom. BUt I still can't figure out why the tracking woks on validation data but not the test data. |
Hi, I am using this reference code to do simultaneous detection and tracking on small objects. The average precision and average recall of the detection model shows a good perfromance but when it comes to tracking the MOTP score is very low? Any suggestions on how I can improve the performance ? Also, the MOTA and other metrics are high, so I can't figure out where the prblem actually lies ? @timmeinhardt
IoU metric: bbox
![image](https://private-user-images.githubusercontent.com/45029614/294170823-cf1432df-7311-491a-8029-c40f333a27dd.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MTg4MDIwNTAsIm5iZiI6MTcxODgwMTc1MCwicGF0aCI6Ii80NTAyOTYxNC8yOTQxNzA4MjMtY2YxNDMyZGYtNzMxMS00OTFhLTgwMjktYzQwZjMzM2EyN2RkLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA2MTklMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwNjE5VDEyNTU1MFomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTRiMTczOWE4M2VmNGUxMTFhYjk4ZDc2NWI0YzdlNDI3MTg1ZDU4YTc4NGViY2ZmNjEzOTgyNzlkMGE0NTE5ZWMmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.YEHz1lCS64ZKpo00sYdB66V6LiMmJnseDdTGB0jhRcs)
Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.660
Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.946
Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.808
Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.660
Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = -1.000
Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = -1.000
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.692
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.705
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.705
Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.705
Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = -1.000
Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = -1.000
INFO - root - mergeOverall: 0.022 seconds.
![image](https://private-user-images.githubusercontent.com/45029614/294170691-8fac0827-1185-4d26-9cbe-82ee84a9ab63.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MTg4MDIwNTAsIm5iZiI6MTcxODgwMTc1MCwicGF0aCI6Ii80NTAyOTYxNC8yOTQxNzA2OTEtOGZhYzA4MjctMTE4NS00ZDI2LTljYmUtODJlZTg0YTlhYjYzLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA2MTklMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwNjE5VDEyNTU1MFomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWFhNzM0YzViMjcyNDNjYjJiOTY0MWI1NzFiMjE0ODBjZmU0YTZhYzM1MDg4OGM5MjZjMGEwM2NkNDUzOTM5YWImWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.296QIy0XcnOoe_Xn0auQ58_ueJLpZE8xDNFenCqDU_g)
IDF1 IDP IDR Rcll Prcn GT MT PT ML FP FN IDs FM MOTA MOTP IDt IDa IDm
Train_Moffat 99.0% 100.0% 98.0% 98.0% 100.0% 1 1 0 0 0 10 0 0 98.0% 0.086 0 0 0
OVERALL 99.0% 100.0% 98.0% 98.0% 100.0% 1 1 0 0 0 10 0 0 98.0% 0.086 0 0 0
Currently, I am only using a single object per image but I will change it to multiple objects when I get improvement on a single object tracking.
The text was updated successfully, but these errors were encountered: