-
-
Notifications
You must be signed in to change notification settings - Fork 5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
SOTA #215
Comments
@bakeryproducts you can go through this medium blog for understanding of the SOTA comparison and results |
@bakeryproducts @HarishGuragol we'll release comparison plots soon with exact commands to reproduce. There are multiple tasks so it takes comparatively more time. Please be patient. |
@bakeryproducts SOTA comparisons added in #260 |
No way professionals like Ultralytics claiming 'new standards in SOTA real-time detection and segmentation' without proper comparison on different resolutions, datasets, batch-sizes, GPU-archs. Surely Ultralytics got better SOTA comparisons then two .png files of accuracy plots vs other YOLO models.
Whats YOLOv8 highest AP on COCO val with 30 FPS or higher (on A100 GPU as you did in #230)? |
@bakeryproducts yes, that's the standard to compare mAP vs speed on coco test set. Every repository does that and thats the main plot. We've provided size comparison too. And more plots are coming in the future. And all these plots will be available in the paper too. FPS too can be calculated using the latency. We need to prioritize bug reports too. Regarding 56.8 map, those are p6 models. That's a different family of models. Base models (trainer at 640 pixel) and larger p6 models( trained at 1280 pixels). 53.9 is the highest-ever map achieved by the base model still being faster and smaller than the next model in the line. |
@glenn-jocher totally agree with @bakeryproducts on that. According to the numbers you're still beyond the Yolo v7 from the paper above. Comparing only to your own results and calling it a "SOTA" is a strong exaggeration |
So right now highest AP is 53.9 (2.9p behind real-time YOLOv7 56.8) and you are already willing to call that 'setting new standards in SOTA'? Are there any results at higher resolution for v8? |
@bakeryproducts hi actually the highest model we got is 57.0 with resolution 1280. But we noticed that the settings we used for the model is suboptimal after the training finished, so the 57.0 is suboptimal. We think it's not ready to release if it's suboptimal. And we don't have enough time to train another before release the code. Thanks for the attention. :) We'll release the highest model soon. |
has there been a followup here @bakeryproducts ? |
Hey @wesboyt, no, but when the Ultralytics team says "soon," it could take years. No need to worry though! It's been only a year since SOTA claims, and we've been waiting for YOLO papers from them for about three years i think. fingers crossed! |
Search before asking
Question
SOTA results have been announced multiple times in the repository and on the website. What are these SOTA results you managed to beat and is there a comparison with current SOTA?
Additional
No response
The text was updated successfully, but these errors were encountered: