Welcome to our Computer Vision project.
Abstract. In this work, we introduce the panoptic segmentation problem of image, its evaluation metric, and provide the 2020 SOTA Detection Transformer (DETR) as the optimal solution. Then explain explicitly how does DETR performs object detection: the Transformer architecture, the process of extracted image's feature transformation to Transformer's digestible form, the queries for questioning detected object and the loss computation, also the optimization algorithm. In order to deploy panoptic segmentation, the panoptic head is added to notice each detected object and render the segmentation version of image. Finally, we clarify its powerful properties by implement COCO-2017 pretrained model and apply it to solve Wheat Head detection problem.
Keywords: detection, segmentation, detr, global-wheat-detection.
⭐ For the detail of our report, see Approaching Object detection and Panoptic segmenation problem by DETR.
⭐ Slide of report can be found here.
⭐ Pre-trained model can be found here.
⭐ We add the code-only version of Wheat head dataset EDA. You can try yourself by running the script.
-
For the implementation of COCO-2017 pretrained model. We install and demonstrate its performance by plotting the prediction (included visualization of attention mechanism). The work can be found at Implement pretrained DETR for object detection and panoptic segmentation
-
We train the pre-trained DETR model to get a prediction of wheat in Wheat Head dataset. This is a really hard problem, our training model have a acceptable result. See Implement DETR for Wheat detection for full detail.
- We add hand-craft annotation of the test images, build an mAP evaluation metric and test on it.
- We train a new DETR by freezing pre-trained backbone (the whole backbone/part of the backbone) for wheat detection task above, sadly it got worse result (evaluate by mAP) 😅. So we keep the best model as the un-frozen model.
Object detection of wheat in Wheat Head dataset
Our mAP scores
Result on confident threshold: 0.9
| No. image | AP | No. image | AP |
|---|---|---|---|
| 1 | 0.345 | 6 | 0.148 |
| 2 | 0.467 | 7 | 0.636 |
| 3 | 0.431 | 8 | 0.407 |
| 4 | 0.529 | 9 | 0.514 |
| 5 | 0.391 | 10 | 0.493 |
| mAP | 0.4365 |