GSoC'22 Multi-tasking computer vision model: object detection, object segmentation and human pose detection #76

obtx · 2022-07-29T04:17:43Z

No description provided.

fengyuentau · 2022-08-02T06:41:41Z

Please read comments carefully below:

The pull request multitask-centernet #74 will be closed. Update your code at this pull request.
Please use Git properly. You need to use git-lfs to push the your model into this pull request.
Please read Contribution Guidelines. Based on what you have in this pull request, I can say there are many things done in a wrong way. First of all, the model directory is put in a wrong location; the model is not named properly; there is a lot of unrelated code. Please, read the guideline and learn from previous pull requests.

…k_centernet/README.md

fengyuentau

You need to move 模型/multitask_center/README.md to models/multitask_centernet and remove 模型/multitask_center.

By the way, please use git in terminal instead of web pages.

And, please, see my comments below.

models/multitask_centernet/demo.py

…k_centernet/README.md

fengyuentau

Could you invite me as a collaborator in your fork of opencv_zoo? I will try to upload the model for you on my side.

models/multitask_centernet/LICENSE

models/multitask_centernet/demo.py

fengyuentau · 2022-09-03T03:56:23Z

models/multitask_centernet/demo.py

+if __name__ == "__main__":
+    parser = argparse.ArgumentParser()
+    parser.add_argument('--imgpath', type=str, default='images/d2645891.jpg', help="image path")
+    parser.add_argument('--modelpath', type=str, default='MCN.onnx')
+    args = parser.parse_args()
+
+    mcn = MCN(args.modelpath)
+    srcimg = cv2.imread(args.imgpath)
+    srcimg = mcn.detect(srcimg)
+    cv2.imwrite('result.png', srcimg)
+
+
+    # winName = 'using MCN in OpenCV'
+    # cv2.namedWindow(winName, 0)
+    # cv2.imshow(winName, srcimg)
+    # cv2.waitKey(0)
+    # cv2.destroyAllWindows()


A demo with webcam stream as input should be provided as well. You can find a example here.

models/multitask_centernet/multitask_centernet.py

fengyuentau · 2022-09-03T04:00:18Z

models/multitask_centernet/multitask_centernet.py

+        img, newh, neww, padh, padw = self.resize_image(srcimg)
+        blob = cv2.dnn.blobFromImage(img, scalefactor=1 / 255.0, swapRB=True)
+        # blob = cv2.dnn.blobFromImage(self.preprocess(img))
+        # Sets the input to the network


Move these lines into preprocess and call preprocess.

fengyuentau · 2022-09-03T04:00:53Z

models/multitask_centernet/multitask_centernet.py

+        # inference output
+        row_ind = 0
+        for i in range(self.nl):
+            h, w = int(self.inpHeight / self.stride[i]), int(self.inpWidth / self.stride[i])
+            length = int(self.na * h * w)
+            if self.grid[i].shape[2:4] != (h, w):
+                self.grid[i] = self._make_grid(w, h)
+
+            outs[row_ind:row_ind + length, 0:2] = (outs[row_ind:row_ind + length, 0:2] * 2. - 0.5 + np.tile(
+                self.grid[i], (self.na, 1))) * int(self.stride[i])
+            outs[row_ind:row_ind + length, 2:4] = (outs[row_ind:row_ind + length, 2:4] * 2) ** 2 * np.repeat(
+                self.anchor_grid[i], h * w, axis=0)
+
+            self.num_coords = outs.shape[1] - self.last_ind
+            outs[row_ind:row_ind + length, self.last_ind:] = outs[row_ind:row_ind + length, self.last_ind:] * 4. - 2.
+            outs[row_ind:row_ind + length, self.last_ind:] *= np.tile(np.repeat(self.anchor_grid[i], h * w, axis=0), (1, self.num_coords//2))
+            outs[row_ind:row_ind + length, self.last_ind:] += np.tile(np.tile(self.grid[i], (self.na, 1)) * int(self.stride[i]), (1, self.num_coords//2))
+            row_ind += length


Move these lines into postprocess

fengyuentau · 2022-09-03T04:04:19Z

models/multitask_centernet/multitask_centernet.py

+        cv2.putText(frame, label, (left, top - 10), cv2.FONT_HERSHEY_SIMPLEX, 0.5, (0, 255, 0), thickness=1)
+        return frame
+
+    def detect(self, srcimg):


Rename this function to infer and keep it simple like the following:

def infer(self, image): input_blob = self.preprocess(image) self.model.setInput(input_blob) output_blob = self.model.forward(self.model.getUnconnectedOutLayersNames()) results = self.postprocess(output_blob) return results

Here is another example for reference.

fengyuentau

Please make demo work as soon as possible. I got the file missing error:

Traceback (most recent call last):
  File "/Some/Path/opencv_zoo/models/multitask_centernet/demo.py", line 20, in <module>
    mcn = MCN(args.modelpath)
  File "/Some/Path/opencv_zoo/models/multitask_centernet/multitask_centernet.py", line 13, in __init__
    with open('crowd_class.names', 'rt') as f:
FileNotFoundError: [Errno 2] No such file or directory: 'crowd_class.names'

fengyuentau · 2022-09-07T13:04:08Z

models/multitask_centernet/multitask_centernet.py

+        person_indices = cv2.dnn.NMSBoxes(person_boxes, person_confidences, config['person_conf_thres'],
+                                          config['person_iou_thres']).flatten()
+        kp_indices = cv2.dnn.NMSBoxes(kp_boxes, kp_confidences, config['kp_conf_thres'],
+                                      config['kp_iou_thres']).flatten()


NMSBoxes returns a empty tuple if no person is detected. Calling flatten on an empty tuple triggers error.
Traceback (most recent call last):

File "/path/opencv_zoo/models/multitask_centernet/demo.py", line 14, in <module> srcimg = mcn.detect(srcimg) File "/path/opencv_zoo/models/multitask_centernet/multitask_centernet.py", line 207, in detect srcimg = self.postprocess(srcimg, outs, padsize=(newh, neww, padh, padw)) File "/path/opencv_zoo/models/multitask_centernet/multitask_centernet.py", line 112, in postprocess kp_indices = cv2.dnn.NMSBoxes(kp_boxes, kp_confidences, config['kp_conf_thres'], AttributeError: 'tuple' object has no attribute 'flatten'

Please doublecheck with images with no person and even no objects at all.

fengyuentau · 2022-09-08T14:59:54Z

You need to properly handle the case when there is no person in the image. Currently your script does not produce any boxes if there is no person in the image, such as the image below.

obtx added 3 commits July 19, 2022 23:57

Add files via upload

57c1038

Add files via upload

1e69177

Merge branch 'opencv:master' into master

065f7f3

fengyuentau changed the title ~~New branch~~ GSoC'22 Multi-tasking computer vision model: object detection, object segmentation and human pose detection Aug 2, 2022

obtx added 3 commits August 6, 2022 14:00

Add files via upload

4a1b022

Delete multitask_centernet directory

d909b65

Update and rename models/multitask_centernet/README.md to 模型/multitas…

356c3bd

…k_centernet/README.md

fengyuentau self-assigned this Aug 8, 2022

fengyuentau mentioned this pull request Aug 8, 2022

multitask-centernet #74

Closed

fengyuentau reviewed Aug 8, 2022

View reviewed changes

models/multitask_centernet/demo.py Outdated Show resolved Hide resolved

fengyuentau added the GSoC Google Summer of Code projected related label Aug 8, 2022

obtx added 7 commits August 16, 2022 10:46

Delete 模型/multitask_centernet directory

0392713

Add files via upload

9c23cf5

Add files via upload

f01fefd

Update and rename models/multitask_centernet/README.md to 模型/multitas…

0168f30

…k_centernet/README.md

Delete 模型 directory

b255e31

Delete demo.py

d0891cf

Rename demo_opencv.py to demo.py

feabcc5

fengyuentau reviewed Sep 3, 2022

View reviewed changes

obtx and others added 3 commits September 4, 2022 14:42

Update LICENSE

1cb40a7

upload fp32 model

c524cc6

add readme

2039902

fengyuentau reviewed Sep 5, 2022

View reviewed changes

obtx added 3 commits September 6, 2022 23:16

Update demo.py

36e9a59

Update multitask_centernet.py

d753ae4

Add files via upload

070d050

fengyuentau reviewed Sep 7, 2022

View reviewed changes

Update multitask_centernet.py

b4e0ad9

fengyuentau closed this Jan 10, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

GSoC'22 Multi-tasking computer vision model: object detection, object segmentation and human pose detection #76

GSoC'22 Multi-tasking computer vision model: object detection, object segmentation and human pose detection #76

obtx commented Jul 29, 2022

fengyuentau commented Aug 2, 2022

fengyuentau left a comment •

edited

Loading

fengyuentau left a comment

fengyuentau Sep 3, 2022

fengyuentau Sep 3, 2022

fengyuentau Sep 3, 2022

fengyuentau Sep 3, 2022

fengyuentau left a comment

fengyuentau Sep 7, 2022

fengyuentau commented Sep 8, 2022

GSoC'22 Multi-tasking computer vision model: object detection, object segmentation and human pose detection #76

GSoC'22 Multi-tasking computer vision model: object detection, object segmentation and human pose detection #76

Conversation

obtx commented Jul 29, 2022

fengyuentau commented Aug 2, 2022

fengyuentau left a comment • edited Loading

Choose a reason for hiding this comment

fengyuentau left a comment

Choose a reason for hiding this comment

fengyuentau Sep 3, 2022

Choose a reason for hiding this comment

fengyuentau Sep 3, 2022

Choose a reason for hiding this comment

fengyuentau Sep 3, 2022

Choose a reason for hiding this comment

fengyuentau Sep 3, 2022

Choose a reason for hiding this comment

fengyuentau left a comment

Choose a reason for hiding this comment

fengyuentau Sep 7, 2022

Choose a reason for hiding this comment

fengyuentau commented Sep 8, 2022

fengyuentau left a comment •

edited

Loading