-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Feature] Support Webcam Demo for Spatio-temporal Action Detection Models #795
Conversation
Codecov Report
@@ Coverage Diff @@
## master #795 +/- ##
==========================================
+ Coverage 84.89% 85.22% +0.33%
==========================================
Files 131 131
Lines 9394 9415 +21
Branches 1605 1612 +7
==========================================
+ Hits 7975 8024 +49
+ Misses 1012 985 -27
+ Partials 407 406 -1
Flags with carried forward coverage won't be shown. Click here to find out more.
Continue to review full report at Codecov.
|
For the |
@kennymckormick This may cause index of range error mmaction2/demo/demo_spatiotemporal_det.py Lines 345 to 364 in 905f07a
|
@kennymckormick @congee524 This pr is ready for review |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Besides, I have run the demo with a video as the input and output to a file. First, it runs very slow: it takes tens of minutes for a one-minute video. Besides, it doesn't write the output to a video file.
It takes 1:37 to run this demo with a 1:16 long video on V100. And I have also tested webcam input(real-time) demo on 1080ti with
It's a bug when creating |
@kennymckormick Could reproduce your bug. reading frames too fast leads to unexpected performance degradation. add |
Hi, Irving. I have used the latest code, and it seems that the problem still occurs, can you run this command (which I used) on your machine to check how long does it take? That might be helpful to figure out the problem. python demo/webcam_demo_spatiotemporal_det.py --input-video demo/ava_1min.mp4 --config configs/detection/ava/slowonly_omnisource_pretrained_r101_8x8x1_20e_ava_rgb.py --checkpoint https://download.openmmlab.com/mmaction/detection/ava/slowonly_omnisource_pretrained_r101_8x8x1_20e_ava_rgb/slowonly_omnisource_pretrained_r101_8x8x1_20e_ava_rgb_20201217-16378594.pth --det-config demo/faster_rcnn_r50_fpn_2x_coco.py --det-checkpoint http://download.openmmlab.com/mmdetection/v2.0/faster_rcnn/faster_rcnn_r50_fpn_2x_coco/faster_rcnn_r50_fpn_2x_coco_bbox_mAP-0.384_20200504_210434-a5d8aa15.pth --det-score-thr 0.9 --action-score-thr 0.5 --label-map demo/label_map_ava.txt --predict-stepsize 40 --output-fps 20 --out-filename tmp.mp4; Besides, here is output.profile, I don't know how to analyze it ... |
Actually, I have already used your configs yesterday... Can you share some logs and the 1min video please.. Will have a try tonight.
|
Oops, I'm sorry, I just found that ava_1min.mp4 is not in the codebase, here is the testing video. ava_1min.mp4 |
..Couldn't reproduce your bug... i7-8700K + 1080ti + 32G... |
kindly ping @kennymckormick Please share part of the terminal logs. |
Maybe it's a problem with my server. I think it's OK to go on since there seems no problem on your side. Besides, I will try to test it on another platform. |
Successfully run this demo on AGX. |
Thanks for the demo code! Demos are always great things to have. Two comments
edit: |
There is NO LICENCE in AlphAction, I'll create a issue to ask for permission.
I do have think about how to refactor this demo. Most of these codes(mmdet wrapper, stdet wrapper, multi-thread codes) shouldn't not move to mmaction directory. Maybe the visualization tools could move to main repo.
One reason i like this alphation visualization tool is that the font is much better than supported opencv fonts. How about
|
You can add an info to tell users that we support customizing font, and how to do it. There's no need to give a specific font since everyone has her/his favorite font. |
ok, I'll ask for permission first. |
Free for non-commercial use only. Will remove related codes later. |
I guess we can use them in our project. After all, OpenMMLAB is also a non-commercial project, right? @innerlee |
@kennymckormick Not really. Licence issues are more complicated. Our project is open-source of course. |
Sorry I misunderstood it. |
when run webcam_demo_spatiotemporal_det on cloud gpu into debug |
Description
This implementation is based on SlowFast Spatio-temporal Action Detection Webcam Demo.
TODO
[ ] yolov4 human detector: it seems human detector is not the bottleneck for this demo.predict_stepsize
must in range[clip_len * frame_interval // 2, clip_len * frame_interval]
. Find a way to supportpredict_stepsize
in range[0, clip_len * frame_interval]
Known issue
model -> test_cfg -> rcnn -> action_thr
should be .0 instead of current default value 0.002. This may cause different bboxes number for different actions.mmaction2/demo/demo_spatiotemporal_det.py
Lines 345 to 364 in 905f07a
j
ofresult[i][j, 4]
may be out of range. Thefor j in range(proposal.shape[0])
loop are assuming that all of the result[i] has the same shape, akathe same bbox number for different actions
.Usage
--output-fps
according to printed logDEBUG:__main__:Read Thread: {duration} ms, {fps} fps
.--predict-stepsize
so that the durations for read and inference, which are both printed by logger, are almost the same.