Move bounding box drawing from the server to the client. #869
Conversation
| @@ -11,6 +11,8 @@ | |||
|
|
|||
| CONFIG_TEMPLATE = "config_template.html" | |||
| HEADER_TEMPLATE = "header_template.html" | |||
| NG_HEADER_TEMPLATE = "ng_header_template.html" | |||
| NG_FOOTER_TEMPLATE = "ng_footer_template.html" | |||
There was a problem hiding this comment.
It doesn't look like footer_template.html is used anywhere?
|
Very cool. This is working great for me! |
| @@ -125,10 +129,9 @@ def process_data( | |||
| box = ((output[0], output[1]), (output[2], output[3])) | |||
| bboxes.append(box) | |||
| self.bbox_count += 1 | |||
| digits.utils.image.add_bboxes_to_image( | |||
There was a problem hiding this comment.
If we remove it here should we just delete it from utils.image?
There was a problem hiding this comment.
I left it there in case there was a use for it in the REST API at some point. I'll just pull it.
|
I take this to mean DIGITS currently isn't equipped to do multiple class detection? Any idea on an ETA on that? That kind of puts me and my whole team in a bit of a bind. |
| @@ -49,6 +49,32 @@ def get_header_template(self): | |||
| """ | |||
| return None, None | |||
|
|
|||
| def get_ng_templates(self): | |||
There was a problem hiding this comment.
do you think we could stick to the existing get_header_template() (possibly we can add a get_footer_template() too)? It feels awkward to name APIs after the Javascript package we are using. If you move to C3 or any other Javascript you don't want to rename the APIs right?
There was a problem hiding this comment.
I agree, @gheinrich. I should at least rename it to something library agnostic, like app_begin_html and app_end_html or something to that effect. I didn't insert the app header html into the existing header because the header is wrapped in a div, and the app code needs to have the interface views as child elements for scope purposes.
<div class="row">
<h3>Summary</h3>
{{header_html|safe}}
</div>
If I were to insert the app code into that header block, the interface view would be isolated from it. Thoughts?
I had added a footer, but it was not used and subsequently removed.
There was a problem hiding this comment.
Decided to rename ng_header and ng_footer to app_begin and app_end. It turns out that trying to work these into the header_html would make it too convoluted.
|
@gheinrich could you have another look before I merge? |
|
I've made some changes to support multi-class object detection visualization.
With an option popup to adjust display options. ###Todo:
Like todo:
|
|
Hi Joe, this is fantastic: |
| @@ -5,18 +5,3 @@ | |||
| {% from "helper.html" import mark_errors %} | |||
|
|
|||
| <small>Draw a bounding box around a detected object. This expected network output is a nested list of list of box coordinates</small> | |||
There was a problem hiding this comment.
if you don't mind fixing this typo: "This expected... " -> "The expected ..." (and maybe "list of list" -> "list of lists" too). Or if you want to completely rephrase it so it does sound English, you're most welcome to do so :-)
There was a problem hiding this comment.
I wrote something new. How English it sounds is arguable.
jmancewicz
force-pushed
the
js-bboxes
branch
2 times, most recently
from
11255df
to
9fc2757
Compare
|
The Travis build failed for some unrelated Torch issue. This looks great - thanks Joe! |
|
Hi, How can I use this multi-class inference visualization? I'm using digits 4 with nvidia caffe 0.15.13 |
|
@szm2015 this pull request simply adds visualization support in DIGITS. It doesn't provide information about how to actually train a working model. You can try to use the |
|
@lukeyeager, Thank you, but my question was indeed about visualization not training, I have already trained a 2-class network but have problem visualizing it, It only shows one class and I don't know whether it's because normal visualization doesn't support more than 1 class or it's that I'm not doing the training right at all |
|
Oh I see. If your second class isn't showing up, then you simply aren't generating any proposals for that class. You can monkey around with the thresholds in the clustering layer to try to generate more boxes. Try changing from to for example. Just make some edits to the |
|
When using the Bounding Boxes, I see this when I have results for two classes
You can use the Raw Data to see the data that describes the Bounding Boxes. The empty boxes are filtered out before going to the Bounding Box visualization, but are present in the Raw Data. You can see here that bbox-list-class1 (pedestrian) has one box and bbox-list-class0 (car) has two.
|
|
@lukeyeager Thank you, I will try this, |
|
@szm2015 thanks for asking! For DIGITS you could cite either of these papers. I'd suggest using the first since it seems the links for the second paper will probably expire next year.
DetectNet doesn't have a paper, unfortunately. I guess you can link to this pull request? NVIDIA/caffe#144 EDIT Actually, this blog post is probably a better link for DetectNet: |
|
@lukeyeager I am trying to train a model to detect more than two class of objects. Result obtained for 3 class object detection Help me to increase the performance of the models |
|
I have trained my network for 3 object classes & I am getting good mAP fo all 3 classes, but not getting bounding box for first 2 objects. It draws bounding box only for last class. I would like to know whether I have to make changes for visualization of bounding boxes for multi classes or DIGITS by default does the changes for visualization of bounding boxes for multi class object detection. |
|
@ShashankVBhat Recheck the last few layer of the network namely "cluster","cluster_gt","score" and "mAP" layer { Now for each object class there should be a "score" layer and "mAP" layer. "score" and "mAP" layer for class0 is given below. layer { |
|
@varunvv Thank you for the response. I have checked these settings as you mentioned but still I am not able to get bounding box. |
|
@ShashankVBhat Hi, you said that u manage to create a network on 3 classes, I tried to do so with 2 classes with the modifications of https://github.com/NVIDIA/caffe/blob/caffe-0.15/examples/kitti/detectnet_network-2classes.prototxt , but it seems that neither the recall or the precision/maP moves from 0. I kept the value of the tutorial https://github.com/NVIDIA/DIGITS/blob/master/examples/object-detection/README.md for the batch size, learning rate, etc. |
|
@jmancewicz Hello, is there any chance you could provide the caffe model for multi-class detection of car@pedestrians in DIGITS? |
Move bounding box drawing from the server to the client.
|
@varunvv i got 1 class trained for car as well as for my own data.But iam struggling hard to find a solution with multiclass detection in DIGITS.i tried various ways from Digits but network is only training for car always.Please help me to solve this. |
|
@ShashankVBhat Hey, did you train entirely for the 3 classes at once or fine tuned for other classes after training on a single class? Please share the details of training parameters that you've used. It would be of great help! |
|
@jmancewicz How are you getting such good results for both the classes (cars and pedestrians)?. Please share the training process details. Did you fine tune over one class or trained the model for both classes at once? |
|
@lukeyeager @senecaur @gheinrich @dusty-nv Getting the following raw output for multi-class inference. Could someone show me an example of the Python Binding layer. Here is my inference output... bbox-list-class1 [[0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.]] bbox-list-class0 [[0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.] [0. 0. 0. 0. 0.]] My Map is zero after 300 epochs. My detectnet Network is included below... DetectNet networkData/Input layersname: "DetectNet" Data transformation layerslayer { Label conversion layerslayer { ###################################################################### Start of convolutional network###################################################################### layer { layer { layer { layer { layer { layer { layer { layer { layer { layer { layer { layer { layer { layer { layer { layer { layer { layer { layer { layer { layer { layer { layer { layer { layer { layer { layer { layer { layer { layer { layer { layer { layer { layer { layer { layer { layer { layer { layer { layer { layer { layer { layer { layer { layer { ###################################################################### End of convolutional network###################################################################### Convert bboxeslayer { Loss layerslayer { Cluster bboxeslayer { Calculate mean average precisionlayer { layer { Any assistance is greatly appreciated! |
In preparation for the multi-class object detection display, I have moved the drawing of the bounding boxes from the server to the client. This will allow quicker interactions with the display. Users will be able to toggle displays of bounding boxes by class, adjust the image desaturation if there are color conflicts between the image a class's bounding box color, and change the line width.
So that the lines are drawn with a fixed width regardless of scale, the images are redrawn when the window is resized such that the image display is resized.
The header text "Found 7 box(es) in 3 image(s)" has been changed to drop the "in
nimages" if the number of images is one and to pluralize "box" as needed.The line width and desaturation settings are saved per browser.