-
Notifications
You must be signed in to change notification settings - Fork 18.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
how to make a prediction in C++ #499
Comments
You could use a Image-Data layer to specify which images need to be On Friday, June 13, 2014, onauparc notifications@github.com wrote:
Sergio |
Hi, Datum datum; Now I have another doubt. How can I get the predicted class? Here https://github.com/BVLC/caffe/blob/a34816263c1e5181ac0eb857a20fdaabd07b80e9/tools/test_net.cpp I saw that I can get the accuracy in this way: |
For doing predictions with just one image, you don't need to create a blob with 10 images. Although you will need to change the prototxt to reflect that. What result contains depends on the last layer in the prototxt. |
Ok, perfect. I'll see. Thank you for your answers. |
Hi, I found how to make the predicion in C++, like the python image_net_classification example. Here https://gist.github.com/onauparc/dd80907401b26b602885 you can find my example, and it works! (It is based on this code https://github.com/BVLC/caffe/blob/master/tools/compute_image_mean.cpp.) The predicted class of the cat image is class 274, so it seems like the python example. The problem is that in my example I did a lot of copy and rounds: I put data into datum, than I put it in in a blobproto, than in a proto, finally I can do the Forward! Since I have a minibatch input made by some opencv Mat, is there any example easier or more direct? |
@onauparc I think you are making things a bit complicated. Also you forgot to subtract the mean_image. I would recommend you use a image_data_layer (which take care of reading a list of images, cropping, resizing, subtracting the mean, ...) at top of your prototxt, and the argmax_layer at the bottom of your prototxt. That way you will only need to specify a file containing the set of images you want to do prediction for, and you would only to do:
|
@onauparc If you have an image list on filesystem the best solution is to plug an image_data layer on top like in this example https://github.com/BVLC/caffe/blob/master/examples/feature_extraction/imagenet_val.prototxt |
Or probably could be also used memory data layer? |
@bhack There is not simple way to transform a opencv::Mat into a blob, since they store the data differently internally. Also usually there are several pre-processing steps to do, like mean subtraction, cropping, mirror, ... |
@bhack what I meant is that one still need to do the pre-processing steps like Image_Data layers does. That means it is not simply copying pixels or using directly in-memory #196. Once data_source and data_prepocessing are separated then will be easier to add more data sources, including a vector of opencv Mat. Although #251 was never finished or integrated, one could easily extend #196 or Image_data layer to do this. https://github.com/BVLC/caffe/blob/dev/src/caffe/layers/image_data_layer.cpp But if you want to do PR for this it will be useful. If you tell me a bit more what do you have in mind I may be able to help more. |
@sguada Yes in layers we have pre-processing steps and data mixed. #407 is very interesting but also this one is still not integrated like #251. Probably taking something from #251 could be a little bit easy as inspiration to unwrap opencv Mat structure in the meantime that a more general solution like #407 will be finalized. |
@sguada The general idea is that users could have a standard computer vision pipeline with opencv (taking Mat from video or single image), than find (with opencv or using opencv inside caffe) some candidate regions inside the frame with an X algorithm (resulting in a vector of cv::Mat or a vector of Rect). Then do some pre-processing on this regions (still in opencv or in the preprocessing step inside Caffe) and then pass this region as fast as (minimizing copy) to the forward pass. So something from #251 actually could be useful. #407 seems a more general arch change that probably impact many things including the future of #251. |
Hi @sguada , This is the prototxt I used to make the net: name: "CaffeNet"
layers {
name: "data"
type: IMAGE_DATA
top: "data"
top: "label"
image_data_param {
source: "my_path/examples/_temp/file_list.txt"
mean_file: "my_path/data/ilsvrc12/imagenet_mean.binaryproto"
batch_size: 4
crop_size: 227
mirror: false
new_height: 256
new_width: 256
}
}
layers {
name: "conv1"
type: CONVOLUTION
bottom: "data"
top: "conv1"
blobs_lr: 1
blobs_lr: 2
weight_decay: 1
weight_decay: 0
convolution_param {
num_output: 96
kernel_size: 11
stride: 4
}
}
layers {
name: "relu1"
type: RELU
bottom: "conv1"
top: "conv1"
}
layers {
name: "pool1"
type: POOLING
bottom: "conv1"
top: "pool1"
pooling_param {
pool: MAX
kernel_size: 3
stride: 2
}
}
layers {
name: "norm1"
type: LRN
bottom: "pool1"
top: "norm1"
lrn_param {
local_size: 5
alpha: 0.0001
beta: 0.75
}
}
layers {
name: "conv2"
type: CONVOLUTION
bottom: "norm1"
top: "conv2"
blobs_lr: 1
blobs_lr: 2
weight_decay: 1
weight_decay: 0
convolution_param {
num_output: 256
pad: 2
kernel_size: 5
group: 2
}
}
layers {
name: "relu2"
type: RELU
bottom: "conv2"
top: "conv2"
}
layers {
name: "pool2"
type: POOLING
bottom: "conv2"
top: "pool2"
pooling_param {
pool: MAX
kernel_size: 3
stride: 2
}
}
layers {
name: "norm2"
type: LRN
bottom: "pool2"
top: "norm2"
lrn_param {
local_size: 5
alpha: 0.0001
beta: 0.75
}
}
layers {
name: "conv3"
type: CONVOLUTION
bottom: "norm2"
top: "conv3"
blobs_lr: 1
blobs_lr: 2
weight_decay: 1
weight_decay: 0
convolution_param {
num_output: 384
pad: 1
kernel_size: 3
}
}
layers {
name: "relu3"
type: RELU
bottom: "conv3"
top: "conv3"
}
layers {
name: "conv4"
type: CONVOLUTION
bottom: "conv3"
top: "conv4"
blobs_lr: 1
blobs_lr: 2
weight_decay: 1
weight_decay: 0
convolution_param {
num_output: 384
pad: 1
kernel_size: 3
group: 2
}
}
layers {
name: "relu4"
type: RELU
bottom: "conv4"
top: "conv4"
}
layers {
name: "conv5"
type: CONVOLUTION
bottom: "conv4"
top: "conv5"
blobs_lr: 1
blobs_lr: 2
weight_decay: 1
weight_decay: 0
convolution_param {
num_output: 256
pad: 1
kernel_size: 3
group: 2
}
}
layers {
name: "relu5"
type: RELU
bottom: "conv5"
top: "conv5"
}
layers {
name: "pool5"
type: POOLING
bottom: "conv5"
top: "pool5"
pooling_param {
pool: MAX
kernel_size: 3
stride: 2
}
}
layers {
name: "fc6"
type: INNER_PRODUCT
bottom: "pool5"
top: "fc6"
blobs_lr: 1
blobs_lr: 2
weight_decay: 1
weight_decay: 0
inner_product_param {
num_output: 4096
}
}
layers {
name: "relu6"
type: RELU
bottom: "fc6"
top: "fc6"
}
layers {
name: "drop6"
type: DROPOUT
bottom: "fc6"
top: "fc6"
dropout_param {
dropout_ratio: 0.5
}
}
layers {
name: "fc7"
type: INNER_PRODUCT
bottom: "fc6"
top: "fc7"
blobs_lr: 1
blobs_lr: 2
weight_decay: 1
weight_decay: 0
inner_product_param {
num_output: 4096
}
}
layers {
name: "relu7"
type: RELU
bottom: "fc7"
top: "fc7"
}
layers {
name: "drop7"
type: DROPOUT
bottom: "fc7"
top: "fc7"
dropout_param {
dropout_ratio: 0.5
}
}
layers {
name: "fc8"
type: INNER_PRODUCT
bottom: "fc7"
top: "fc8"
blobs_lr: 1
blobs_lr: 2
weight_decay: 1
weight_decay: 0
inner_product_param {
num_output: 1000
}
}
layers {
name: "prob"
type: SOFTMAX
bottom: "fc8"
top: "prob"
}
layers {
name: "output"
type: ARGMAX
bottom: "prob"
top: "output"
} And the file file_list.txt contains this string: I noticed that argmax returns 0. Actually it returns the number near the image path. It seems it is a label and argmax returns this label. If I change the label with 1, argmax returns 1! I don't know if I'm making some mistakes, but this is not the behavior that I expect. I would like to have a network, train it, (now I'm doing these steps with pretrained examples), than pass to the net new images and make a prediction with this net. Where am I wrong? |
Since your prototxt have two free So you should look at the second blob in result. That way you can also Sergio 2014-06-20 3:24 GMT-07:00 onauparc notifications@github.com:
|
Fatastic!!! It works :) Thank you so much!!! |
Hi, @sguada , I find the image_data_param have not define a 'channel' varible that specify the input image is color or gray. How can I specify whether the input image is grayscale or color? |
@onauparc, I tried the ImageDataLayer as you asked in #251 using the code snippet of @sguada and your prototxt. It turns out that passing in a vector of cv::Mat can get exactly the same prediction results as loading the images from disk. The predicted class label of the cat.jpg is 281 which corresponds to "n02120505 grey fox, gray fox, Urocyon cinereoargenteus" in the
|
Ok, thank you @kloudki, it prefectly works! :) |
You're right in that it's not reliable to create a net without the "source" field of the ImageDataLayer. The ConvolutionLayer or more generally any layer in the network depends on the size of the bottom blob that is set up by the layer below.
In this case, the images listed in the source file_list.txt actually initialize the blob but are not used in the Forward method of the ImageDataLayer. They are overridden by the images added by AddImagesAndLabels. If it works for your purpose right now, then just use it. No need to wait for a more meticulously designed solution that separates the data sources and pre-processing steps. |
@onauparc : Could you share how you retrieve the list of probabilities/classes when you feed a list of in memory images (more than 1)? I really appreciate it! Currently, I'm feeding one by one, and calling net.forward for each. However, I figure this might not very optimal performance-wise. |
@anguyen8 : To make a prediction of more images I use a prototoxt like that I posted above, the first layer (IMAGE_DATA) allows to take a list of images by a file (file_list.txt). Than I used the code posted above by sguada to make the forward on all the images (ForwardPrefilled) and get the results. Pay attention to the top free blobs that you have in your prototxt, for example in my prototxt I have two free blobs so the result of ForwardPrefilled will contain two blobs. I hope this helps. |
@onauparc : Thanks so much, sir! Now I know that ForwardPrefilled is the one to call! |
Right: Caffe always processes a To fix the variable results you need to set the phase to TEST. By default See #192 for context. p.s. Please ask further questions on the caffe-users mailing list. As of the latest release we prefer to keep issues reserved for Caffe development. Thanks! |
I set phase to TEST and now the classification is deterministic indeed. |
@kloudki: I couldn't seem to find the method AddImagesAndLabels in the image_data_layer.cpp. Could you please let me know where it is implemented? I'm currently on the dev branch. Thanks. |
@hhuytho : the method AddImagesAndLabels is available in this branch #251 |
@anguyen8: thanks. I just got it from the fork! |
In #251, it seems like that it's not necessary to include "source" in the prototxt anymore. However, when I removed it, I got a segmentation fault as a result of not being able to inititalize the ImageDataLayer due to no bottom blobs. So do we still need a list of images in the "source" field? Thanks. |
@hhuytho : yes you still need the "source" field. Please see the comment above #499 (comment) |
Thanks @anguyen8. I have made some changes to this implementation to handle the case where there is no source. |
I follow the code offered by @kloudkl :
But the problem is image_data_layer does't have the member function AddImagesAndLabels(), was it deleted by the latest version? how to solve this problem? |
@jasonustc You need to try to use this #1068 |
Yes you need to try to use with this PR. The old one was closed without merge. |
@bhack got it, I will add these changes to my code manually, thanks a lot! |
@bhack on your opencv_io branch, I tried compiling and got
I tried using MemoryDataLayer and it compiled, but got this at runtime "pure virtual method called" |
Try to use MemoryData layer as input to prevent fake image list.
|
How to test the trained MNIST data sets .Can any one suggest any tutorials for training the data sets. Regards, |
What do you mean training on test data? On Monday, October 26, 2015, anukuppan notifications@github.com wrote:
Full name: Tran Lam An Homepage: http://antran89.github.io/ |
How to test caffe classifier trained by MNIST data sets? |
Hello, how do I push-back the needed blobs before calling forward? Thanks! |
How to implement caffe into my C++ application an example?How to run classfication.cpp ? |
@arieal You can refer to the example documents http://caffe.berkeleyvision.org/gathered/examples/cpp_classification.html. |
@antran89 Thanks very much! what I meant is how to link caffe as library and call functions in my own project not modify the example already configured and build successfully in caffe. |
@arieal I think it is not difficult, link again libcaffe.so and include some .h files should be enough. |
Hi,
I'm using caffe and I'm trying to make a prediction in C++ with a pretrained network like this python example http://nbviewer.ipython.org/github/BVLC/caffe/blob/master/examples/imagenet_classification.ipynb. But I don't find a C++ method for prediction and I don't get how to pass an input image and make the prediction. I've seen test_net.cpp but it does not pass any image in input. How can I pass an image or more images in input to the network? How can I make a prediction?
The text was updated successfully, but these errors were encountered: