Cropping the mean file

[lukeyeager]

Since the huge update, I'm having to fix lots of API changes with my python interfaces to caffe. How do I wade through the commit stream to find out what all needs to change?

This is my most recent problem. I try this:

classifier = caffe.Classifier(deploy_file, snapshot_file,
             image_dims      = (height, width),
             mean            = image_mean,
             raw_scale       = 255,
             channel_swap    = channel_swap,
             )
classifier.predict([image], oversample=False)

And I get this error:

ValueError: operands could not be broadcast together with shapes (3,227,227) (3,256,256) (3,227,227)

Apparently, the mean_file has to be cropped before passing it to the Classifier constructor now. How was I supposed to know that? I don't see it in any of these commits. And how am I supposed to do it? Should I be cropping it myself manually, or should I try to use Classifier.transformer.preprocess()?

[shelhamer]

It's true a lot has changed in the last release, which is why we're shooting for more gradual releases from here on. While the release notes are still coming in at #1849 you can look there for a list of changes with interface differences bolded.

With regards to the new Transformer in particular you can look at the classification example history instead of classify.py itself. In particular 26c16bf notes

This is a quick translation of the examples to the caffe.io.Transformer
interface. The results are not strictly identical to the earlier
implementation! The models now use a mean pixel instead of a mean image
for simplicity. The output classifications and detections are preserved
but scores may differ.

Note: the examples will be rewritten to make use of caffe.Net alone
since it is the true interface, but not yet.

In general this step tried to be less magic while keeping configurability. The coming switch to caffe.Net examples should clarify the standard practices with the interface, whereas it seems caffe.{Classifier,Detector} have confused many.

Re: what to do now about the mean, you could crop it yourself or I suggest taking the mean pixel as in the example. The mean pixel approach simplifies working with images of different size / aspect / resolution and has been shown to work for classification, detection, and segmentation. This is how the BVLC GoogLeNet was learned: https://github.com/BVLC/caffe/blob/master/models/bvlc_googlenet/train_val.prototxt#L13-L15.

p.s. Historically the caffe.Classifier interface was actually resizing the mean behind the scenes, which was fine for classification but seems like an odd choice in retrospect.

[lukeyeager]

Thanks for the quick response, Evan. I had read through that list of changes before and didn't see anything about the mean pixel instead of mean image change. I'll have to look through that more carefully.

[wang4249]

I have the same problem with you, can you tell me have to fix it? Thanks in advance！

[lukeyeager]

As Evan said, you can look at the classification example

Use this (note the mean(1).mean(1) at the end):

mean=np.load(caffe_root + 'python/caffe/imagenet/ilsvrc_2012_mean.npy').mean(1).mean(1)

[wang4249]

sorry, I am a beginner with Caffe and don't know how to write my python script yet, I met this problem when run the classify.py file with the 'bvlc_reference_caffenet.caffemodel'. So I need to change the variable in the classify.py file to 'mean=np.load(caffe_root + 'python/caffe/imagenet/ilsvrc_2012_mean.npy').mean(1).mean(1)'?

[wang4249]

Thanks a lot, Have fixed it! :)

[Coderx7]

Today I faced this issue(Here is the SO Question) ! the irony is I did as Evan and @lukeyeager instructed! yet I keep getting the infamous error:
ValueError: operands could not be broadcast together with shapes (3,224,224) (3,)
meaning, doing :

mu = mean_file.mean(1).mean(1)
.. = (img[:,0:224,0:224] - mu)

wouldn't work either and made me crop the mean file to get rid of that error.
What is the correct way of doing this ?
Please note that using transformer preprocessing and setting the mean is not a feasible solution since its extremely slow.(for more information about why transformer.preprocessing) is not used, please check this link : https://stackoverflow.com/questions/45294732/caffes-transformer-preprocessing-takes-too-long-to-complete )

Update:
Found the cause! and explained it in the stackoverflow question linked above!