Improve and polish pycaffe #816

shelhamer · 2014-07-29T03:33:46Z

fix input preprocessing configuration in picking up on Small improvements to pycaffe #733
~~add console output with human-readable labels and a grayscale flag to classify.py courtesy of @petewarden's Add MNIST support to classify.py script #735~~ to be continued...
fix Mean and scaling, C++ Caffe vs Python wrapper #525 by scaling data to [0, 255] according to Net.raw_scale, doing mean subtraction, and then input scaling
make mean argument an array instead of a file path for flexibility
update examples' outputs given correct preprocessing, always include caffe pythonpath, and reword classification + time on a gtx 770

shelhamer · 2014-07-29T03:55:51Z

@longjon please take a look. In 3cac223 I added the preprocessing option dicts as members on the C++ side–let me know what you think.

shelhamer · 2014-07-30T07:17:56Z

Let's not merge until sorting out #525 and we should probably address #613 too. @longjon if you have a chance to help, just push to my fork to update the PR.

shelhamer · 2014-08-01T01:21:36Z

@longjon a97a41b settles #525:

pycaffe represents images as single in [0, 255]
input feature scaling is done after mean subtraction
image resizing behaves (with special cases for performance)
- images (RGB or intensity) are resized by normalizing / denormalizing to the whims of skimage.transform.resize
- general K channel images are resized by scipy.ndimage.zoom

shelhamer · 2014-08-01T23:22:26Z

I want to break the interface by changing Net.set_mean to take an array instead of a filename because that was a limiting mistake. @longjon thoughts?

longjon · 2014-08-01T23:53:46Z

Yes please. That interface in particular seems okay to break because it should go away when we finally have unified input preprocessing.

(I'll have a closer look at the rest of this soon.)

shelhamer · 2014-08-04T19:48:27Z

@longjon please review and merge. Further input preprocessing optimization can be a follow-up. I'd like to have the preprocessing fix in and release the fix to master soon since it's not the nicest bug.

shelhamer · 2014-08-04T19:48:48Z

Note the build is fine -- Travis just times out downloading CUDA.

longjon · 2014-08-05T04:47:09Z

python/caffe/io.py

+    else:
+        # ndimage interpolates anything but more slowly.
+        scale = tuple(np.array(new_dims)
+                      / np.array(im.shape[:2], dtype=np.float32))


Not sure what dtype=np.float32 is doing here... won't it just be promoted to float64 anyway? E.g.,

$ ipython --no-banner In [1]: import numpy as np In [2]: (np.array(5) / np.array(4, dtype=np.float32)).dtype Out[2]: dtype('float64')

Right. This doesn't belong and I'll drop it.

On Monday, August 4, 2014, longjon notifications@github.com wrote:

In python/caffe/io.py:

@@ -40,7 +43,18 @@ def resize_image(im, new_dims, interp_order=1):
Give
im: resized ndarray with shape (new_dims[0], new_dims[1], K)
"""

return skimage.transform.resize(im, new_dims, order=interp_order)

if im.shape[-1] == 1 or im.shape[-1] == 3:

# skimage is fast but only understands {1,3} channel images in [0, 1].

im_min, im_max = im.min(), im.max()

im_std = (im - im_min) / (im_max - im_min)

resized_std = resize(im_std, new_dims, order=interp_order)

resized_im = resized_std \* (im_max - im_min) + im_min

else:

# ndimage interpolates anything but more slowly.

scale = tuple(np.array(new_dims)

/ np.array(im.shape[:2], dtype=np.float32))

Not sure what dtype=np.float32 is doing here... won't it just be promoted
to float64 anyway? E.g.,

$ ipython --no-banner
In [1]: import numpy as np
In [2]: (np.array(5) / np.array(4, dtype=np.float32)).dtypeOut[2]: dtype('float64')

—
Reply to this email directly or view it on GitHub
https://github.com/BVLC/caffe/pull/816/files#r15795016.

longjon · 2014-08-05T06:43:33Z

Looks good except as noted. There were a couple kinda awkward things I noticed that existed before this PR:

some might consider set_* functions unpythonic, since Python has properties and direct access is always overridable
the mean wrangling in Detector.configure_crop duplicates Net.deprocess

In addition, I'm not totally sure why we have mnist_words.txt, since it's basically the identity. I suppose it does serve as a simple example, so that's fine.

These things said, I'm happy to merge any PR that is a strict improvement, and I'd rather merge something incremental sooner than make everything just right eventually.

define `Net.{mean, input_scale, channel_swap}` on the boost::python side so that the members always exist. drop ugly initialization logic.

With the right input processing, the actual image classification output is sensible. - filter visualization example's top prediction is "tabby cat" - net surgery fully-convolutional output map is better Fix incorrect class names too.

shelhamer · 2014-08-06T00:24:09Z

Rebased to address comments and hold off on integrating @petewarden's changes -- they will be included in a follow-up once this fix is in.

@longjon please take a last look and merge.

longjon · 2014-08-06T05:57:53Z

python/caffe/pycaffe.py

    - reorder channels (for instance color to BGR)
-    - subtract mean
    - transpose dimensions to K x H x W


Should raw scale be noted here?

- load an image as [0,1] single / np.float32 according to Python convention - fix input scaling during preprocessing: - scale input for preprocessing by `raw_scale` e.g. to map an image to [0, 255] for the CaffeNet and AlexNet ImageNet models - scale feature space by `input_scale` after mean subtraction - switch examples to raw scale for ImageNet models - fix BVLC#525 - preserve type after resizing. - resize 1, 3, or K channel images with special casing between skimage.transform (1 and 3) and scipy.ndimage (K) for speed

…-examples Improve and polish pycaffe

…ttrs-examples Improve and polish pycaffe

shelhamer mentioned this pull request Jul 29, 2014

Add MNIST support to classify.py script #735

Closed

shelhamer assigned longjon Jul 29, 2014

This was referenced Jul 30, 2014

Mean and scaling, C++ Caffe vs Python wrapper #525

Closed

Viewing class probabilities assigned to images associated with test protobuf file, using either python wrappers or test_net.cpp #391

Closed

shelhamer mentioned this pull request Jul 30, 2014

How to test MNIST with example images? #729

Closed

shelhamer added interface labels Jul 30, 2014

shelhamer mentioned this pull request Aug 2, 2014

CAFFE python feature extraction speed 7s vs 300ms. #613

Closed

shelhamer removed the work in progress label Aug 4, 2014

longjon reviewed Aug 5, 2014
View reviewed changes

shelhamer added 4 commits August 5, 2014 15:55

define caffe.Net input preprocessing members by boost::python

99f27fc

define `Net.{mean, input_scale, channel_swap}` on the boost::python side so that the members always exist. drop ugly initialization logic.

[example] add caffe to pythonpath in all examples

6c87b92

[example] fix example outputs

943b98e

With the right input processing, the actual image classification output is sensible. - filter visualization example's top prediction is "tabby cat" - net surgery fully-convolutional output map is better Fix incorrect class names too.

[example] include prediction in classification, time on GTX 770

f1eb982

longjon reviewed Aug 6, 2014
View reviewed changes

shelhamer added 4 commits August 5, 2014 23:17

take array in pycaffe Net.set_mean() instead of file path

e1b3413

fix pycaffe context cropping with or without mean

4f77269

drop np.asarray() in favor of declaration (~1.75x speedup)

0db9478

shelhamer added a commit that referenced this pull request Aug 6, 2014

Merge pull request #816 from shelhamer/pycaffe-labels-grayscale-attrs…

52d7a48

…-examples Improve and polish pycaffe

shelhamer merged commit 52d7a48 into BVLC:dev Aug 6, 2014

shelhamer deleted the pycaffe-labels-grayscale-attrs-examples branch August 6, 2014 06:24

This was referenced Aug 6, 2014

pycaffe preprocessing incorrect for certain options in master (fixed in dev) #867

Closed

Next: 0.9999 #880

Merged

Python Web Demo Example Broken in Latest Release #899

Closed

mitmul pushed a commit to mitmul/caffe that referenced this pull request Sep 30, 2014

Merge pull request BVLC#816 from shelhamer/pycaffe-labels-grayscale-a…

52a9453

…ttrs-examples Improve and polish pycaffe

RazvanRanca pushed a commit to RazvanRanca/caffe that referenced this pull request Nov 4, 2014

Merge pull request BVLC#816 from shelhamer/pycaffe-labels-grayscale-a…

bd9dcbe

…ttrs-examples Improve and polish pycaffe

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Improve and polish pycaffe #816

Improve and polish pycaffe #816

shelhamer commented Jul 29, 2014

shelhamer commented Jul 29, 2014

shelhamer commented Jul 30, 2014

shelhamer commented Aug 1, 2014

shelhamer commented Aug 1, 2014

longjon commented Aug 1, 2014

shelhamer commented Aug 4, 2014

shelhamer commented Aug 4, 2014

longjon Aug 5, 2014

shelhamer Aug 5, 2014

longjon commented Aug 5, 2014

shelhamer commented Aug 6, 2014

longjon Aug 6, 2014

Improve and polish pycaffe #816

Improve and polish pycaffe #816

Conversation

shelhamer commented Jul 29, 2014

shelhamer commented Jul 29, 2014

shelhamer commented Jul 30, 2014

shelhamer commented Aug 1, 2014

shelhamer commented Aug 1, 2014

longjon commented Aug 1, 2014

shelhamer commented Aug 4, 2014

shelhamer commented Aug 4, 2014

longjon Aug 5, 2014

Choose a reason for hiding this comment

shelhamer Aug 5, 2014

Choose a reason for hiding this comment

longjon commented Aug 5, 2014

shelhamer commented Aug 6, 2014

longjon Aug 6, 2014

Choose a reason for hiding this comment