Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Already on GitHub? Sign in to your account

Dense DAISY feature description #384

Merged
merged 21 commits into from Dec 30, 2012

Conversation

Projects
None yet
4 participants

Hi there,

I propose to include the DAISY feature descriptor [1], which I recently reimplemented in Python following the paper and the original C++ code 2.

Thanks for a great library!
Anders

[1]: Tola, Engin, Vincent Lepetit, and Pascal Fua. "Daisy: An efficient dense descriptor applied to wide-baseline stereo." Pattern Analysis and Machine Intelligence, IEEE Transactions on 32.5 (2010): 815-830.

Anders Boesen Lindbo Larsen added some commits Dec 8, 2012

Anders Boesen Lindbo Larsen Add dense DAISY feature description. 2eb03c0
Anders Boesen Lindbo Larsen PEP8 style corrections. 979a15c

@stefanv stefanv commented on an outdated diff Dec 9, 2012

skimage/feature/_daisy.py
+ P = ceil((M-radius*2)/step)
+ Q = ceil((N-radius*2)/step)
+ R = (rings*histograms + 1)*orientations
+
+ References
+ ----------
+ [1]: Tola et al. "Daisy: An efficient dense descriptor applied to
+ wide-baseline stereo." Pattern Analysis and Machine Intelligence,
+ IEEE Transactions on 32.5 (2010): 815-830.
+ [2]: http://cvlab.epfl.ch/alumni/tola/daisy.html
+ '''
+
+ # Validate image format.
+ if img.ndim > 2:
+ raise ValueError('Only grey-level images are supported.')
+ if img.dtype.kind == 'u':
@stefanv

stefanv Dec 9, 2012

Owner

skimage.img_as_float

Owner

stefanv commented Dec 9, 2012

Hi Anders, thanks for your contribution! This is exciting--our first real feature detector PR. Would you please add a demo script (such as those in the gallery), so that we could visualize the DAISY descriptors? If they can easily be used for matching, the standard "lines between matching features" demo would be great. Thanks again!

@stefanv stefanv commented on an outdated diff Dec 9, 2012

skimage/feature/_daisy.py
@@ -0,0 +1,165 @@
+import numpy as np
+from scipy import sqrt, pi, arctan2, cos, sin, exp
+from scipy.ndimage import gaussian_filter
+from scipy.special import iv
+
+
+def daisy(img, step=4, radius=15, rings=3, histograms=8, orientations=8,
+ normalization='l1', sigmas=None, ring_radii=None):
+ '''Extract DAISY feature descriptors densely for the given image.
+
+ DAISY is a feature descriptor similar to SIFT formulated in a way that
+ allows for fast dense extraction. Typically, this is practical for
+ bag-of-features image representations.
+
+ The implementation follows Tola et al. [1] but deviate on the following

@stefanv stefanv commented on an outdated diff Dec 9, 2012

skimage/feature/_daisy.py
+
+
+def daisy(img, step=4, radius=15, rings=3, histograms=8, orientations=8,
+ normalization='l1', sigmas=None, ring_radii=None):
+ '''Extract DAISY feature descriptors densely for the given image.
+
+ DAISY is a feature descriptor similar to SIFT formulated in a way that
+ allows for fast dense extraction. Typically, this is practical for
+ bag-of-features image representations.
+
+ The implementation follows Tola et al. [1] but deviate on the following
+ points:
+ * Histogram bin contribution are smoothed with a circular Gaussian
+ window over the tonal range (the angular range).
+ * The sigma values of the spatial Gaussian smoothing in this code do
+ not match the sigma values in the original code by Tola et al. [2].

@stefanv stefanv commented on an outdated diff Dec 9, 2012

skimage/feature/_daisy.py
+ normalization='l1', sigmas=None, ring_radii=None):
+ '''Extract DAISY feature descriptors densely for the given image.
+
+ DAISY is a feature descriptor similar to SIFT formulated in a way that
+ allows for fast dense extraction. Typically, this is practical for
+ bag-of-features image representations.
+
+ The implementation follows Tola et al. [1] but deviate on the following
+ points:
+ * Histogram bin contribution are smoothed with a circular Gaussian
+ window over the tonal range (the angular range).
+ * The sigma values of the spatial Gaussian smoothing in this code do
+ not match the sigma values in the original code by Tola et al. [2].
+ In their code, spatial smoothing is applied to both the input image
+ and the center histogram. However, this smoothing is not documented
+ in [1] and, therefore, it is omitted.

@stefanv stefanv commented on an outdated diff Dec 9, 2012

skimage/feature/_daisy.py
+ orientations : int, optional
+ Number of orientations (bins) per histogram.
+ normalization : {'l1', 'l2', 'daisy', 'off'}, optional
+ How to normalize the descriptors:
+ * 'l1': L1-normalization of each descriptor.
+ * 'l2': L2-normalization of each descriptor.
+ * 'daisy': L2-normalization of individual histograms.
+ * 'off': Disable normalization.
+ sigmas : 1D array of float, optional
+ Standard deviation of spatial Gaussian smoothing for the center
+ histogram and for each ring of histograms. The array of sigmas should
+ be sorted from the center and out. I.e. the first sigma value defines
+ the spatial smoothing of the center histogram and the last sigma value
+ defines the spatial smoothing of the outermost ring. Specifying sigmas
+ overrides the following parameter:
+ rings = len(sigmas)-1
@stefanv

stefanv Dec 9, 2012

Owner

Place code snippet in double quotes

Owner

ahojnnes commented Dec 9, 2012

Great contribution, thank you! Would you add an usage example script to doc/examples?

Anders Boesen Lindbo Larsen Better Sphinx documentation. e5fb8a2

Stefan: Alas, my code only performs dense feature extraction - not local feature detection and extraction. Dense feature extraction is useful for e.g. visual recognition based on bag-of-features image representations.

Johannes: I assume your example request is about feature detection/extraction/matching which indeed is very suitable for an example. Dense feature extraction, however, is not that easy to demonstrate. I can try to make a simple visual recognition example, but it will require lots of image data. Would that be ok?

Member

amueller commented Dec 9, 2012

@andersbll This is great :) Do you know of a comparison of Daisy and SIFT for recognition?
Maybe a good example would be one that is sort of similar to the hog one.
You could visualize some of the feature channels for example. Do you think that is useful?

A full-blown recognition example is maybe a bit over-the-top for an example as you'd also need to have a classifier.

@amueller: DAISY and SIFT are very similar by construction and have the same performance (according to the papers below). I prefer DAISY descriptors as they are more flexible in terms of configuration and because the formulation is more elegant than SIFT.
I agree with you that a recognition example is going to be too extensive. On the other hand, I do not think that a HOG image is useful to illustrate DAISY since the spatial layout of the dense features is typically disregarded in a bag-of-features application.
Maybe I can make an illustration similar to: http://www.vlfeat.org/demo/sift_basic_4.jpg ?

DAISY vs. SIFT:

  • Winder, S., Hua, G., & Brown, M. (2009, June). Picking the best daisy. In Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on (pp. 178-185). IEEE.
  • Dahl, A. L., Aanæs, H., & Pedersen, K. S. (2011, May). Finding the Best Feature Detector-Descriptor Combination. In 3D Imaging, Modeling, Processing, Visualization and Transmission (3DIMPVT), 2011 International Conference on (pp. 318-325). IEEE.
  • Kaneva, B., Torralba, A., & Freeman, W. T. (2011, November). Evaluation of image features using a photorealistic virtual world. In Computer Vision (ICCV), 2011 IEEE International Conference on (pp. 2282-2289). IEEE.
Member

amueller commented Dec 9, 2012

Thanks for the references. I didn't know the last one.
I agree that daisy will probably work good, it is just that I don't know of any literature using it for recognition.
As far as I can tell, the references you gave all do keypoint matching, not bag of words.

Sorry, I didn't catch from your first post that you asked for descritor evaluations in recognition applications. Unfortunately, I haven't seen any such evaluations!

Member

amueller commented Dec 9, 2012

Np. It would be great if there was something like that.

I actually have a student on the task of evaluating BRIEF, LBP, DAISY and SIFT for superpixel-bases segmentation. But he is kind of taking his time ;)

Having Daisy in here would definitely make the comparison easier.

Anders Boesen Lindbo Larsen added some commits Dec 15, 2012

Anders Boesen Lindbo Larsen Add circle perimeter to draw module. 2849cbf
Anders Boesen Lindbo Larsen Add DAISY visualization + example plot. 6c19cf0

I have added visualization of the descriptors and an example script to doc/examples:
daisy

I had to implement a circle drawing function since the draw module only supported filled circles.

Anders Boesen Lindbo Larsen Handle all non-float images. 7aa9435
Member

amueller commented Dec 16, 2012

that is pretty cool :)
To bad they don't sit on keypoints ;) But that looks really great!

Yep, interest point description would be useful!
I actually plan to work on interest points in the near future. I will consider doing a Python version if the Matlab code I'm considering goes bad. :)

@ahojnnes ahojnnes and 1 other commented on an outdated diff Dec 16, 2012

skimage/feature/_daisy.py
+ # Normalize descriptors.
+ if normalization != 'off':
+ descs += 1e-10
+ if normalization == 'l1':
+ descs /= np.sum(descs, axis=2)[:, :, np.newaxis]
+ elif normalization == 'l2':
+ descs /= sqrt(np.sum(descs ** 2, axis=2))[:, :, np.newaxis]
+ elif normalization == 'daisy':
+ for i in range(0, desc_dims, orientations):
+ norms = sqrt(np.sum(descs[:, :, i:i + orientations] ** 2,
+ axis=2))
+ descs[:, :, i:i + orientations] /= norms[:, :, np.newaxis]
+
+ if visualize:
+ from skimage import draw
+ from skimage import color
@ahojnnes

ahojnnes Dec 16, 2012

Owner

Is there a specific reason to have the imports here? In general I would move them to the top.

@andersbll

andersbll Dec 16, 2012

I copied that code practice from the HOG-implementation. I also prefer imports at the top.

@ahojnnes ahojnnes and 1 other commented on an outdated diff Dec 16, 2012

skimage/feature/_daisy.py
+ # Draw histogram bins
+ bin_size = descs[i, j, orientations + r_num *
+ histograms * orientations +
+ t_num * orientations + o_num]
+ bin_size /= max_bin
+ dy = sigmas[r_num + 1] * bin_size * sin(o)
+ dx = sigmas[r_num + 1] * bin_size * cos(o)
+ coords = draw.line(hist_y, hist_x, hist_y + dy,
+ hist_x + dx)
+ set_color(descs_img, coords, color)
+ return descs, descs_img
+ else:
+ return descs
+
+
+def set_color(img, coords, color):
@ahojnnes

ahojnnes Dec 16, 2012

Owner

You should be able to do:

rr, cc = coords
rr_inside = np.logical_and(rr >= 0, rr < img.shape[0])
cc_inside = np.logical_and(cc >= 0, cc < img.shape[1])
inside = np.logical_and(rr_inside, cc_inside)
img[rr[inside], cc[inside]] = color

And I think it might be a good idea to move this function as a public helper function in the draw module. This might be useful for other people as well.

@andersbll

andersbll Dec 16, 2012

I agree, will do!

Owner

ahojnnes commented Dec 16, 2012

Very nice contribution!

@andersbll if you are interested in interest point detection, I have an open PR with some of the basic interest point detectors (see #321).

@ahojnnes: Thanks for reference!

Anders Boesen Lindbo Larsen added some commits Dec 16, 2012

Anders Boesen Lindbo Larsen Move set_color() to draw module. 097f25d
Anders Boesen Lindbo Larsen No need to normalize angular smoothing. 3b43cf0
Owner

ahojnnes commented Dec 16, 2012

From what I can tell, this PR seems to be ready to merge. Although you still want to make some changes @andersbll?

I believe it's pretty solid by now. Thank you for the comments and corrections.

Anders Boesen Lindbo Larsen Fixed submodule import. c276ec0

Ok, this time i mean it! :)

Owner

stefanv commented Dec 23, 2012

I have some code for extracting FAST features. Could DAISY and FAST be combined to do keypoints?

Owner

ahojnnes commented Dec 26, 2012

I guess we are done here - thank you very much.

@andersbll If you like, you can add yourself to the list of contributors in CONTRIBUTORS.txt berfore I finally merge this PR?

Anders Boesen Lindbo Larsen Add myself as contributor. 8b6b51e

@stefanv: ahojnnes has already pointed me to some interest point detectors. Unfortunately, it's not that simple to convert dense feature description to interest points as individual interest point description is more complex. Maybe another day/PR! :)

@ahojnnes: I have added myself, thanks.

Owner

ahojnnes commented Dec 27, 2012

@andersbll Sorry, but it seems we need another rebase on the current master. Could you do that?

Anders Boesen Lindbo Larsen added some commits Dec 8, 2012

Anders Boesen Lindbo Larsen Add dense DAISY feature description. 4173d16
Anders Boesen Lindbo Larsen PEP8 style corrections. 24d3921
Anders Boesen Lindbo Larsen Better Sphinx documentation. 633443e
Anders Boesen Lindbo Larsen Add circle perimeter to draw module. 2313361
Anders Boesen Lindbo Larsen Add DAISY visualization + example plot. 6770161
Anders Boesen Lindbo Larsen Handle all non-float images. 74e6c38
Anders Boesen Lindbo Larsen Move set_color() to draw module. b770578
Anders Boesen Lindbo Larsen No need to normalize angular smoothing. 8d8b8ef
Anders Boesen Lindbo Larsen Fixed submodule import. 4206804
Anders Boesen Lindbo Larsen Add myself as contributor. 101bcc6
Anders Boesen Lindbo Larsen Merge branch 'master' of github.com:andersbll/scikit-image
Conflicts:
	CONTRIBUTORS.txt
1c674f6

@ahojnnes: I hope this resolves the conflicts you encountered.

@ahojnnes ahojnnes added a commit that referenced this pull request Dec 30, 2012

@ahojnnes ahojnnes Merge pull request #384 from andersbll/master
Dense DAISY feature description
7d328ae

@ahojnnes ahojnnes merged commit 7d328ae into scikit-image:master Dec 30, 2012

1 check passed

default The Travis build passed
Details
Owner

ahojnnes commented Dec 30, 2012

Merged #384.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment