Performance issue on iPhone

[aledalgrande]

Hi Pablo,

I am trying to see if I can work with AKAZE on my mobile project, but it seems quite slow compared to ORB. I think the difference is about an order of magnitude, as OpenCV ORB extracts and matches the features in under a second, while AKAZE takes several seconds (binary descriptor). Is this expected or am I not using the library in the right way?

Feature extraction:

AKAZEOptions options;
options.descriptor = MLDB;
options.img_width = view.cols;
options.img_height = view.rows;
Mat view32;
view.convertTo(view32, CV_32F, 1.0/255.0,0);
AKAZE akazeDetector(options);
akazeDetector.Create_Nonlinear_Scale_Space(view32);
vector<KeyPoint> keypoints;
akazeDetector.Feature_Detection(keypoints);
Mat descriptors;
akazeDetector.Compute_Descriptors(keypoints, descriptors);
pair<Mat,vector<KeyPoint> > result(descriptors, keypoints);
return result;

Feature matching:

vector<Point2f> matches, inliers;
Ptr<DescriptorMatcher> matcherL1 = DescriptorMatcher::create("BruteForce-Hamming");
vector<vector<cv::DMatch> > dmatches;
matcherL1->knnMatch(firstViewFeatures.first, secondViewFeatures.first, dmatches, 2);
matches2points_nndr(firstViewFeatures.second, secondViewFeatures.second, dmatches, matches, 0.80f);
compute_inliers_ransac(matches, inliers, 2.50f, false);
vector<Point2f> points1, points2;

for (int i = 0; i < inliers.size(); i+=2)
{
    points1.push_back(inliers[i]);
    points2.push_back(inliers[i+1]);
}

return pair<vector<Point2f>,vector<Point2f> >(points1, points2);

The example images I'm working with:

Thanks

[pmoulon]

Hi Alessandro.
Did you have compare the feature count in ORB and in AKAZE mode?
Perhaps the timings is different because ORB detects less features.

You can also see how many octave ORB handle and AKAZE handle by default.

[pablofdezalc]

Hi Alessandro,

Be sure you compile AKAZE with paralellization support and optimized compiling options.
In any case, AKAZE detector is slower to the one in ORB. However, the performance of AKAZE detector is much better at the cost of being slower.

[aledalgrande]

Hi guys, yes it might be I didn't compile with parallelization, as I imported in XCode. Is the number of octaves the nsublevels in AKAZE?

[pablofdezalc]

The number of octaves is different than the number of sublevels. The number of sublevels is the number of levels you compute for one octave in the scale space. You can reduce the number of octaves and sublevels to get faster detection.

[aledalgrande]

I tried today with omax = 2 and nsublevels = 1, it improved to 6-7 seconds. I still want to have it sub-second, probably need to dive more in the code, but from my experiments with findHomography it's the only feature detector/extractor/matcher that gives a really good, near perfect result! That's astounding. I will experiment more today with the 5-point algorithm, which is recovering the pose for me and that's the reason for which I switched to AKAZE (tried ORB and SURF before that).

[pmoulon]

You can also make the resize your image too a smaller size before detection
and let default settings for omax and nsublevels if you see that matching
performance are still good.

2014-07-12 3:22 GMT+02:00 Alessandro Dal Grande notifications@github.com:

I tried today with omax = 2 and nsublevels = 1, it improved to 6-7
seconds. I still want to have it sub-second, probably need to dive more in
the code, but from my experiments with findHomography it's the only
feature detector/extractor/matcher that gives a really good, near perfect
result! That's astounding. I will experiment more today with the 5-point
algorithm, which is recovering the pose for me and that's the reason for
which I switched to AKAZE (tried ORB and SURF before that).

—
Reply to this email directly or view it on GitHub
#16 (comment).

Regards/Cordialement,
Pierre M

[aledalgrande]

It's already resized to 640x480 so don't think it will improve with a smaller format. I think it's not parallel at all as I imported vanilla AKAZE into Xcode and clang doesn't support OMP (not sure Arm supports it anyway). If I paralellized or delegated to GPU I would speed up. The feature extraction seems fast, most of the time is spent on matching, I usually get 10-100 valid matches.—
Alessandro Dal Grande

On Fri, Jul 11, 2014 at 11:27 PM, Pierre Moulon notifications@github.com
wrote:

You can also make the resize your image too a smaller size before detection
and let default settings for omax and nsublevels if you see that matching
performance are still good.
2014-07-12 3:22 GMT+02:00 Alessandro Dal Grande notifications@github.com:

I tried today with omax = 2 and nsublevels = 1, it improved to 6-7
seconds. I still want to have it sub-second, probably need to dive more in
the code, but from my experiments with findHomography it's the only
feature detector/extractor/matcher that gives a really good, near perfect
result! That's astounding. I will experiment more today with the 5-point
algorithm, which is recovering the pose for me and that's the reason for
which I switched to AKAZE (tried ORB and SURF before that).

—
Reply to this email directly or view it on GitHub
#16 (comment).

Regards/Cordialement,

Pierre M

Reply to this email directly or view it on GitHub:
#16 (comment)

[brspurri]

I use it on my iPhone as well. I find that the image size makes the largest difference in speed, so you'll need to optimize primarily based on that. I resize all my images to 200px wide, and I'm using omax=8 and nsublevels=4. This generally results in about 100 keypoints per image (if there are a lot of heavy features). With the added performance of AKAZE, I see this as being enugh for accurate matching.

When running this way, my code performs the feature extraction and description is 150ms per image.

I'm using a different matching algorithm, so I can't really comment on speed there.

Also, there is an OpenCV form by @BloodAxe (https://github.com/BloodAxe/opencv) that contains KAZE/AKAZE. He integrates OpenCV parallel_for_ method in a lot of places AKAZE uses OMP. Bear in mind, this enables parallelizing on the iPhone, but only to two cores because thats all the phone has (as of now).

[aledalgrande]

I don't think OpenMP is supported by CLANG and iOS?

[brspurri]

No, OpenMP is defiantly not supported in iOS. However, cv::parallel_for abstracts the parallelization on for specific devices. So for iOS, cv::parallel_for_ spawns embarrassingly parallel jobs via GCD ( https://github.com/Itseez/opencv/blob/master/modules/core/src/parallel.cpp).

It's a great tool, but admittedly not very well documented.

AKAZE, out of the box, will not utilize this without code changes, but the @BloodAxe fork of OpenCV has done a good job of doing just that.

[aledalgrande]

Oh thanks, very insightful.

Alessandro Dal Grande

On Wednesday, 13 August 2014 at 14:07, brspurri wrote:

No, OpenMP is defiantly not supported in iOS. However, cv::parallel_for abstracts the parallelization on for specific devices. So for iOS, cv::parallel_for_ spawns embarrassingly parallel jobs via GCD ( https://github.com/Itseez/opencv/blob/master/modules/core/src/parallel.cpp).
It's a great tool, but admittedly not very well documented.
AKAZE, out of the box, will not utilize this without code changes, but the @BloodAxe (https://github.com/bloodaxe) fork of OpenCV has done a good job of doing just that.

—
Reply to this email directly or view it on GitHub (#16 (comment)).

[pablofdezalc]

If you use a small image like 200px wide, there is no point in using omax=8. By default in AKAZE the smallest image size that handles is 80x40, so it will be using a smaller number of octaves.

The bloodaxe fork of OpenCV has been improved considerably thanks to a Google Summer of Code Program and OpenCV. There are also some speed-ups about 20-30%. The code will come out soon in OpenCV master branch, and I will also update the AKAZE library.

[aledalgrande]

Seems to be working fast enough on OpenCV 3.0