OpenCV 3.1 MSER not thread-safe

[light64]

• OpenCV => 3.1.0-dev
• Operating System / Platform => Ubuntu 14.04 LTS 64-bit
• Compiler => gcc 4.8.4

It seems the Mser_Impl implementation of OpenCV 3.1.0-dev is not thread-safe for some reason.
When the Mser detection is started with OMP parallelization with 4-8 threads the program is stopped with a Segmentation fault error.
Each OMP thread holds its own instance of image, contours and bboxes, e.g.:

cv::Ptr<cv::MSER> mserD = cv::MSER::create(delta, ..., edgeBlurSize);

#pragma omp parallel for num_threads(8)
for (size_t image_id = 0; image_id < image_list.size(); image_id++) {
	
	cv::Mat image = cv::imread(image_list[image_id], cv::IMREAD_GRAYSCALE);

	std::vector<std::vector< cv::Point> > cur_contours;
	std::vector<cv::Rect> bboxes;

	mserD->detectRegions(image, cur_contours, bboxes);

	#pragma omp critical
	{
		// merge results
	}
}

Before I tried the last version of the OpenCV 2.4 branch and the Mser implementation using mserD->detect(...) worked perfectly with multithreading.

As segmentation faults are quite difficult to track down with multithreading I used valgrind.
Attached is the log file with some excerpts of the valgrind output.

It fails to read and write in updateTree, growHistory, checkAndCapture, etc. In all cases CompHistory is involved. In OpenCV 2.4 CompHistory does not exist in that form. I think this problem could only be solved by code analysis, as debugging with multithreading is irreproducible.

log.txt

[alalek]

Almost all OpenCV algorithms are not thread-safe by design (for performance reasons and code complexity).

[light64]

So far I have used various OpenCV implementations like BRISK, ORB and also MSER in the branch 2.4 and never had a problem with multithreading. The results also seemed quite reasonable and reproducible.
When I have a 40-CPU machine I would love to make use of all cores, by heading a different image for each thread over to the feature-detection or -description (e.g. for a database of 40000 images). When it is not thread-safe I can only process one image at a time.
In that case it makes a big difference if it takes 4 hours, or 160 hours to finish the calculation.

[alalek]

In general it is not very hard to add required synchronization primitives.
In your case try to move MSER algorithm creation into for body. It is not very efficient approach (thread-local initialization is more optimal via private/firstprivate OpenMP directives).

[light64]

Letting each thread individually initialize MSER is what I also thought of already. Though our framework usually creates the appropriate Feature-Detector and -Descriptor objects already beforehand for design reasons. For testing I can move the creation right before detectRegions and see whether the Segfault also appears in that case.

[light64]

UPDATE:
my fault, used the wrong binary.
So it works when creating an MSER object for each thread individually.
Sharing the same object between multiple threads doesn't seem to work with the current OpenCV 3.1.0 implementation.
In the latest OpenCV 2.4 branch implementation this works though.
Comparing OpenCV 3.1 with OpenCV 2.4 I would consider the MSER implementation as bug.
Otherwise I would put it as feature request to make the 3.1 implementation "thread-safe" again.

Letting each thread create its own MSER object fails with the same result.
Now I'm really confused. Each thread has its own MSER, image, vector<vector> contours, and vector bboxes.
I don't see another possibility for a race condition without shared variables.
Maybe that is a general issue of OpenCV 3, and other detectors also fail for multithreading, as MSER is now the first one I tried after upgrading to OpenCV 3.

[alalek]

It is not a bug.
Generic thread-safe implementation will be ineffective.

[krayni]

Dear all,
I try to test the text recognition in "multi-threading" (using parallel_for_) on a set of frames but I noticed that there is not speed ( independently on the number of thread).

My function is like

void text_recon(Mat &frameI)
{
//some code using MSER and Morphological Text Extraction;
}
parallel_for_ (CV::range(0,listframes.size()), paralleltext(listframes));