PCL 1.7.2 breaks OpenCV 3.1 highgui

[daandres]

Hi,

I want to use PCL and OpenCV in one project. PCL is used to visualize the point clouds, which I calculate with OpenCV.
But I also want to show my input images using OpenCV with its highgui.
But this is not working with PCL 1.7.2 and OpenCV 3.1

If PCL is used in my code I can't call namedWindow, imshow or waitKey from OpenCV anymore. The application terminates before executing any other line of code. There is no error message or something else.
If I debug this, I see in the stack trace, that there is an access to adress 0x0.

If either the PCL code is commented out or the OpenCV highgui the application works. Otherwise not.

Example code:

#include <opencv2/core/core.hpp>
#include <opencv2/highgui/highgui.hpp>
#include <iostream>

#include <boost/thread/thread.hpp>
#include <pcl/common/common_headers.h>
#include <pcl/visualization/pcl_visualizer.h>

int main(int argc, char* argv[])
{
std::cout << "Started" << std::endl;
boost::shared_ptr<pcl::visualization::PCLVisualizer> viewer(new pcl::visualization::PCLVisualizer("Viewer"));
cv::namedWindow("image", cv::WINDOW_AUTOSIZE);
cv::Mat img = cv::Mat::eye(640, 480, CV_64FC1);
cv::imshow("image", img);
cv::waitKey(0);
return 0;
}

Thanks for help

(tested on Ubuntu 14.04)

[taketwo]

PCL has an image viewer class, I'd recommend using it. Otherwise you are mixing different visualization frameworks (vtk from PCL side and whatever from OpenCV side), which never goes well.

[daandres]

thanks for fast reply.
the image viewer class isn't sufficient enough for me. I have to convert images and use only data part from opencv Mat class. Therefore I get crappy results.

On the other hand, I found many tutorials and examples, which use both frameworks. PCL is used for 3D PointClouds and OpenCV for 2D images.

OpenCV should only be used to display the input image, which was caputered by the camera.

[r9y9]

I'm using PCL master and opencv 3.1.0 and getting no problem for now. Perhaps disabling viz module (i.e. cmake -DBUILD_opencv_viz to build opencv) may help?

Complete stack traces would be great to track down the issue.

[r9y9]

Your example code works fine for me on osx 10.10.4. It showed two windows without segfaults.

EDIT: I'm using vtk trunk (v7.1), but as far as I remember, it also worked with vtk v6.3.0.

[daandres]

opencv without viz also doesn't work.
I use vtk from my distro repository. it should be vtk 6

stack trace isn't that maningful, but here it is:

Thread #1 [main] 13668 [core: 0] (Suspended : Signal : SIGSEGV:Segmentation fault)  
    0x7ffff024029c  
    QMetaType::registerNormalizedType() at 0x7fffe903177e   
    0x7fffe953f447  
    0x7ffff7dea13a  
    0x7ffff7dea223  
    0x7ffff7ddb30a  
    0x1 
    0x7fffffffe176  
    0x0 

[taketwo]

I wonder where Qt meta types come from?

[SergioRAgostinho]

The window backend used by OpenCV is Qt.
http://docs.opencv.org/3.1.0/d7/dfc/group__highgui.html#ga5afdf8410934fd099df85c75b2e0888b

[daandres]

OpenCV is compiled with QT support. But I compiled it now without Qt and then it works.
I've used Qt 5.

thanks

[TouchDeeper]

Hi,
I also want to use opencv to show 2d image and pcl to show the pointcloud and my pcl didn't break the opencv highgui. But the fps of the 2d image is too low as the pcl viewer work well.
The core code is below:

char key_cv = ' ';
while (key_cv != 'q' && !viewer->wasStopped()) {
    if (mutex_input.try_lock()) {
        float *p_data_cloud = data_cloud.getPtr<float>();
        int index = 0;

        // Check and adjust points for PCL format
        for (auto &it : p_pcl_point_cloud->points) {
            float X = p_data_cloud[index];
            if (!isValidMeasure(X)) // Checking if it's a valid point
                it.x = it.y = it.z = it.rgb = 0;
            else {
                it.x = X;
                it.y = p_data_cloud[index + 1];
                it.z = p_data_cloud[index + 2];
                it.rgb = convertColor(p_data_cloud[index + 3]); // Convert a 32bits float into a pcl .rgb format
            }
            index += 4;
        }
        // Unlock data and update Point cloud
        mutex_input.unlock();
        viewer->updatePointCloud(p_pcl_point_cloud);
        viewer->spinOnce(10);
    }
    // else
    //      sleep_ms(1);

    // Check that grab() is successful
    if (zed.grab(runtime_parameters) == SUCCESS) {
        // Retrieve left image
        zed.retrieveImage(image_zed, VIEW_LEFT, MEM_CPU, new_width, new_height);
        // Display image with OpenCV
        cv::imshow("VIEW", image_ocv);
        key_cv = cv::waitKey(10);
    }
}`

Can you give me some advice? Thanks a lot!