Incorporating RViz2 into custom GUI (Qt app)

[Abishalini]

I am trying to port librviz to ROS2/RViz2
I could not find visualization_manager class (I built RViz2 along with ROS2 and I couldn't find the appropriate files for it on local). So I copied over visualization_manager.cpp/hpp into my workspace and linked it to rviz_common::rviz_common in CMakeLists.txt

Doing so gave me the following errors -

[rviz_rendering:debug] Available Renderers(1): OpenGL Rendering Subsystem, at /tmp/binarydeb/ros-foxy-rviz-rendering-8.2.1/src/rviz_rendering/render_system.cpp:259
[rviz_rendering:info] Stereo is NOT SUPPORTED, at /tmp/binarydeb/ros-foxy-rviz-rendering-8.2.1/src/rviz_rendering/render_system.cpp:508
[rviz_rendering:info] OpenGl version: 4.6 (GLSL 4.6), at /tmp/binarydeb/ros-foxy-rviz-rendering-8.2.1/src/rviz_rendering/render_system.cpp:242
[rviz_common:debug] Load pixmap at package://rviz_default_plugins/icons/classes/TF.svg, at /tmp/binarydeb/ros-foxy-rviz-common-8.2.1/src/rviz_common/load_resource.cpp:71
[rviz_common:debug] Error retrieving file [file:///opt/ros/foxy/share/rviz_default_plugins/icons/classes/TF.svg]: Couldn't open file /opt/ros/foxy/share/rviz_default_plugins/icons/classes/TF.svg, at /tmp/binarydeb/ros-foxy-rviz-common-8.2.1/src/rviz_common/load_resource.cpp:55
[rviz_common:debug] Load pixmap at package://rviz_default_plugins/icons/classes/TF.png, at /tmp/binarydeb/ros-foxy-rviz-common-8.2.1/src/rviz_common/load_resource.cpp:71
[rviz_common:debug] Load pixmap at package://rviz_common/icons/classes/Identity.svg, at /tmp/binarydeb/ros-foxy-rviz-common-8.2.1/src/rviz_common/load_resource.cpp:71
[rviz_common:debug] Error retrieving file [file:///opt/ros/foxy/share/rviz_common/icons/classes/Identity.svg]: Couldn't open file /opt/ros/foxy/share/rviz_common/icons/classes/Identity.svg, at /tmp/binarydeb/ros-foxy-rviz-common-8.2.1/src/rviz_common/load_resource.cpp:55
[rviz_common:debug] Load pixmap at package://rviz_common/icons/classes/Identity.png, at /tmp/binarydeb/ros-foxy-rviz-common-8.2.1/src/rviz_common/load_resource.cpp:71
[rviz_common:debug] Error retrieving file [file:///opt/ros/foxy/share/rviz_common/icons/classes/Identity.png]: Couldn't open file /opt/ros/foxy/share/rviz_common/icons/classes/Identity.png, at /tmp/binarydeb/ros-foxy-rviz-common-8.2.1/src/rviz_common/load_resource.cpp:55
[rviz_common:debug] Load pixmap at package://rviz_common/icons/default_class_icon.png, at /tmp/binarydeb/ros-foxy-rviz-common-8.2.1/src/rviz_common/load_resource.cpp:71

when I tried to make an instance of the visualization_manager class

render_panel_ = new rviz_common::RenderPanel();

rviz_common::ros_integration::RosNodeAbstractionIface::WeakPtr ros_node_abstraction_ = std::weak_ptr<rviz_common::ros_integration::RosNodeAbstractionIface>();
rviz_common::WindowManagerInterface* wm_ = nullptr;
rclcpp::Clock::SharedPtr clock_ = std::make_shared<rclcpp::Clock>(RCL_SYSTEM_TIME);

manager_ = new rviz_common::VisualizationManager( render_panel_ , ros_node_abstraction_, wm_,  clock_); 

How do I go about this? The goal is to incorporate Rviz into a custom Qt App

[schornakj]

I'm also very interested in figuring out how to do this, since embedded Rviz displays are a useful feature to have in nicely-packaged custom robotics applications.

[schornakj]

I wanted to provide some detailed information about what I'm trying to do and my current issues. I've been referring to the functions used while initializing the rviz2 application while trying to get this working. This is also building from the code that @Abishalini wrote while originally working on this issue.

I created a branch of ros2/rviz that moves the VisualizationManager to the public API, so that's what I'm linking against in the code below. Based on examples from ROS1 for how to set up a custom application like this I think this change is needed to achieve my desired result, but since my application doesn't work yet that's a speculative assumption.

Basically I would like to have an app that looks something like this (minus the sliders and labels -- I deleted those from my code for clarity):

Please let me know if I should provide additional information. One of the motivations behind this is to be able to port the MoveIt Setup Assistant to ROS2, since the ROS1 version of that application follows a similar pattern.

Code

myviz.h

#pragma once

#include <QWidget>
#include <QMainWindow>
#include <rclcpp/rclcpp.hpp>

namespace rviz_common
{
class Display;
class RenderPanel;
class VisualizationManager;
}

class MyViz: public QMainWindow, public rclcpp::Node
{
Q_OBJECT
public:
  MyViz( QWidget * parent = 0 );

private:
  rviz_common::VisualizationManager* manager_;
  rviz_common::RenderPanel* render_panel_;
  rviz_common::Display* grid_;
};

myviz.cpp

#include "myviz.h"

#include <QColor>
#include <QGridLayout>
#include <QVBoxLayout>

#include "rclcpp/clock.hpp"
#include "rviz_common/render_panel.hpp"
#include "rviz_common/display.hpp"
#include <rviz_common/display_context.hpp>
#include "rviz_common/ros_integration/ros_node_abstraction_iface.hpp"
#include "rviz_common/ros_integration/ros_client_abstraction.hpp"
#include "rviz_common/ros_integration/ros_node_abstraction.hpp"
#include "rviz_common/visualization_manager.hpp"
#include "rviz_rendering/render_window.hpp"

MyViz::MyViz( QWidget* parent )
  : QMainWindow( parent )
  , Node("rviz_gui")

{
  QWidget * center_widget = new QWidget(this);

  auto ros_node_abs = std::make_shared<rviz_common::ros_integration::RosNodeAbstraction>("rviz_render_node");
  rviz_common::WindowManagerInterface* wm_ = nullptr;
  auto clock_ = std::make_shared<rclcpp::Clock>(RCL_SYSTEM_TIME);

  render_panel_ = new rviz_common::RenderPanel();
  manager_ = new rviz_common::VisualizationManager( render_panel_ , ros_node_abs, wm_,  clock_);

  QVBoxLayout* main_layout = new QVBoxLayout;
  main_layout->addWidget( render_panel_ );

  render_panel_->getRenderWindow()->initialize();

  render_panel_->initialize(manager_);
  manager_->initialize();
  manager_->startUpdate();

  center_widget->setLayout(main_layout);
  this->setCentralWidget(center_widget);

  // Create a Grid display.
  // TODO: currently crashes due to failure to load rviz/Grid plugin, need to revisit
//  grid_ = manager_->createDisplay( "rviz/Grid", "adjustable grid", true );
//  assert( grid_ != NULL );
//  grid_->subProp( "Line Style" )->setValue( "Billboards" );
//  grid_->subProp( "Color" )->setValue( QColor( Qt::yellow ) );
}

main.cpp

#include <QApplication>
#include "myviz.h"

int main(int argc, char *argv[])
{
  QApplication app( argc, argv );
  rclcpp::init(argc, argv);
  auto myviz = std::make_shared<MyViz>();
  myviz->show();
  rclcpp::executors::SingleThreadedExecutor exec;
  exec.add_node(myviz);
  while (rclcpp::ok())
  {
      app.processEvents();
      exec.spin_some();
  }
}

CMakeLists.txt

cmake_minimum_required(VERSION 3.5)
project(rviz_embed_test)

# Default to C++14
if(NOT CMAKE_CXX_STANDARD)
  set(CMAKE_CXX_STANDARD 14)
endif()

find_package(ament_cmake REQUIRED)
find_package(rclcpp REQUIRED)
find_package(rviz_common REQUIRED)
find_package(rviz_default_plugins REQUIRED)
find_package(rviz_ogre_vendor REQUIRED)
find_package(rviz_rendering REQUIRED)

find_package(Qt5Core REQUIRED)
find_package(Qt5Quick REQUIRED)
find_package(Qt5Widgets REQUIRED)

set(THIS_PACKAGE_INCLUDE_DEPENDS
  rclcpp
  rclcpp_components
  rviz_common
  rviz_default_plugins
  rviz_ogre_vendor
  rviz_rendering
)

include_directories(
  ${rclcpp_INCLUDE_DIRS}
  ${Qt5Core_INCLUDE_DIRS}
  ${Qt5Quick_INCLUDE_DIRS}
  ${Qt5Widgets_INCLUDE_DIRS}
)

set(CMAKE_AUTOMOC ON)
set(CMAKE_AUTOUIC ON)
set(CMAKE_AUTORCC ON)
set(CMAKE_INCLUDE_CURRENT_DIR ON)

set(SRC_FILES
  src/myviz.cpp
  src/main.cpp
)

add_executable(${PROJECT_NAME} ${SRC_FILES})
ament_target_dependencies(${PROJECT_NAME} ${THIS_PACKAGE_INCLUDE_DEPENDS})

target_link_libraries(${PROJECT_NAME}
  Qt5::Core
  Qt5::Quick
  Qt5::Widgets
  rviz_common::rviz_common
  rviz_rendering::rviz_rendering
  rviz_ogre_vendor::OgreMain
  rviz_ogre_vendor::OgreOverlay
  )

## Install
install(TARGETS ${PROJECT_NAME} RUNTIME DESTINATION lib/${PROJECT_NAME})

ament_package()

Exception Info

Currently, I get a segfault in the function Ogre::GLXWindow::windowMovedOrResized(), specifically in these lines:

XQueryTree(xDisplay, mWindow, &root, &parent, &children, &nChildren);
if (children)
  XFree(children);

It looks like children and nChildren get populated with garbage data when XQueryTree is called. In the rviz2 application, children gets set to a null pointer by this function, which causes XFree to be skipped. In my application, since children is a non-null pointer (to an invalid address), we end up going into that if statement and segfault when we try to call XFree.

Stack Trace

1  __GI_raise                                                 raise.c                     50   0x7fa29d29f18b 
2  __GI_abort                                                 abort.c                     79   0x7fa29d27e859 
3  __libc_message                                             libc_fatal.c                155  0x7fa29d2e93ee 
4  malloc_printerr                                            malloc.c                    5347 0x7fa29d2f147c 
5  _int_free                                                  malloc.c                    4177 0x7fa29d2f2cbc 
6  XFree                                                                                       0x7fa29a70554d 
7  Ogre::GLXWindow::windowMovedOrResized                      OgreGLXWindow.cpp           694  0x7fa2801f2e2d 
8  rviz_rendering::RenderWindowImpl::resize                   ogre_render_window_impl.cpp 267  0x7fa29c62ddc5 
9  rviz_rendering::RenderWindow::windowMovedOrResized         render_window.cpp           123  0x7fa29c63925c 
10 rviz_common::RenderPanel::resizeEvent                      render_panel.cpp            300  0x7fa29e7cbac4 
11 QWidget::event(QEvent *)                                                                    0x7fa29dfa3947 
12 QApplicationPrivate::notify_helper(QObject *, QEvent *)                                     0x7fa29df60a66 
13 QApplication::notify(QObject *, QEvent *)                                                   0x7fa29df6a0f0 
14 QCoreApplication::notifyInternal2(QObject *, QEvent *)                                      0x7fa29d8cf93a 
15 QWidgetPrivate::sendPendingMoveAndResizeEvents(bool, bool)                                  0x7fa29df9be6e 
16 QWidgetPrivate::show_helper()                                                               0x7fa29df9fbd7 
17 QWidgetPrivate::setVisible(bool)                                                            0x7fa29dfa2d6b 
18 QWidgetPrivate::showChildren(bool)                                                          0x7fa29df9fb61 
19 QWidgetPrivate::show_helper()                                                               0x7fa29df9fbf3 
20 QWidgetPrivate::setVisible(bool)                                                            0x7fa29dfa2d6b 
21 main                                                       main.cpp                    17   0x559e6196fba4 

[schornakj]

It's possibly worth noting that I also get errors about failures to load certain icon images:

[rviz_common:debug] Error retrieving file [file:///home/jschornak/workspaces/moveit2_foxy_debug_ws/install/rviz_default_plugins/share/rviz_default_plugins/icons/classes/MoveCamera.svg]: Couldn't open file /home/jschornak/workspaces/moveit2_foxy_debug_ws/install/rviz_default_plugins/share/rviz_default_plugins/icons/classes/MoveCamera.svg, at /home/jschornak/workspaces/moveit2_foxy_debug_ws/src/rviz/rviz_common/src/rviz_common/load_resource.cpp:55

In this case the image MoveCamera.svg does not exist at the requested path.

[clalancette]

Assigning to @jacobperron to take a look. Most likely this results in this being marked as 'help-wanted' or 'backlog'. Contributions here would be welcome.

[schornakj]

@clalancette We were able to work through this and get our application working. I'll find a good place to post a minimal open-source example based on that librviz tutorial. I'll also post some follow-up comments here.

Some changes to the core RViz packages were required, so I submitted #649.

[clalancette]

Great, thanks for the heads up.

[JaehyunShim]

@schornakj Did it work for you guys? I downloaded the source code and tried it but it dies when calling the this function

Thank you in advance
Jaehyun

[clalancette]

Closing this as #649 was merged.

[SEUZTh]

Do you have the missing SVG images now

[SEUZTh]

Hi, have you solve the problem?