rviz plugin: joints tab #1308

rhaschke · 2019-01-15T09:49:56Z

I was missing for a long time the possibility to directly actuate joints in the rviz motion planning plugin.
This PR adds a new tab "Joints" to the panel, which lists all joints of the current move group and allows to operate them via sliders within their bounds. For unbounded joints, an editing field will be displayed.
Additionally to this, sliders are displayed to navigate the redundant space of the robot. If there is only a single redundant DoF, the approach works quite nice already (see the Panda video).
For multiple redundant DoFs, we need some more work to nicely align nullspace directions between different frames.

This PR also introduces methods harmonizePositions() in JointModel and RobotState that map joint angles of revolute joints that are outside the boundaries back into the feasible range by adding / subtracting multiples of 2pi. Such a function was implemented in several IK plugins before - I have seen it in LMA and IKFast...

simonschmeisser · 2019-01-16T15:16:29Z

Looks nice! The redundancy slider is not completely obvious to me however. Maybe move it below the joints list with a clear visual separation and a header? Will this slider also appear in singularities?

rhaschke · 2019-01-17T08:13:46Z

Maybe move it below the joints list with a clear visual separation and a header?

To emphasize the different semantics of the NS sliders, I deliberately placed them next to the joint sliders and not below. If there are many joints, the NS sliders wouldn't be visible as well.

Will this slider also appear in singularities?

Yes, of course. As an example, have a look at the Panda arm in zero pose, which has two nullspace dimensions (because three joint axes are aligned).

moveit_core/robot_state/include/moveit/robot_state/robot_state.h

moveit_core/robot_model/include/moveit/robot_model/joint_model.h

moveit_core/robot_model/src/joint_model.cpp

moveit_ros/visualization/motion_planning_rviz_plugin/src/motion_planning_frame_joints.cpp

davetcoleman · 2019-01-19T00:56:54Z

This is a great feature, well done!

Maybe move it below the joints list with a clear visual separation and a header?

I agree with @simonschmeisser the current UI layout isn't the most intuitive. We need some kind of label or icon indicating what the vertical slider does and how its different than the other sliders. I like his idea of having it below.

Also, the highlighting of a particular joint row is confusing to me - can we disable that behavior? In the gif it mislead my initial understanding of what the redundant slider does.

Not necessary in this PR but just a future suggestion: in the original arm_navigation project we had circular sliders overlayed on each joint that allowed this same per-joint functionality. It was pretty slick and was basically a simplified interactive marker:

gavanderhoorn · 2019-01-19T16:30:25Z

@davetcoleman wrote:

Not necessary in this PR but just a future suggestion: in the original arm_navigation project we had circular sliders overlayed on each joint that allowed this same per-joint functionality. It was pretty slick and was basically a simplified interactive marker:

Something like this: awesomebytes/move_joints_interactive?

It's not perfect, but is the closest to how arm_navigation did it I've found so far.

rhaschke · 2019-01-21T08:18:29Z

I like his idea of having the sliders below.

Ok, moved them below and added a label now.

The highlighting of a particular joint row is confusing to me - can we disable that behavior?

This is the standard highlighting of the selected joint row (which stays highlighted even if focus moves somewhere else). @davetcoleman, what kind of behaviour do you want to have?

... in the original arm_navigation project we had circular sliders overlayed on each joint ...

I like the idea to have joint sliders, but only if there is no IK solver available / no Cartesian marker.
Otherwise the multitude of markers will become overwhelming / confusing too. Having Cartesian markers on all end-effectors (finger tips) of our bimanual Shadow hand setup as well as two markers on the arm end-effectors, we already sum up to 14 markers:

Also, choosing the right size and position of the joint markers is tricky...

davetcoleman · 2019-01-22T21:04:25Z

Something like this: awesomebytes/move_joints_interactive?

Yes exactly. I think this behavior should be an optional view that you wouldn't want at the same time as the normal goal pose interactive marker.

This whole interface @rhaschke is adding could likely be in its own Display panel, rather than inside the moveit one. What do you think about that @rhaschke? Or are you using a lot of IK dependencies that would be bulky to load in a different Rviz plugin?

rhaschke · 2019-01-22T21:23:26Z

I think this behavior should be an optional view that you wouldn't want at the same time as the normal goal pose interactive marker.

I agree. I was thinking of this mode as a default when no Cartesian pose marker is available. Using the marker menu, we could switch between these modes.

This whole interface @rhaschke is adding could likely be in its own Display panel.

I don't agree. This interface was originally intended to monitor the joint values (particularly closeness to joint limits) of the start and goal query states. Hence, it needs to be integrated with the current display.

davetcoleman · 2019-01-22T21:27:59Z

It concerns me we now need scroll bars to access all the tabs in the motion planning plugin:

I think this new functionality could be a separate panel.

I'm really excited with this PR playing around with the joints:

However:

The slider interface is very hard to use. It took me a while to realize you have to double click to gain control of the slider bars. Once 'enabled', you don't need to hold down the mouse. But you do have to keep the mouse hovering inside the progress bar box. Why did you choose this non-standard GUI approach?
The nullspace exploration bar would make you think it represents the min and max range of the free joint. In fact all it really is doing is providing a left and right button without any known min or max. I think two buttons with arrows would be more intuitive e.g. < >

moveit_core/robot_model/include/moveit/robot_model/revolute_joint_model.h

moveit_core/robot_model/src/revolute_joint_model.cpp

...anning_rviz_plugin/include/moveit/motion_planning_rviz_plugin/motion_planning_frame_joints.h

moveit_ros/visualization/motion_planning_rviz_plugin/src/motion_planning_frame_joints.cpp

davetcoleman · 2019-01-22T21:23:41Z

moveit_ros/visualization/motion_planning_rviz_plugin/src/motion_planning_frame_joints.cpp

+{
+  ui_->joints_view_->setModel(model);
+  ui_->joints_view_label_->setText(
+      QString("Group joints of %1 state").arg(model == start_state_model_.get() ? "start" : "goal"));


Suggested change

QString("Group joints of %1 state").arg(model == start_state_model_.get() ? "start" : "goal"));

QString("Manual Joint Adjustment of %1 State:").arg(model == start_state_model_.get() ? "Start" : "Goal"));

As said, I see this primarily as a display and not as an adjustment tool.
Why do you want to change case of all worlds?

When I ran your code locally it looked sloppy without capitalization, as I see this as a section title

Also, you presented this primarily as a adjustment tool in the description of this PR:

I was missing for a long time the possibility to directly actuate joints in the rviz motion planning plugin.

Touché. But, actually I started from a display and it turned into an interaction later...
No matter. I don't like to switch to "normal" sliders for the following reasons:

Continuous or unbounded joints cannot be handled with a slider in a meaningful way. For them, an input field is necessary.

Using a model-view approach is much more flexible here.

I could have a look into how to improve the interaction though. I remember there are several modes available for a view to enter the editor of an item. But that's nothing I can promise anytime soon. MoveIt is already consuming way to much of my time and there definitely more pressing issues in MoveIt.

Why not just have a QT Group (perhaps encapsulated as a widget) with horizontal layout with a label widget and traditional slider widget? The slider could still reflect the status of the joints via update calls to adjust them automatically.

That's exactly what I suggested, although there is nothing like a Qt Group. Do you mean QGroupBox? I guess, we don't want to have it's frame.

I agree that the look-and-feel is not yet optimal. But, just to clarify the situation: If we don't drop the idea of the model-view approach (which I will not do, somebody else is cordially invited to do so though), we cannot show the sliders all the time. If editing is triggered, the display will change from a progress bar to a slider (+ label), which I think is strange too.

Yes I think QGroupBox is what I meant, from the top of my head. I'm fairly certain you can disable the frame via the properties as I've done that before.

I'm not sure why you are saying the model-view approach is preventing you from having a better UI. Why could there not be a callback to auto-adjust the sliders as the robot moves, which is disabled if the user clicks on the sliders?

QGroupBox is intended to group buttons, not arbitrary widgets, but yeah might work for this purpose.

The model-view approach distinguishes between display and editing. Usually, there is only one editor (slider) open in a view at any time. If that's fine for you, great. If you want to see all the sliders all the time, we need a completely different approach, namely creating all those slider widgets as you do, e.g. in setup assistant.

rhaschke · 2019-01-22T21:41:21Z

It concerns me we now need scroll bars to access all the tabs in the motion planning plugin

Not at my side. Seems like you use a wider system font...
I wouldn't care too much to have scroll bars though.

The slider interface is very hard to use. Why did you choose this non-standard GUI approach?

In first place this is a TableView and not a gui interaction. As I said, primarily I wanted to monitor the joint values (instead of changing them).

The nullspace exploration bar would make you think it represents the min and max range of the free joint.

There is no min and max written. Why would you think so? By the way, there is nothing like a "free joint", nor "nullspace coordinates". There is only a non-linear manifold which you can navigate.

In fact all it really is doing is providing a left and right button without any known min or max. I think two buttons with arrows would be more intuitive e.g. < >

No, that's not true. It's a velocity control slider: The more far to the left/right you move, the faster the robot moves in the nullspace. That's not possible with simple buttons.

davetcoleman · 2019-01-23T23:22:31Z

No, that's not true. It's a velocity control slider: The more far to the left/right you move, the faster the robot moves in the nullspace. That's not possible with simple buttons.

I observed it closer and I've confirmed it does move faster, my bad. It can easily be missed, though.

I think this is a great contribution and don't want to hold it up too long from being merged. However, unlike more internal code cleanliness and layout, its much less desirable to be iterating on user-facing UIs and I'd like to get that right now. From a product manager perspective, I'd like to see the individual joint slider interactions cleaned up before this gets merged

rhaschke · 2019-01-24T12:22:29Z

I tweaked the view settings a bit to get rid of the double-click experience (f4d33d2). However, mouse interaction still feels strange, as the progress-bar sliders reacts w/o mouse button being pressed and there is another click+release needed to finish the editor.
I also quickly added a QSlider as an editor (4ea2c87). Here usage is even worse: you cannot "leave" the editor other than selecting something else.
Maybe indeed, we need to drop the idea of using the model-view approach but use classic widgets instead. I don't have cycles for this though and everybody taking over is very welcome.

... to display joint values of recently modified start/goal state

…_widget.*

- immediately start editing when item is clicked once - drop indentation in first column

davetcoleman · 2019-02-28T06:01:14Z

I re-ran this locally and I appreciate the improved UI with slider widget. However, the robot doesn't move until the mouse looses focus from the slider bar section. This is still unintuitive...

davetcoleman

A lot of good work was put into this PR, yet it has sat dormant for too long. @rhaschke as long as you are open to us further tweaking the user interaction in the future, I think we should move forward with merging this asap.

davetcoleman · 2019-11-21T01:09:48Z

I changed the branch to master.

felixvd · 2019-11-26T01:49:21Z

I'm excited about this no matter if the sliders behave a little unintuitively. Would love to see it merged.

rhaschke · 2019-11-28T15:59:41Z

I've put some more effort into this to get a more intuitive/immediate mouse behaviour.

codecov-io · 2019-11-28T17:13:48Z

Codecov Report

Merging #1308 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #1308   +/-   ##
=======================================
  Coverage   47.53%   47.53%           
=======================================
  Files         290      290           
  Lines       22932    22932           
=======================================
  Hits        10901    10901           
  Misses      12031    12031

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a1c0ade...c982526. Read the comment docs.

- get rid of PercentageRole - use Q_PROPERTY for editor

rhaschke force-pushed the rviz-plugin-joints-tab branch from ae28160 to 8ef3438 Compare January 15, 2019 12:36

mlautman reviewed Jan 18, 2019

View reviewed changes

rhaschke force-pushed the rviz-plugin-joints-tab branch from 8ef3438 to ca1f14a Compare January 21, 2019 09:11

davetcoleman requested changes Jan 22, 2019

View reviewed changes

rhaschke added enhancement help wanted please consider improving this request! more work needed Everyone is invited to contribute labels Jan 24, 2019

rhaschke force-pushed the rviz-plugin-joints-tab branch from 4ea2c87 to 2bf863b Compare February 1, 2019 02:38

rhaschke added 7 commits February 4, 2019 17:32

"Joints" tab for MotionPlanningFrame

e1c3ac6

... to display joint values of recently modified start/goal state

ProgressBarDelegate

c750b08

ProgressBarEditor

5180c16

nullspace exploration

dbe88e3

harmonizePositions()

b5b7a5c

rename motion_planning_frame_joints.* -> motion_planning_frame_joints…

e95a13d

…_widget.*

tweak view settings for better user experience

921fdb9

- immediately start editing when item is clicked once - drop indentation in first column

rhaschke force-pushed the rviz-plugin-joints-tab branch from 2bf863b to 921fdb9 Compare February 4, 2019 16:33

davetcoleman approved these changes Nov 21, 2019

View reviewed changes

davetcoleman changed the base branch from melodic-devel to master November 21, 2019 01:09

rhaschke force-pushed the master branch from 68982c0 to 11c82ec Compare November 24, 2019 14:20

rhaschke added 3 commits November 28, 2019 16:56

Show joint value in edit mode as well

c9fd9c4

fix signed/unsigned warnings

bbdbb9a

improve editor activation

64f282f

rhaschke force-pushed the rviz-plugin-joints-tab branch from 626bbca to 64f282f Compare November 28, 2019 15:58

rhaschke requested a review from jonbinney as a code owner November 28, 2019 15:58

rhaschke removed help wanted please consider improving this request! more work needed Everyone is invited to contribute labels Nov 28, 2019

clang-tidy fixes

48381cd

rhaschke added the more work needed Everyone is invited to contribute label Nov 28, 2019

simplify code

c982526

- get rid of PercentageRole - use Q_PROPERTY for editor

rhaschke merged commit acead73 into moveit:master Nov 28, 2019

rhaschke deleted the rviz-plugin-joints-tab branch November 28, 2019 21:02

rhaschke added a commit to ubi-agni/moveit that referenced this pull request Nov 28, 2019

rviz plugin: Joints tab (moveit#1308)

345b716

felixvd mentioned this pull request Mar 7, 2020

Update tutorials, add default velocity/acceleration scaling factors moveit/moveit_tutorials#468

Merged

2 tasks

sjahr pushed a commit to sjahr/moveit that referenced this pull request Jun 21, 2024

Decorate MoveItConfigs with dataclaass (moveit#1308)

4502659

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

rviz plugin: joints tab #1308

rviz plugin: joints tab #1308

rhaschke commented Jan 15, 2019 •

edited

Loading

simonschmeisser commented Jan 16, 2019

rhaschke commented Jan 17, 2019

davetcoleman commented Jan 19, 2019

gavanderhoorn commented Jan 19, 2019

rhaschke commented Jan 21, 2019

davetcoleman commented Jan 22, 2019

rhaschke commented Jan 22, 2019

davetcoleman commented Jan 22, 2019

davetcoleman Jan 22, 2019

rhaschke Jan 22, 2019

davetcoleman Jan 22, 2019

davetcoleman Jan 22, 2019

rhaschke Jan 22, 2019

davetcoleman Jan 23, 2019

rhaschke Jan 23, 2019

rhaschke Jan 23, 2019

davetcoleman Jan 24, 2019

rhaschke Jan 24, 2019

rhaschke commented Jan 22, 2019

davetcoleman commented Jan 23, 2019 •

edited

Loading

rhaschke commented Jan 24, 2019

davetcoleman commented Feb 28, 2019

davetcoleman left a comment

davetcoleman commented Nov 21, 2019

felixvd commented Nov 26, 2019

rhaschke commented Nov 28, 2019

codecov-io commented Nov 28, 2019 •

edited

Loading

	QString("Group joints of %1 state").arg(model == start_state_model_.get() ? "start" : "goal"));
	QString("Manual Joint Adjustment of %1 State:").arg(model == start_state_model_.get() ? "Start" : "Goal"));

rviz plugin: joints tab #1308

rviz plugin: joints tab #1308

Conversation

rhaschke commented Jan 15, 2019 • edited Loading

simonschmeisser commented Jan 16, 2019

rhaschke commented Jan 17, 2019

davetcoleman commented Jan 19, 2019

gavanderhoorn commented Jan 19, 2019

rhaschke commented Jan 21, 2019

davetcoleman commented Jan 22, 2019

rhaschke commented Jan 22, 2019

davetcoleman commented Jan 22, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rhaschke commented Jan 22, 2019

davetcoleman commented Jan 23, 2019 • edited Loading

rhaschke commented Jan 24, 2019

davetcoleman commented Feb 28, 2019

davetcoleman left a comment

Choose a reason for hiding this comment

davetcoleman commented Nov 21, 2019

felixvd commented Nov 26, 2019

rhaschke commented Nov 28, 2019

codecov-io commented Nov 28, 2019 • edited Loading

Codecov Report

rhaschke commented Jan 15, 2019 •

edited

Loading

davetcoleman commented Jan 23, 2019 •

edited

Loading

codecov-io commented Nov 28, 2019 •

edited

Loading