Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Attempt to use SCHED_FIFO for Servo regardless of RT kernel #2653

Merged
merged 4 commits into from
Jan 26, 2024

Conversation

stephanie-eng
Copy link
Contributor

@stephanie-eng stephanie-eng commented Jan 18, 2024

Description

This PR changes Servo to always try to use SCHED_FIFO, regardless of presence of a RT kernel. SCHED_FIFO can be used on a non-realtime kernel, and using it will boost the priority above any SCHED_OTHER tasks. While the non-real-time kernel isn't optimized for low jitter, setting SCHED_FIFO will still help to minimize scheduling latency.

To use SCHED_FIFO, it is still required for users have the proper permissions to allow setting of real-time priorities.

To show the difference this change makes on a non-RT kernel, I ran some tests on my laptop:
These are histograms of scheduling latencies I measured running Servo at 100 Hz (10 ms deadline) for 100 seconds (for a total of 10 000 scheduling latency measurements in the histogram)

Wakeup Latency Using SCHED_OTHER (non-RT kernel)

Wakeup Latency Using SCHED_FIFO (non-RT kernel)

In the top histogram, you can see the max wakeup latency in a short 100 second test was about 8.5 ms, which is almost a whole Servo period! A longer duration test may reveal complete deadline misses. However, when using SCHED_FIFO, the scheduling latency is much better controlled.

Copy link

codecov bot commented Jan 18, 2024

Codecov Report

Attention: 1 lines in your changes are missing coverage. Please review.

Comparison is base (4e58ff1) 51.16% compared to head (a5446b7) 51.16%.
Report is 1 commits behind head on main.

Files Patch % Lines
moveit_ros/moveit_servo/src/servo_node.cpp 75.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2653      +/-   ##
==========================================
+ Coverage   51.16%   51.16%   +0.01%     
==========================================
  Files         393      393              
  Lines       32753    32753              
==========================================
+ Hits        16754    16756       +2     
+ Misses      15999    15997       -2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Co-authored-by: AndyZe <andyz@utexas.edu>
Copy link
Contributor

@sjahr sjahr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like this change but how is the scheduler changed back for a user after they ran a servo node? As a user I wouldn't expect a ROS node to automatically mess with my system settings. Is this call reverted by a reboot or at shutdown of the node?

Co-authored-by: Sebastian Castro <4603398+sea-bass@users.noreply.github.com>
@stephanie-eng
Copy link
Contributor Author

stephanie-eng commented Jan 19, 2024

I like this change but how is the scheduler changed back for a user after they ran a servo node? As a user I wouldn't expect a ROS node to automatically mess with my system settings. Is this call reverted by a reboot or at shutdown of the node?

This isn't really what I'd call a system setting. The scheduler is not set for the user, but on a process. This is not a "user setting" that makes all further processes spawned by your user use a specific scheduler - that's not what we're doing here, and on top of that, I'm not sure that's even possible.

A user can have multiple processes running with different schedulers simultaneously. In fact, you probably have some processes running with the FIFO scheduler on your computer now:

ps -e -o pid,class,rtprio,comm

I see a line like this (amongst many other processes, I had to scroll)

1230 FF 50 irq/200-AudioDS

The first column is PID, second is scheduler (FF = FIFO), third is priority (50 in my case), and last is the command. It looks like this is related to audio, which makes sense as audio has real-time has real-time requirements or else we'd risk hearing cracks and pops in our audio.

This change sets the scheduler for the servo_loop_ thread. When this thread is joined, this change is "undone" insofar as there will be no more FIFO-scheduled servo thread (or any servo thread at all), so there are no persistent effects on the user.

TL;DR: the scheduler is set on the process level, not the user level, so the ROS node isn't messing with your system settings, it is messing with its own process settings. There's nothing to "revert" by rebooting.

Copy link
Contributor

@sjahr sjahr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome explanation, thank you! I did not know that

@sjahr sjahr enabled auto-merge (squash) January 26, 2024 07:43
@sjahr sjahr merged commit d56a8b9 into moveit:main Jan 26, 2024
10 of 11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants