Realtime Debian Configuration

Realtime Capable Debian

Tested on a fresh installation of Debian 13.5.0 (Kernel 6.12.94). With an intel i210 Ethernet card and an intel processor.

For Ethercat we used these terminals: EK1100, EL1002,EL3204,EL1008,EL4008,EL2004

Setup

Install the PREEMPT_RT kernel patches, development tools, and Rust:

# Assumes you added your user to the sudo group
sudo apt update && sudo apt upgrade -y
# Installs the realtime linux patch
sudo apt install linux-image-rt-amd64 linux-headers-rt-amd64 rt-tests curl ca-certificates build-essential git ethtool -y

# If you dont like the | sh piping
# instead do: sudo apt install cargo rustc -y
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y
. "$HOME/.cargo/env"

Open /etc/default/grub as root and replace GRUB_CMDLINE_LINUX_DEFAULT="quiet" with the following optimization parameters:

GRUB_CMDLINE_LINUX_DEFAULT="quiet preempt=full isolcpus=2,3 nohz_full=2,3 rcu_nocbs=2,3 irq_affinity=0 intel_idle.max_cstate=0 processor.max_cstate=0"

Apply the changes and reboot:

sudo update-grub
sudo update-initramfs -u
sudo reboot

Note: On reboot, select Advanced options for Debian... in the GRUB menu and ensure you boot into the kernel ending in -rt-amd64.

Verify your parameters are applied successfully with cat /proc/cmdline:

BOOT_IMAGE=/boot/vmlinuz-6.12.94+deb13-rt-amd64 root=... ro quiet preempt=full isolcpus=2,3 nohz_full=2,3 rcu_nocbs=2,3 irq_affinity=0 intel_idle.max_cstate=0 processor.max_cstate=0

Run cyclictest on isolated core 3 with maximum priority to verify system determinism

sudo cyclictest --priority=99 --interval=1000 --threads=1 -a 3 --loops=100000

The output will show timings in microseconds where max is the maximum deviation from the expected interval. Expected Result:

# /dev/cpu_dma_latency set to 0us
policy: fifo: loadavg: 0.11 0.30 0.17 1/616 2779          

T: 0 ( 2769) P:99 I:1000 C:  68756 Min:      1 Act:    2 Avg:    1 Max:       8

A Max value of around 10 microseconds indicates excellent real-time performance.

OS Level Optimizations for ethercat

Prevent NetworkManager from polluting your dedicated EtherCAT interface (replace enp4s0 with your target interface):

# Permanently unmanage enp4s0 (persists across reboots)
sudo tee -a /etc/NetworkManager/NetworkManager.conf << 'EOF' > /dev/null
	[keyfile]
	unmanaged-devices=interface-name:enp4s0
EOF
# Restart NetworkManager to apply changes, This will temporarily cut off any established connections
sudo systemctl restart NetworkManager.service 

Unmanaging stops Networkmanager from sending router solicitations and other things. Which lowers noise in our ethercat communication. Now there are a few optimizations we can do to remove any remaining noise from our isolated cpu cores.

The following script routes all hardware interrupts to Core 0, locks the isolated cores to maximum frequency, and disables network driver offloading to minimize jitter. The optimizations.sh script needs to be run after every reboot.

Run this command to create optimizations.sh:

cat << 'EOF' >> optimizations.sh
#!/bin/bash
#example: sudo ./rt-setup.sh 2,3 enp4s0
#where 2,3 are the isolated cpus and enp4s0 is the network interface
echo "setting all interrupt handlers to run on cpu 0"
for irq in /proc/irq/[0-9]*; do sudo sh -c "echo 1 > $irq/smp_affinity" 2>/dev/null; done

input_string=$1
# Replace all commas with spaces, then wrap in ()
cpu_list=( ${input_string//,/ } )

# Print the array to verify
declare -p cpu_list
for item in "${cpu_list[@]}"; do
    echo "setting frequencies to max for cpu $item"
    cpu_max_freq=$(cat /sys/devices/system/cpu/cpu$item/cpufreq/scaling_max_freq)
    cpu_scaling_min_path=/sys/devices/system/cpu/cpu$item/cpufreq/scaling_min_freq
    echo "$cpu_max_freq" | sudo tee /sys/devices/system/cpu/cpu$item/cpufreq/scaling_min_freq
done

echo "setting timeouts to 0, disabling offloads for eth interface: $2"
sudo ethtool -C $2 rx-usecs 0 tx-usecs 0
sudo ethtool -K $2 gso off gro off tso off sg off rxvlan off txvlan off
sudo ethtool -G $2 rx 128 tx 128
sudo ip link set $2 up
EOF
chmod +x optimizations.sh

Hardware Note: The ethtool configurations above are optimized for Intel i210 PCI cards utilizing the standard igb driver. Mileage may vary on other hardware architectures.

Important!: keep in mind to provide adequate cooling, as the optimizations.sh script sets the maximum possible frequency for the given CPU cores.

You can execute it like this:

# 2,3 are our isolated cores, change if different for you
# enp4s0 is the interface to be configured change to fit yours
./optimizations.sh 2,3 enp4s0

Run cyclictest again after executing optimizations.sh on isolated core 3 with maximum priority to verify system determinism

sudo cyclictest --priority=99 --interval=1000 --threads=1 -a 3 --loops=100000

Potential latency spikes exceeding 20 microseconds after these optimizations are often caused by System Management Interrupts (SMIs) generated at the BIOS level. Because SMI behavior depends heavily on specific hardware configurations, run hwlatdetect (included in rt-tests) to isolate and rule out BIOS-induced latency issues. No SMI anomalies were observed on the reference hardware used for this setup.

Downloading qitech_lib

To test how well the system can keep up with a given cycle time you can run our rt.rs example from qitech_lib

git clone github.com/qitechgmbh/qitech_lib.git
cd qitech_lib
# Currently only stable on a testing branch, changing very soon
git switch testing-futures
cargo build --release --example rt
# specify enp4s0 as the interface, 250 microseconds as cycle time, and 100000 cycles as test duration
# This currently ONLY tests how well the Ethercat logic can keep the cycle time
sudo ./target/release/examples/rt enp4s0 250 100000

The end Output should look like this:

================ BENCHMARK RESULTS ================
Target Cycle Time:    250 µs
Total Cycles Run:     1000000
---------------------------------------------------
Cycle Time Metrics:
  Min:                246 µs
  Avg:                249.43 µs
  Max:                252 µs
  Std Dev:            0.51 µs
  99th Percentile:    250 µs
---------------------------------------------------
Jitter Metrics (Deviation from Target):
  99th Pct Jitter:    1 µs
  Max Jitter:         4 µs
===================================================

Note that Running a higher number of cycles increases the sample size, yielding a more accurate and statistically significant baseline for maximum deviation.