Skip to content

Commit

Permalink
rps:udev: act upon move action
Browse files Browse the repository at this point in the history
SR-IOV devices get moved (renamed) upon their creation.
This causes the `set-rps-mask.sh` to fail since the queue's path has changed.

We should add additional udev rule to act upon the move of a physical device
and set the rps mask correctly.

The `set-rps-mask.sh` script has modified to act upon those two
different scenarios:
1. when queues are being added.
2. when net device is moved.

In addition we fail silently (return 0) when queues are failed to get
updated due to the mentioned above.
The queues that were failed to get updated, are expected to be updated
by the instance that get trrigered after the device move (renaming).

Signed-off-by: Talor Itzhak <titzhak@redhat.com>
  • Loading branch information
Tal-or committed Dec 14, 2023
1 parent 4f9b5ac commit 202d813
Show file tree
Hide file tree
Showing 3 changed files with 11 additions and 14 deletions.
@@ -1 +1,4 @@
SUBSYSTEM=="queues", ACTION=="add", ENV{DEVPATH}=="/devices/pci*/queues/rx*", TAG+="systemd", PROGRAM="/bin/systemd-escape --path --template=update-rps@.service $env{DEVPATH}", ENV{SYSTEMD_WANTS}="%c"

# SR-IOV devices are moved (renamed), hence we want to catch this event as well
SUBSYSTEM=="net", ACTION=="move", ENV{DEVPATH}!="/devices/virtual/net/*", TAG+="systemd", PROGRAM="/bin/systemd-escape --path --template=update-rps@.service $env{DEVPATH}", ENV{SYSTEMD_WANTS}="%c"
18 changes: 6 additions & 12 deletions assets/performanceprofile/scripts/set-rps-mask.sh
@@ -1,24 +1,19 @@
#!/usr/bin/env bash

function set_queue_rps_mask() {
queue_path=${path%rx*}

queue_num=${path#*queues/}

# replace '/' with '-'
queue_num="${queue_num/\//-}"

# replace x2d with hyphen (-) which is an escaped character
# that was added by systemd-escape in order to escape the systemd unit name that invokes this script
path=${path/x2d/-}
# set rps affinity for the queue
echo "${mask}" 2> /dev/null > "/sys${queue_path}${queue_num}/rps_cpus"

echo "${mask}" 2> /dev/null > "/sys/${path}/rps_cpus"
# the 'echo' command might failed if the device path which the queue belongs to has changes
# this can happen in case of SRI-OV devices renaming
exit 0
return 0
}

function set_net_dev_rps_mask() {
# in case of device we want to iterate through all queues
for i in /sys"${path}"/queues/rx-*; do
for i in /sys/"${path}"/queues/rx-*; do
echo "${mask}" > "${i}/rps_cpus"
done
}
Expand All @@ -34,4 +29,3 @@ function set_net_dev_rps_mask() {
else
set_net_dev_rps_mask
fi

Expand Up @@ -33,7 +33,7 @@ spec:
path: /usr/local/bin/hugepages-allocation.sh
user: {}
- contents:
source: data:text/plain;charset=utf-8;base64,IyEvdXNyL2Jpbi9lbnYgYmFzaAoKcGF0aD0kezF9ClsgLW4gIiR7cGF0aH0iIF0gfHwgeyBlY2hvICJUaGUgZGV2aWNlIHBhdGggYXJndW1lbnQgaXMgbWlzc2luZyIgPiYyIDsgZXhpdCAxOyB9CgptYXNrPSR7Mn0KWyAtbiAiJHttYXNrfSIgXSB8fCB7IGVjaG8gIlRoZSBtYXNrIGFyZ3VtZW50IGlzIG1pc3NpbmciID4mMiA7IGV4aXQgMTsgfQoKcXVldWVfcGF0aD0ke3BhdGglcngqfQoKcXVldWVfbnVtPSR7cGF0aCMqcXVldWVzL30KIyByZXBsYWNlICcvJyB3aXRoICctJwpxdWV1ZV9udW09IiR7cXVldWVfbnVtL1wvLy19IgoKIyBzZXQgcnBzIGFmZmluaXR5IGZvciB0aGUgcXVldWUKZWNobyAiJHttYXNrfSIgPiAiL3N5cyR7cXVldWVfcGF0aH0ke3F1ZXVlX251bX0vcnBzX2NwdXMi
source: data:text/plain;charset=utf-8;base64,IyEvdXNyL2Jpbi9lbnYgYmFzaAoKZnVuY3Rpb24gc2V0X3F1ZXVlX3Jwc19tYXNrKCkgewojIHJlcGxhY2UgeDJkIHdpdGggaHlwaGVuICgtKSB3aGljaCBpcyBhbiBlc2NhcGVkIGNoYXJhY3RlcgojIHRoYXQgd2FzIGFkZGVkIGJ5IHN5c3RlbWQtZXNjYXBlIGluIG9yZGVyIHRvIGVzY2FwZSB0aGUgc3lzdGVtZCB1bml0IG5hbWUgdGhhdCBpbnZva2VzIHRoaXMgc2NyaXB0CnBhdGg9JHtwYXRoL3gyZC8tfQojIHNldCBycHMgYWZmaW5pdHkgZm9yIHRoZSBxdWV1ZQplY2hvICIke21hc2t9IiAgMj4gL2Rldi9udWxsID4gIi9zeXMvJHtwYXRofS9ycHNfY3B1cyIKIyB0aGUgJ2VjaG8nIGNvbW1hbmQgbWlnaHQgZmFpbGVkIGlmIHRoZSBkZXZpY2UgcGF0aCB3aGljaCB0aGUgcXVldWUgYmVsb25ncyB0byBoYXMgY2hhbmdlcwojIHRoaXMgY2FuIGhhcHBlbiBpbiBjYXNlIG9mIFNSSS1PViBkZXZpY2VzIHJlbmFtaW5nCnJldHVybiAwCn0KCmZ1bmN0aW9uIHNldF9uZXRfZGV2X3Jwc19tYXNrKCkgewogICMgaW4gY2FzZSBvZiBkZXZpY2Ugd2Ugd2FudCB0byBpdGVyYXRlIHRocm91Z2ggYWxsIHF1ZXVlcwogICBmb3IgaSBpbiAvc3lzLyIke3BhdGh9Ii9xdWV1ZXMvcngtKjsgZG8KICAgICBlY2hvICIke2l9L3Jwc19jcHVzIgogICAgIGVjaG8gIiR7bWFza30iID4gIiR7aX0vcnBzX2NwdXMiCiAgZG9uZQogfQoKIHBhdGg9JHsxfQogWyAtbiAiJHtwYXRofSIgXSB8fCB7IGVjaG8gIlRoZSBkZXZpY2UgcGF0aCBhcmd1bWVudCBpcyBtaXNzaW5nIiA+JjIgOyBleGl0IDE7IH0KCiBtYXNrPSR7Mn0KIFsgLW4gIiR7bWFza30iIF0gfHwgeyBlY2hvICJUaGUgbWFzayBhcmd1bWVudCBpcyBtaXNzaW5nIiA+JjIgOyBleGl0IDE7IH0KCiBpZiBbWyAiJHtwYXRofSIgPX4gInF1ZXVlcyIgXV07IHRoZW4KICAgc2V0X3F1ZXVlX3Jwc19tYXNrCiBlbHNlCiAgIHNldF9uZXRfZGV2X3Jwc19tYXNrCiBmaQo=
verification: {}
group: {}
mode: 448
Expand Down Expand Up @@ -68,7 +68,7 @@ spec:
path: /etc/sysctl.d/99-default-rps-mask.conf
user: {}
- contents:
source: data:text/plain;charset=utf-8;base64,U1VCU1lTVEVNPT0icXVldWVzIiwgQUNUSU9OPT0iYWRkIiwgRU5We0RFVlBBVEh9PT0iL2RldmljZXMvcGNpKi9xdWV1ZXMvcngqIiwgVEFHKz0ic3lzdGVtZCIsIEVOVntTWVNURU1EX1dBTlRTfT0idXBkYXRlLXJwc0AlcC5zZXJ2aWNlIg==
source: data:text/plain;charset=utf-8;base64,U1VCU1lTVEVNPT0icXVldWVzIiwgQUNUSU9OPT0iYWRkIiwgRU5We0RFVlBBVEh9PT0iL2RldmljZXMvcGNpKi9xdWV1ZXMvcngqIiwgVEFHKz0ic3lzdGVtZCIsIFBST0dSQU09Ii9iaW4vc3lzdGVtZC1lc2NhcGUgLS1wYXRoIC0tdGVtcGxhdGU9dXBkYXRlLXJwc0Auc2VydmljZSAkZW52e0RFVlBBVEh9IiwgRU5We1NZU1RFTURfV0FOVFN9PSIlYyIKCiMgU1ItSU9WIGRldmljZXMgYXJlIG1vdmVkIChyZW5hbWVkKSwgaGVuY2Ugd2Ugd2FudCB0byBjYXRjaCB0aGlzIGV2ZW50IGFzIHdlbGwKU1VCU1lTVEVNPT0ibmV0IiwgQUNUSU9OPT0ibW92ZSIsIEVOVntERVZQQVRIfSE9Ii9kZXZpY2VzL3ZpcnR1YWwvbmV0LyoiLCBUQUcrPSJzeXN0ZW1kIiwgUFJPR1JBTT0iL2Jpbi9zeXN0ZW1kLWVzY2FwZSAtLXBhdGggLS10ZW1wbGF0ZT11cGRhdGUtcnBzQC5zZXJ2aWNlICRlbnZ7REVWUEFUSH0iLCBFTlZ7U1lTVEVNRF9XQU5UU309IiVjIgo=
verification: {}
group: {}
mode: 420
Expand Down

0 comments on commit 202d813

Please sign in to comment.