Skip to content

Commit

Permalink
bcm53xx: enable & setup packet steering
Browse files Browse the repository at this point in the history
Packet steering can improve NAT masquarade performance on Northstar by
40-50%. It makes reaching 940-942 Mb/s possible on BCM4708 (and
obviously BCM47094 too). Add scripts setting up the most optimal
Northstar setup.

Below are testing results for running iperf TCP traffic from LAN to WAN.
They were used to pick up golden values.

┌──────────┬──────────┬────────────────────┬────────────────────┐
│   eth0   │  br-lan  │ flow_offloading=0  │ flow_offloading=1  │
│          │          ├─────────┬──────────┼─────────┬──────────┤
│ rps_cpus │ rps_cpus │ BCM4708 │ BCM47094 │ BCM4708 │ BCM47094 │
├──────────┼──────────┼─────────┼──────────┼─────────┼──────────┤
│        0 │        0 │     387 │      671 │     707 │      941 │
│        0 │        1 │     343 │      576 │     705 │      941 │
│        0 │        2 │   ✓ 574 │    ✓ 941 │     704 │      940 │
│        1 │        0 │     320 │      549 │     561 │      941 │
│        1 │        1 │     327 │      551 │     553 │      941 │
│        1 │        2 │     523 │    ✓ 940 │     559 │      940 │
│        2 │        0 │     383 │      652 │   ✓ 940 │      941 │
│        2 │        1 │     448 │      754 │   ✓ 942 │      941 │
│        2 │        2 │     404 │      655 │   ✓ 941 │      941 │
└──────────┴──────────┴─────────┴──────────┴─────────┴──────────┘

Above tests were performed with all eth0 interrupts handled by CPU0.
Setting "echo 2 > /proc/irq/38/smp_affinity" was tested on BCM4708 but
it didn't increased speeds (just required different steering):

┌──────────┬──────────┬───────────┐
│   eth0   │  br-lan  │ flow_offl │
│   rx-0   │   rx-0   │ oading=0  │
│ rps_cpus │ rps_cpus │  BCM4708  │
├──────────┼──────────┼───────────┤
│        0 │        0 │       384 │
│        0 │        1 │     ✓ 574 │
│        0 │        2 │       348 │
│        1 │        0 │       383 │
│        1 │        1 │       412 │
│        1 │        2 │       448 │
│        2 │        0 │       321 │
│        2 │        1 │       520 │
│        2 │        2 │       327 │
└──────────┴──────────┴───────────┘

Signed-off-by: Rafał Miłecki <rafal@milecki.pl>
  • Loading branch information
Rafał Miłecki committed Jun 24, 2022
1 parent 82d0dd8 commit fcbd396
Show file tree
Hide file tree
Showing 2 changed files with 47 additions and 0 deletions.
44 changes: 44 additions & 0 deletions target/linux/bcm53xx/base-files/etc/init.d/fastnetwork
@@ -0,0 +1,44 @@
#!/bin/sh /etc/rc.common

START=25
USE_PROCD=1

start_service() {
reload_service
}

service_triggers() {
procd_add_reload_trigger "network"
procd_add_reload_trigger "firewall"
procd_add_reload_interface_trigger "lan"
}

reload_service() {
local packet_steering="$(uci -q get network.@globals[0].packet_steering)"
local num_cpus="$(grep -c "^processor.*:" /proc/cpuinfo)"
local flow_offloading="$(uci -q get firewall.@defaults[0].flow_offloading)"
local flow_offloading_hw="$(uci -q get firewall.@defaults[0].flow_offloading_hw)"

# Any steering on 1 CPU (BCM47081) worsens network performance
[ "$num_cpus" != 2 ] && return

[ "$packet_steering" != 1 ] && {
echo 0 > /sys/class/net/br-lan/queues/rx-0/rps_cpus
echo 0 > /sys/class/net/eth0/queues/rx-0/rps_cpus
return
}

if [ ${flow_offloading_hw:-0} -gt 0 ]; then
# HW offloading
echo 0 > /sys/class/net/br-lan/queues/rx-0/rps_cpus
echo 0 > /sys/class/net/eth0/queues/rx-0/rps_cpus
elif [ ${flow_offloading:-0} -gt 0 ]; then
# SW offloading
# br-lan setup doesn't seem to matter for offloading case
echo 2 > /sys/class/net/eth0/queues/rx-0/rps_cpus
else
# Default
echo 2 > /sys/class/net/br-lan/queues/rx-0/rps_cpus
echo 0 > /sys/class/net/eth0/queues/rx-0/rps_cpus
fi
}
@@ -0,0 +1,3 @@
# SPDX-License-Identifier: GPL-2.0-only

uci set network.@globals[0].packet_steering="1"

0 comments on commit fcbd396

Please sign in to comment.