Skip to content

Add support for recvmmsg and UDP GRO.#9904

Closed
brbzull0 wants to merge 3 commits intoapache:masterfrom
brbzull0:udpgro
Closed

Add support for recvmmsg and UDP GRO.#9904
brbzull0 wants to merge 3 commits intoapache:masterfrom
brbzull0:udpgro

Conversation

@brbzull0
Copy link
Contributor

@brbzull0 brbzull0 commented Jun 23, 2023

WIP
I will leave this as draft as I still need to :

  • Make another look to the code and improve some stuffs.
  • Ammend some stuffs and run some more tests and update the description, I still need to tweak some stuffs but the idea would be this.
  • Make sure I have all CI green in the meantime.
    WIP

Descrition

This PR adds support for UDP GRO when reading the socket using recvm(m)gsg. This PR also includes support for recvmmsg when it is available by the OS.

There is now a new configuration variable:

# records.yaml
ts:
  udp:
    enable_gro: 1 # Enabled by default.

Notes

All the tests were done using GET only and setting the h2load parameter to --max-udp-payload-size=2k.

Feature Total # Req # Clients # recvmmsg # recvmsg
recvmmsg+gro 100k 100 22834 0
recvmmsg 100k 100 23501 0
recvmsg+gro 100k 100 247970
recvmmsg 100k 100 249097

As I didn't see much of a difference between having GRO enabled or disabled I did run a test with debug
enabled and I was able to count how many packets came with the GRO set in the ancillary data.
Then I realized why I wasn't actually seeing much difference, not too many packets were spliced by the kernel.

Packets with (No GRO/GRO) = Number of times that ATS detected a packet with spliced with GRO.

Feature Total # Req # Clients # recvmmsg # recvmsg # Packets with (No GRO/GRO) GRO %
recvmmsg+gro 100k 100 13929 0 223080 / 180 0.8%
recvmsg+gro 100k 100 0 278718 231760 / 34484 14%

Using recvmmsg reduced the number of syscall significantly

This is what I used for this tests:

bpftrace

bpftrace -e 'tracepoint:syscalls:sys_enter_recvm* { @[comm, probe] = count();  }' 

h2load

/opt/bin/h2load -n 100000 -c 100 --npn-list=h3 URL --max-udp-payload-size=2k

I was using only GET requests with around 15 header with about 80 bytes each.
I have not tried POST yet which I believe will give us better insights.

@brbzull0 brbzull0 self-assigned this Jun 23, 2023
@brbzull0 brbzull0 changed the title Udpgro UDP GRO spport Jun 23, 2023
@brbzull0 brbzull0 closed this Jun 23, 2023
@brbzull0 brbzull0 changed the title UDP GRO spport Add support for recvmmsg and UDP GRO. Jun 23, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant