Kernel TLS support and native sendfile for TLS connections

[tarasko]

Hi guys,

Perhaps you would be interested in adding Kernel TLS support and native TLS sendfile to aiohttp through my little library aiofastnet?
This will let aiohttp to serve FileResponses over TLS connections much more efficiently.

TLS in aiofastnet is also just faster than in asyncio and uvloop, because it uses openssl directly and remove a lot of python plumbing.

I did a branch showing how aiofastnet can be possibly used from aiohttp.
tarasko#1

I also added example that will let you to try kTLS and compare it against fallback FileResponse, with or without uvloop.
https://github.com/tarasko/aiohttp/blob/feature/aiofastnet/examples/ktls_static_file.py

On my laptop sending 2GB file over loopback interface is roughly 40% faster with kTLS enabled.

kTLS is available on Linux (and allegedly on FreeBSD, but I haven't tried)

To try it out:

• Load tls kernel module:

sudo modprobe tls

• Check that you have more or less recent Linux kernel >=5.16.

• Check that your Python _ssl module is loading system libssl and libcrypto. I think this is normally the case unless you use conda which install its own libssl and libcrypto into python environment. They are build on CI server with an old kernel. You can just locate them and soft-link to the system libssl and libcrypto

• Check that system openssl is >=3.0.x. The newer the better.

• Make a new environment, clone my copy of aiohttp and check out feature/aiofastnet branch.

git clone https://github.com/tarasko/aiohttp.git 
cd aiohttp
git checkout feature/aiofastnet
pip install -e . 

Run example with debug log level to verify that kTLS will be enable when client connects:

USE_AIOFN=1 python examples/ktls_static_file.py --level DEBUG

It will create a 2Gb file, which can be downloaded:

kTLS: https://127.0.0.1:8444/huge.bin
non-kTLS: https://127.0.0.1:8443/huge.bin

You can use curl to download and measure speed:

curl -k https://127.0.0.1:8444/huge.bin  --out-null

If kTLS is enabled you will see in the server's log:

DEBUG:aiofastnet.ssl:[fd=9 SSLTransport_Socket server #0 wbuf_size=0]: KTLS SEND enabled: 1
DEBUG:aiofastnet.ssl:[fd=9 SSLTransport_Socket server #0 wbuf_size=0]: KTLS RECV enabled: 1

Let me know if you're interested, but couldn't make it work. OpenSSL is terrible at explaining why it refuses to enable kTLS, it often doesn't give any clue, just silently refuse to do it. I can help with troubleshooting.

You can also experiment with strace to check what syscalls are being called:

USE_AIOFN=1 strace python examples/ktls_static_file.py

After OpenSSL has enabled kTLS, SSL_sendfile just calls regular sendfile and let kernel tls module load and encrypt file content:

epoll_wait(4, [{events=EPOLLOUT, data=0x57c400000009}], 4, -1) = 1
sendfile(9, 10, [2621440] => [4440064], 2144862208) = 1818624
sendfile(9, 10, [4440064] => [4571136], 2143043584) = 131072
sendfile(9, 10, [4571136] => [4767744], 2142912512) = 196608
sendfile(9, 10, [4767744] => [4898816], 2142715904) = 131072
sendfile(9, 10, [4898816] => [5046272], 2142584832) = 147456
sendfile(9, 10, [5046272] => [5242880], 2142437376) = 196608
sendfile(9, 10, [5242880] => [5455872], 2142240768) = 212992
sendfile(9, 10, [5455872] => [5505024], 2142027776) = 49152
sendfile(9, 10, [5505024], 2141978624)  = -1 EAGAIN (Resource temporarily unavailable)
epoll_wait(4, [{events=EPOLLOUT, data=0x57c400000009}], 4, -1) = 1
sendfile(9, 10, [5505024] => [7176192], 2141978624) = 1671168
sendfile(9, 10, [7176192] => [7225344], 2140307456) = 49152

All functional tests of aiohttp pass with aiofastnet, with a couple exceptions, where the test itself is not really functional, but mock some loop/transport methods.
To run the tests in the branch:

USE_AIOFN=1 pytest -s -v -n 0 -k functional

[Dreamsorcerer]

@bdraco is probably best to review. But, if we're happy with it, then it'd need to be an optional speedups dependency and likely used by default when available (not via envvar).

[Dreamsorcerer]

To ensure it doesn't drop off our radar, I'd open the PR here.

[tarasko]

Thank you, I'll make a PR.

Agree, extras are more conventional. I used envvar only as a POC, to enable quick switching between implementations.

[tarasko]

Actually, I have just added monkey-patching to aiofastnet. With it nothing has to be changed in aiohttp:

Users would only need to:

aiofastnet.install_policy()

or

asyncio.run(main(), aiofastnet.loop_factory())

And everything magically becomes faster, including native TLS sendfile support. Works nicely with the current aiohttp.
I added 2 examples:

aiohttp_ws_speedup.py
aiohttp_ktls_fileresponse.py

You can try and measure for yourself. On my laptop aiohttp_ws_speedup.py becomes around 50% faster when run with --uvloop --aiofastnet comparing to when run just with --uvloop

And aiohttp_ktls_fileresponse.py becomes 40% faster with kTLS enabled.

Would you mind adding aiofastnet to 3rdparty library list, to spread the word?

[Dreamsorcerer]

I've checked with bdraco, and this seems like something we'd want.

So, we'd rather you removed the monkey-patching, and added it to the speedups extra in aiohttp so people will get it automatically. It will then need to work the same way as the other speedups and be used by default if it is installed and importable, falling back to the current implementation otherwise.