Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

proposal: crypto/tls: support kernel-provided TLS #44506

Open
howardjohn opened this issue Feb 22, 2021 · 11 comments
Open

proposal: crypto/tls: support kernel-provided TLS #44506

howardjohn opened this issue Feb 22, 2021 · 11 comments
Labels
Proposal Proposal-Crypto Proposal related to crypto packages or other security issues
Milestone

Comments

@howardjohn
Copy link

howardjohn commented Feb 22, 2021

Lots of background and a implementation, albeit from 3+ years ago: https://blog.filippo.io/playing-with-kernel-tls-in-linux-4-13-and-go/

Basically, Linux now supports handling TLS encryption in the kernel. The primary benefit here is the possibility of sendfile/splice to work with TLS. Currently, we need to choose between TLS and splice (or a custom TLS implementation, I suppose).

It would be great to have first class support in go for this.

@seankhliao seankhliao changed the title crypto/tls: support Kernal TLS proposal: crypto/tls: support Kernal TLS Feb 22, 2021
@seankhliao seankhliao added Proposal Proposal-Crypto Proposal related to crypto packages or other security issues labels Feb 22, 2021
@gopherbot gopherbot added this to the Proposal milestone Feb 22, 2021
@seankhliao
Copy link
Member

seankhliao commented Feb 22, 2021

cc @FiloSottile

@ianlancetaylor ianlancetaylor added this to Incoming in Proposals (old) Feb 24, 2021
@rsc rsc changed the title proposal: crypto/tls: support Kernal TLS proposal: crypto/tls: support kernel-provided TLS Feb 24, 2021
@ShivanshVij
Copy link

ShivanshVij commented Jun 26, 2021

I would love to have this happen as well! It's a major use case for L7 load balancers written in golang, and could transparently provide significant performance boosts for a lot of systems (including Kubernetes)

@FiloSottile
Copy link
Contributor

FiloSottile commented Jun 26, 2021

Can we get some benchmarks and numbers for the performance improvement? My patch linked above might be a good starting point. It's a lot of complexity and it would have to be justified by very good numbers.

@jim3ma
Copy link

jim3ma commented Nov 29, 2021

Hi, all

I have updated kernel tls support based on @FiloSottile's original code. It now supports more ciphers like AES_GCM_256, AES_CCM_128 and CHACHA20_POLY1305.

Code: https://github.com/jim3ma/go/tree/dev.ktls.1.16.3.

And I have fixed some kernel issues when in coding: torvalds/linux@974271e, torvalds/linux@d8654f4

In my simple tests, when enable kernel tls, I have got 30% time cost decreased.

@totallyunknown
Copy link

totallyunknown commented Feb 14, 2022

I made some real-world tests with one of our internal applications (CDN node specialised in delivering video segments for DASH and HLS streams).

  • Kernel 5.13.12
  • Curve: prime256v1

I compared https vs http, vs http + sendfile and ktls + sendfile.

Most of the TLS stuff is working, except TLS 1.3 with Chrome and k6. k6 reports tls: oversized record received with length 62464.

With ktls, the latency is increased - but this can also be related to the difference in the used Go-Versions.

The ktls implementation reduces overall CPU usage, around 10%. We'll deploy the Nvidia ConnectX-6 (200 Gbit/s) in our latest hardware setup, and we hope we can use the TLS NIC offloading in the future.

https://docs.google.com/spreadsheets/d/1XaiFczae9GLixu__8y2kuKPsw7RGqW9vMDkYxuTLx28/edit#gid=0

@jrfastab
Copy link

jrfastab commented Feb 14, 2022

@totallyunknown If the latency issue is related to the kernel implementation (rule out golang side) we can take a look at kernel side improvements. We've been using the openssl implementation lately so I'll check there as well, but I don't recall extra latency last time I did metrics. Having a golang implementation would be very useful on my side as well. fwiw I'm one of the ktls maintainers on kernel side so we shouldn't have trouble getting improvements there as needed and happy to help where I can to get this moving forward.

@jim3ma
Copy link

jim3ma commented Feb 15, 2022

I made some real-world tests with one of our internal applications (CDN node specialised in delivering video segments for DASH and HLS streams).

  • Kernel 5.13.12
  • Curve: prime256v1

I compared https vs http, vs http + sendfile and ktls + sendfile.

Most of the TLS stuff is working, except TLS 1.3 with Chrome and k6. k6 reports tls: oversized record received with length 62464.

With ktls, the latency is increased - but this can also be related to the difference in the used Go-Versions.

The ktls implementation reduces overall CPU usage, around 10%. We'll deploy the Nvidia ConnectX-6 (200 Gbit/s) in our latest hardware setup, and we hope we can use the TLS NIC offloading in the future.

https://docs.google.com/spreadsheets/d/1XaiFczae9GLixu__8y2kuKPsw7RGqW9vMDkYxuTLx28/edit#gid=0

Which version do you test ? I have update some go code for http with ktls.

@totallyunknown
Copy link

totallyunknown commented Feb 15, 2022

@jim3ma
Copy link

jim3ma commented Feb 15, 2022

@jim3ma Your branch: https://github.com/jim3ma/go/tree/dev.ktls.1.16.3

Okay, I will merge some optimized code into this branch tomorrow.

@kkkygytb

This comment was marked as duplicate.

@ultperf

This comment was marked as duplicate.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Proposal Proposal-Crypto Proposal related to crypto packages or other security issues
Projects
Status: Incoming
Development

No branches or pull requests

10 participants