-
Notifications
You must be signed in to change notification settings - Fork 41.7k
Description
What would you like to be added?
As of go 1.22, for string to bytes conversion, we can replace the usage of unsafe.Slice(unsafe.StringData(s), len(s)) with type casting []bytes(str), without the worry of losing performance.
As of go 1.22, string to bytes conversion []bytes(str) is faster than using the unsafe package. Both methods have 0 memory allocation now.
I saw at least two places in the codebase still using the unsafe way:
-
kubernetes/staging/src/k8s.io/apiserver/pkg/authentication/token/cache/cached_token_authenticator.go
Lines 277 to 286 in e342ab0
// toBytes performs unholy acts to avoid allocations func toBytes(s string) []byte { // unsafe.StringData is unspecified for the empty string, so we provide a strict interpretation if len(s) == 0 { return nil } // Copied from go 1.20.1 os.File.WriteString // https://github.com/golang/go/blob/202a1a57064127c3f19d96df57b9f9586145e21c/src/os/file.go#L246 return unsafe.Slice(unsafe.StringData(s), len(s)) } -
kubernetes/staging/src/k8s.io/apiserver/pkg/storage/value/encrypt/envelope/kmsv2/envelope.go
Lines 511 to 520 in e342ab0
// toBytes performs unholy acts to avoid allocations func toBytes(s string) []byte { // unsafe.StringData is unspecified for the empty string, so we provide a strict interpretation if len(s) == 0 { return nil } // Copied from go 1.20.1 os.File.WriteString // https://github.com/golang/go/blob/202a1a57064127c3f19d96df57b9f9586145e21c/src/os/file.go#L246 return unsafe.Slice(unsafe.StringData(s), len(s)) }
Here's my benchmark results, comparing two ways of conversion:
╰─○ go test -v -run=none -bench=. -benchmem=true
goos: darwin
goarch: amd64
pkg: example.com/m/v2
cpu: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
BenchmarkStringToBytesUnsafe
BenchmarkStringToBytesUnsafe-12 1000000000 0.5636 ns/op 0 B/op 0 allocs/op
BenchmarkStringToBytesCasting
BenchmarkStringToBytesCasting-12 1000000000 0.2548 ns/op 0 B/op 0 allocs/op
PASS
ok example.com/m/v2 1.448sMy test code:
package main
import (
"testing"
"unsafe"
)
const (
str = "some random string"
)
func toBytes(s string) []byte {
// unsafe.StringData is unspecified for the empty string, so we provide a strict interpretation
if len(s) == 0 {
return nil
}
// Copied from go 1.20.1 os.File.WriteString
// https://github.com/golang/go/blob/202a1a57064127c3f19d96df57b9f9586145e21c/src/os/file.go#L246
return unsafe.Slice(unsafe.StringData(s), len(s))
}
func toBytesRaw(s string) []byte {
return []byte(s)
}
func BenchmarkStringToBytesUnsafe(b *testing.B) {
for i := 0; i < b.N; i++ {
_ = toBytes(str)
}
}
func BenchmarkStringToBytesCasting(b *testing.B) {
for i := 0; i < b.N; i++ {
_ = toBytesRaw(str)
}
}
Why is this needed?
- Kubernetes as of now is built with go 1.22, in go 1.22 type casting is faster than unsafe, regarding only string to bytes conversion
- We can make string to bytes conversion faster with type casting
- We don't need to be 'unsafe' anymore