-
Notifications
You must be signed in to change notification settings - Fork 17.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Feature: provide no-copy conversion from []byte to string #25484
Comments
That's not quite the right way to do it. See the description at the end of package We don't want to provide this functionality (unsafe-converting a If and when we do #19367, adding copy-free |
As @randall77 notes, this would have to live in the |
I understand the concern, but this just seems to be such a common use case with very obvious gains. I understand that Go is focused more on engineering practicality than language correctness. Since we already have an Here's my new way to do it by avoiding an instance of
It's actually subtly tricky to ensure that the byte slice doesn't get GC'ed. I took into account the comment here to do it all in one line. |
I think this is safer.
That way you're never allocating an instance of |
Thanks, but that doesn't compile in go1.10.2. Is it even possible to do
|
Ah, I forgot a final "return". |
With the missing return, it seems to work: FWIW, you can also write it without referring to |
Thanks! Here's my final version with one more little optimization. See, the subtleties of getting this right are exactly why this should be a std func. |
The func BytesToString(bytes []byte) (s string) {
slice := (*reflect.SliceHeader)(unsafe.Pointer(&bytes))
str := (*reflect.StringHeader)(unsafe.Pointer(&s))
str.Data = slice.Data
str.Len = slice.Len
runtime.KeepAlive(&bytes) // this line is essential.
return s
} |
Thanks very much, go101. For a closed issue this one is getting a lot of input. ;) Again, I imagine very, very few people out there are getting this critical code right. See this blog and this blog for wrong ways of doing this. Folk, I think this should be in Go std lib, if only to save people from themselves. Many people will try to do this and mess up. |
@go101 Unless I have missed something, the version without |
If #19367 is accepted, then it will be much easier to get right: that proposal uses The only reason this is tricky today is because the headers are in |
@bcmills func ByteSlice2String(bs []byte) string {
return *(*string)(unsafe.Pointer(&bs))
} is a more efficient way. |
Thanks, go101. For reference, this is the source referred to. |
If you read bytes from an
io.Reader
, but need them as astring
, then you're going to have to convert. And that will cost you memory as data is copied between quite different variables.Annoyed by this waste, some of us has been doing things like this:
Of course, proper use requires you to never change the original byte array, but that's a reasonable contract in many cases.
First, I would like to know if there are any repercussions to this usage. I'm always nervous about using
unsafe
, but in this case it seems to adhere to the spec, and also I can't foresee any problems with garbage collection. But maybe I'm missing something important. I've been using this a lot and have not seen any issues.Second, assuming this code is A-OK, then why not provide something like this in the standard library? (Probably in the
bytes
package.) It could allow for significant memory savings for programs that read a lot of text.The text was updated successfully, but these errors were encountered: