-
Notifications
You must be signed in to change notification settings - Fork 17.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
spec: clarify that conversions to slices don't guarantee slice capacity? #24163
Comments
Interesting, thanks! Here we can see that the cap is 32: But uncomment the following line and the cap changes to 0: Definitely something messed up in the compiler. /cc @randall77 @mdempsky |
Actually, does the language guarantee the size of the cap on string->[]rune conversions? https://golang.org/ref/spec#Conversions_to_and_from_a_string_type says only:
But doesn't say anything about the slice's capacity. /cc @griesemer |
Here's another example using |
@mattbostock Thanks for the examples. I think this example summarizes/demonstrates the root issue you're noticing: https://play.golang.org/p/Er5s4t9rO7R (Also, note that the upper bound for slice operations is Like @bradfitz points out, I think the real question is whether string->[]byte or string->[]rune conversions guarantee a particular slice capacity. It seems the spec doesn't guarantee that. |
So seems everything is working fine, but the language spec could possibly be more specific here. Moving to NeedsDecision to see whether the language gets changed. /cc @ianlancetaylor too. |
I think that for these conversions the result should be a slice with cap == len. I see no benefit to leaving it unspecified. |
I think there's a small performance benefit. Suppose something like:
For the conversion, the runtime will need to allocate memory. Also, it will pad the allocation up to the next allocation bucket size. If we expose this capacity to the user via the resulting slice's capacity, then the append might be able to fit into the existing space without requiring a second allocation. |
As bad as it sounds, there may be existing code that relies on this feature... Perhaps the spec just needs to say that it doesn't say anything about the capacity in these cases. |
Considering that escape analysis changes somewhat regularly, I kinda doubt anybody's relying on this. We've never heard anything about it. |
For conversions from
Prints:
So escape analysis doesn't really figure into this discussion, it just affects the constants. IMO when Go does an allocation and the user didn't specify a capacity, the runtime should be free to allocate capacity somewhat beyond the requested size. Calling *Although you could argue that only the 3-arg version of make requests an explicit capacity, the 2-arg version could allow cap>len. |
I think @bradfitz's point was that because escape analysis changes, the constants particular user code sees sometimes changes too. Thus it's arguably safe if we want to intentionally change the constants.
I'd argue even for the 3-arg version, that the explicit cap argument should just be a lower bound on what the runtime returns. But unfortunately these seems out of scope for Go 1. |
@mdempsky Wow, you're even more of a rebel than I am. |
To be clear, the spec does currently appear to forbid this.
There's an analogy with making maps, where the len arg is just a hint. On the other hand, we definitely wouldn't want the len arg to buffered channels to be just a hint, since they are used to strictly control concurrency. Another option is to add a "round up" function to package runtime that accepts a size (obtained presumably by unsafe.Sizeof) and a quantity and returns a recommended quantity. Then in cases in which it really matters, people would have the tools to do it--at the cost of using both package unsafe and package runtime. Combine with generics and you could even build append as a library function. :) (Note that if one wanted to go really nuts and build this yourself without package runtime support, you could, by writing an init function that makes byte slices of various sizes and calls append on them and records the cap you get back, and inferring malloc slab sizes from them.) |
I think the discussions about |
What did you do?
Converted a string to a slice of runes then back to a string, and referenced non-existent elements on the slice of runes:
https://play.golang.org/p/MmJNLu6h6FT
What did you expect to see?
A runtime panic with a 'slice bounds out of range' error.
What did you see instead?
System details
The text was updated successfully, but these errors were encountered: