language: Go 2: allow setting slice cap < len, prohibiting writes

[dans-stuff]

I'm not sure why there is no way to create a slice's from another, with a capacity lower than it's length.

It has a few uses. For example, I could pass in a copy of my slice set to 0 capacity to benefit from copy-on-write semantics. The length would still appear to have N items, but trying to write to any of them would trigger the slice to re-grow. This would allow for immutable, lightweight slices in general.

I'm not certain if it would break any code, I've seen almost noone calling "cap" in my experience.

[rogpeppe]

You can do this already. See "Full slice expressions" in https://golang.org/ref/spec#Slice_expressions.

[dans-stuff]

@rogpeppe a bit quick on the trigger there!

"The indices are in range if 0 <= low <= high <= max <= cap(a)"

I'm discussing relaxing the last constraint, and allowing cap to be smaller than the max/length. Then normal semantics kick in, where the slice will grow if you append outside it's length. However, the only new behavior is that writing to an index that is below the cap is meaningless and won't actually do anything, or could even panic. If a function tried to write to index 0, when my cap was 0, it should panic.

It's a method of creating simple, effective readonly slices.

Here's an example where a function will, once in a blue moon, edit one of the underlying array items. I don't want to have to pass in a copy, because it would most likely be wasting memory/cpu to make a copy each time. However, I do want to ensure my slice is readonly. Right now I have no choices. However, it intuitively makes sense that if I create a new slice with 0 capacity, then no one can shove anything new into my slice. Anything people try to do to my slice would be over it's capacity, and so internally would need to grow. The length would still be 10, so they can still read all my items, just can't write to them.

https://play.golang.org/p/xf8F6_-4usD

[cespare]

It doesn't seem intuitive to me to allow the slice's capacity to be smaller than its length. It especially doesn't seem intuitive that s[i] could panic when i < len(s).

There are other discussions around immutability that seem more promising to me. For example, #22876 and #20443 seem relevant, and there are other proposals and mailing list discussions.

[dans-stuff]

Why not, @cespare ?

All the other proposals I've seen include adding new keywords and changing types entirely.

Also, I believe it makes perfect sense for the capacity to be 0, how else would you describe an item that cannot hold anything? Slices with zero capacity but a positive length are simply saying "I have something, but I can't hold anything", which is precisely the definition of readonly.

To me it's a natural extension.

[randall77]

I agree with @cespare , cap < len is weird.
Bounds checks are now different for a[i] = x and x = a[i] (the former also has an i < cap(a) check).

What does slicing even mean now?
x = a[:i] used to always succeed if i<cap(a), and sets len(a)=i. Now what happens? Can this still set the length if i>cap(a) but i < len(a)?

Plus all the readonly problems, like ranging over a slice of large objects without copying them:

for i := 0; i < len(a); i++ {
    p := &a[i]  //oops! if cap(a)<len(a)
    ... do something with p ...
}

Or needing multiple versions of library functions like bytes.Index, one to search only up to cap(a) because the caller wants to do a modification, and one which searches up to len(a) because the caller is only reading.

[dans-stuff]

@randall77

I appreciate your feedback, but I'm not sure you understand the proposal.

Slicing would be the same as it is now - the cap would work as it does now, where the new slice will have zero capacity and len(a)=1, as you put it.

Your loop is a good example, as getting a reference to a specific element is a unique bit of syntactic sugar that would have to be modified to check for cap, not just len. It is the compilers job, in that situation, to determine if your code can safely use a reference. Alternatively, try take a reference, and when the caller passes you a cap<len slice it will panic, and the caller will now know it needs write-access to your slice. Then you might get curious why it wants write-access, and investigate, in case you misunderstood what the function was going to do.

I'm not sure where the multiple versions comment comes from. bytes.Index would remain unchanged, like most functions that only use "len". The only functions that would "change" are functions that edit a slice in-place, and even then, the change would be to panic only if passed a cap<len slice. It's 100% backwards compatible.

[randall77]

Your loop is a good example, as getting a reference to a specific element is a unique bit of syntactic sugar that would have to be modified to check for cap, not just len. It is the compilers job, in that situation, to determine if your code can safely use a reference. Alternatively, try take a reference, and when the caller passes you a cap<len slice it will panic, and the caller will now know it needs write-access to your slice. Then you might get curious why it wants write-access, and investigate, in case you misunderstood what the function was going to do.

But what if the ... do something with p... is only reading from p? Somehow you would now want two different &a[i] operations, one of which gives you back a writeable pointer (and panics if i>=len(a) || i >= cap(a)) and one of which gives you back a read-only pointer (and panics if i>=len(a)).
It would be unfortunate if you had to give write permission to a function which only reads a slice, solely to allow it to construct pointers to the elements. As a result, I don't think the cap<len solution can work in isolation; a more holistic read-only data structure solution is required.

For library functions, I think you need multiple versions because the "length" of the operation now depends on what you're going to do with the result. For example:

    a[bytes.IndexByte('a')] = 'b' // replace an 'a' with a 'b' 
    a[bytes.IndexByte('a') + 1] // return the character after 'a'

So does IndexByte stop searching at len(a), or min(len(a),cap(a))?
(This is maybe not the best example, I will try and think of a better one.)

[dans-stuff]

@randall77 I think maybe what's not clear is that this does not change the default behavior of anything. There is no "give write permissions", there is only removing write permissions.

If a function wants a reference to a specific index, Go would simply add i<cap to the bounds check. Only if I explicitly pass a cap<len slice would it be any different than it usually is. I would only pass in a cap<len slice because, now or in the future, I'm not anticipating this function changing my slice elements in any way. There's no read/writeable pointer differentiation, it simply would not allow you to create a reference to an element outside of a slice's cap, because that value does not exist. In can not be referenced, nor dereferenced, only read. This makes complete sense when you remember that you're looking at an element the slice does not have the capacity to be holding.

Also, for all those situations (and all existing code), you would use len as normal. Using slices never changes in any scenario. The only real change is that you can use a cap<len for slice creation, and if you choose to do so, some functions may reveal themselves as not-just-reading from your slices.

[ianlancetaylor]

@dantoye What do you mean when you say "trying to write to any of them would trigger the slice to re-grow"?

[dans-stuff]

@ianlancetaylor By "write to", I did mean append to. Apologies.

Writing to an array would be changed in one way: perform a cap check first, then a len check. In most situations it would have equal performance, but would have an extra check if writing to an index between len and cap. If the cap check fails, but the len check passes, a new panic is raised: "runtime error: index not within slice capacity".

For appending, however, the standard mechanics apply, though I'm sure some changes must be made if the append logic assumes the len<cap invariant.

[ianlancetaylor]

It seems to me to be fairly unfortunate to have to do an extra memory load and comparison for every write to a slice, but I agree that this seems to work. I'm going to turn into a language change proposal. Fair warning, though: I do not think it will be accepted.