Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

spec: allow conversion from slice to array #46505

Open
bradfitz opened this issue Jun 1, 2021 · 54 comments
Open

spec: allow conversion from slice to array #46505

bradfitz opened this issue Jun 1, 2021 · 54 comments

Comments

@bradfitz
Copy link
Contributor

@bradfitz bradfitz commented Jun 1, 2021

I recently tried to use the new #395 feature (for converting a slice to an array pointer: https://golang.org/cl/216424) in:

https://go-review.googlesource.com/c/go/+/322329

But in review, it was pointed out that it was a little ugly, as what I wanted to return was an array, which required a dereference:

	return *(*[Size224]byte)(sum[:Size224])

It would've been nicer if I could've just converted to an array instead:

	return ([Size224]byte)(sum[:Size224])

Talking to @ianlancetaylor and @griesemer, we couldn't remember great reasons for not also allowing this as part of #395. It does mean there's an subtle copy, but arguably the * dereference above is also a somewhat subtle copy.

Could we also add support for converting to the array?

/cc @katiehockman @josharian @rogpeppe @mdempsky

@gopherbot gopherbot added this to the Proposal milestone Jun 1, 2021
@randall77
Copy link
Contributor

@randall77 randall77 commented Jun 1, 2021

Note the [:Size224] is unnecessary. There, I saved you more characters than the 2 *s you were worried about!

I think the decision to omit direct-to-array casts was just to keep the feature change minimal. We need the cast-to-pointer regardless to support aliasing to the same backing store, and cast-to-array is easily implemented using cast-to-pointer.

Loading

@ianlancetaylor ianlancetaylor added this to Incoming in Proposals Jun 1, 2021
@go101
Copy link

@go101 go101 commented Jun 2, 2021

Some like #36890

Loading

@mdempsky
Copy link
Member

@mdempsky mdempsky commented Jun 2, 2021

I don't immediately see anything wrong with allowing slice to array conversions, but I feel like saving two asterisks in an already rather niche feature doesn't justify the extra complexity on compilers and tooling.

Loading

@bcmills
Copy link
Member

@bcmills bcmills commented Jun 2, 2021

Since this conversion would do a copy anyway, would we also add a conversion from string to [N]byte for consistency?

Loading

@rsc rsc changed the title proposal: allow conversion from slice to array proposal: spec: allow conversion from slice to array Jun 2, 2021
@rsc rsc moved this from Incoming to Active in Proposals Jun 2, 2021
@rsc
Copy link
Contributor

@rsc rsc commented Jun 2, 2021

This proposal has been added to the active column of the proposals project
and will now be reviewed at the weekly proposal review meetings.
— rsc for the proposal review group

Loading

@rogpeppe
Copy link
Contributor

@rogpeppe rogpeppe commented Jun 2, 2021

FWIW I didn't propose this originally because it could easily be implemented, even prior to #395, by using copy, albeit not quite so concisely.

I'm not entirely convinced that it's worth it tbh, especially given the possibility for the expression to panic.

Loading

@katiehockman
Copy link
Member

@katiehockman katiehockman commented Jun 2, 2021

Question: is it possible to make changes to copy() so that it is equally performant to the *(*array)(slice) conversion? Then we don't need to support this additional conversion at all, and it can just be common practice to use copy in such situations.

FWIW I'm a lot less concerned about a few extra characters in the line than I am about readability. I was pretty confused as a reader to see a conversion in the form *(*array)(slice). I read it as "Cast a slice to a pointer to an array, then dereference this array", which felt more like a hack than a best practice. But maybe I just need to get used to it.

Loading

@mdempsky
Copy link
Member

@mdempsky mdempsky commented Jun 2, 2021

Question: is it possible to make changes to copy() so that it is equally performant to the *(*array)(slice) conversion? Then we don't need to support this additional conversion at all, and it can just be common practice to use copy in such situations.

Probably. I don't think there's any fundamental reason they'd perform differently. Can you file an issue with examples of code that you think should perform equivalently, and we can look into it for Go 1.18?

I was pretty confused as a reader to see a conversion in the form *(*array)(slice). I read it as "Cast a slice to a pointer to an array, then dereference this array"

Assuming you meant "dereference this [pointer to] array", that description is pretty much how I think of it, though arguably it's more of a (checked) "assertion" (as in type assertions), as conversions generally can't fail. A slice is a pointer + dynamic length (and capacity). The conversion here is changing it to a pointer + static length, with a runtime check to ensure the lengths are compatible.

Loading

@katiehockman
Copy link
Member

@katiehockman katiehockman commented Jun 2, 2021

Can you file an issue with examples of code that you think should perform equivalently, and we can look into it for Go 1.18?

Brad wrote a little benchmark that demonstrated some of the performance differences which probably shouldn't be different: https://play.golang.org/p/BkYM9Gbi4t2. He got the following results:

BenchmarkNamed-8        201051339                6.263 ns/op
BenchmarkCopy-8         172203873                6.071 ns/op
BenchmarkPointer-8      1000000000               1.031 ns/op

You can also check out the Sum functions in https://go-review.googlesource.com/c/go/+/322329 which are good examples of code that should perform ~equivalently.

Loading

@mdempsky
Copy link
Member

@mdempsky mdempsky commented Jun 2, 2021

@katiehockman Thanks, filed #46529.

As noted in that issue, the first two functions have different semantics than the last function when len(b) < Size256. I suspect that doesn't matter for your use case, but the compiler would need some way to discern that can't happen before it can optimize them identically.

Loading

@kortschak
Copy link
Contributor

@kortschak kortschak commented Jun 2, 2021

ISTM that the need for a dereference is a nice visual signal to an author that *(*[n]T)(s[:n]) is more costly than (*[n]T)(s[:n]). As it is it's reasonably common the users are surprised that an array expression results in a copy of the array (for example in a range over an array value).

Loading

@bcmills
Copy link
Member

@bcmills bcmills commented Jun 3, 2021

ISTM that the need for a dereference is a nice visual signal to an author

I am personally unlikely to notice that signal among the line-noise of the rest of the expression.
Also note that ([]byte)(s) for a string s, or string(b) for a slice b, has no such visual signal — the destination type is in some sense “good enough”.

As it is it's reasonably common the users are surprised that an array expression results in a copy

Sure, but the ship has already sailed on that one. 😅 Given that we already have array values, it doesn't seem particularly more confusing to allow conversions to them.

Loading

@bcmills
Copy link
Member

@bcmills bcmills commented Jun 3, 2021

I am personally unlikely to noticed that signal among the line-noise of the rest of the expression.

Looking at this again, I'm not sure why the slice part of the expression is needed at all.
@bradfitz, could you not have written that as

	return *(*[Size224]byte)(sum)

?

And then the direct-conversion alternative becomes

	return ([Size224]byte)(sum)

which reads a lot cleaner to me. (Without the redundant slice expression, those redundant * tokens are a larger fraction of the noise.)

Loading

@kortschak
Copy link
Contributor

@kortschak kortschak commented Jun 3, 2021

Given that we already have array values, it doesn't seem particularly more confusing to allow conversions to them.

It's not a matter of confusion in this case, but signalling that there is more work being done; the pointer conversion is essentially free. WRT the signalling for string/[]byte, the difference there is that they are clearly distinct types, in this situation it is potentially a single character that makes the difference. between behaviours under the proposal.

Loading

@rsc
Copy link
Contributor

@rsc rsc commented Jun 9, 2021

Strictly speaking this is unnecessary. You can always write

*(*[10]int)(x)

instead of the proposed

[10]int(x)

That said, copying data out from a slice into a fixed-size array is pretty common when using fixed-size arrays (like checksums etc), so I don't really see the harm in making that less star-full.

/cc @robpike

Loading

@rsc
Copy link
Contributor

@rsc rsc commented Jul 14, 2021

Any objections to adding this?

Loading

@randall77
Copy link
Contributor

@randall77 randall77 commented Jul 14, 2021

I'm leaning against. I don't think adding a new conversion case is worth saving two *s.

Loading

@mdempsky
Copy link
Member

@mdempsky mdempsky commented Jul 14, 2021

I'm also leaning against.

We've had a string of fixes to x/tools to update code for slice-to-array-pointer conversions. None of the CLs have been very involved, but just cases where we forgot to update it. I don't think this feature is going to be so frequently used to justify forcing tools authors to support both forms.

Loading

@rsc
Copy link
Contributor

@rsc rsc commented Jul 21, 2021

Does anyone want to make an argument in favor?

Loading

@Splizard
Copy link

@Splizard Splizard commented Aug 4, 2021

Does anyone want to make an argument in favor?

I believe that this proposal would result in improved discoverability around this kind of type conversion, when I first started writing Go (~8 years ago) I recall at one point wondering why I couldn't convert a slice to an array just like this. This seemed like the obvious way to do it.

[10]int(x)

At the time (even if #395 existed), it would never have occurred to me that I could do something like this:

*(*[10]int)(x)

I had previously been using languages without pointers and I wasn't familiar with the semantics around dereferencing or how slices relate to them.

It's the kind of small thing as a beginner you don't even bother looking into when the compiler throws an error and you simply write it another way until you find out months or years later when you see it somewhere and/or learn more about pointer semantics and realise that this is possible.

IE. somebody who starts learning about pointer semantics in Go can infer that *(*[10]int)(x) is possible if they are already familiar with [10]int(x) but if this proposal is declined then beginners in Go are locked out from benefiting from #395 until they learn more about the language (and this is hindered further if they come from languages without pointers).

Instead of framing this proposal around "saving two asterisks", I would argue that it helps discoverability around #395 because it is an obvious way to convert from a slice to an array.

Loading

@deefdragon
Copy link

@deefdragon deefdragon commented Aug 11, 2021

@mdempsky Inside the generator function I started with slices. As I said, the generator func was small enough and self contained enough that I could have converted the entire generator func to use arrays internally as well, so it wasn't a perfect example.

Loading

@timothy-king
Copy link
Contributor

@timothy-king timothy-king commented Aug 12, 2021

Maybe I missed this, but I feel like size 0 arrays need to be addressed.

What does ([0]T)(x) do when x is nil? If it does panic,*(*[0]T)(x) and ([0]T)(x) are equivalent but this would be an instance of treating nil and a slice of length 0 differently. If it does not panic, *(*[0]T)(x) and ([0]T)(x) would not be equivalent.

Casting to a size 0 array is slightly pointless so maybe it should not be allowed? Would not allowing it be a "weird" discrepancy between the pointer-to-array cast and array cast?

Loading

@rsc
Copy link
Contributor

@rsc rsc commented Aug 18, 2021

The general answer is that [0]T(x) is fine when x is nil, just as it is fine when x is any other zero-length slice, and just as [N]T(x) is fine when x is an N-length slice. It should not panic, nor should it be disallowed. Let the general behavior speak for itself.

Loading

@timothy-king
Copy link
Contributor

@timothy-king timothy-king commented Aug 18, 2021

@rsc Having [0]T(x) always succeed works for me. As is it not equivalent to *(*[0]T)(x), I think it will end up being a special case for tooling like x/tools/go/ssa. This is not a big deal as the information to do this is always available in the type.

It might be worth making sure the 0 case is clear when/if the spec gets updated.

Loading

@rsc rsc moved this from Likely Accept to Accepted in Proposals Aug 18, 2021
@rsc
Copy link
Contributor

@rsc rsc commented Aug 18, 2021

No change in consensus, so accepted. 🎉
This issue now tracks the work of implementing the proposal.
— rsc for the proposal review group

Loading

@rsc rsc changed the title proposal: spec: allow conversion from slice to array spec: allow conversion from slice to array Aug 18, 2021
@rsc rsc removed this from the Proposal milestone Aug 18, 2021
@rsc rsc added this to the Backlog milestone Aug 18, 2021
@mdempsky
Copy link
Member

@mdempsky mdempsky commented Aug 18, 2021

We allow s[:0] on nil slices without nil panicking, so I agree allowing the conversion [0]T(s) is the consistent way to handle that case.

Loading

@go101
Copy link

@go101 go101 commented Aug 24, 2021

Is the syntax allowed to show up as l-values? And when as l-values, no duplications are made, right? (edit: sorry, it looks r-values also don't need duplications)

Loading

@ianlancetaylor
Copy link
Contributor

@ianlancetaylor ianlancetaylor commented Aug 24, 2021

Currently a conversion is never an l-value.

Loading

@go101
Copy link

@go101 go101 commented Aug 24, 2021

So *(*[N]T)(slice2) = *(*[N]T)(slice1) couldn't be written as ([N]T)(slice2) = ([N]T)(slice1)?

Loading

@mdempsky
Copy link
Member

@mdempsky mdempsky commented Aug 24, 2021

@go101 That question was answered above: #46505 (comment)

Loading

@kortschak
Copy link
Contributor

@kortschak kortschak commented Aug 24, 2021

Why would you do that rather than just use copy?

Loading

@go101
Copy link

@go101 go101 commented Aug 24, 2021

For small arrays, the way is faster: #46529

Loading

@go101
Copy link

@go101 go101 commented Aug 24, 2021

@mdempsky I think making an exception for this is reasonable and worthy it. Considering that []int{1, 2}[1] = 3 is valid, I think this is a consistency.

Loading

@DmitriyMV
Copy link

@DmitriyMV DmitriyMV commented Aug 24, 2021

([N]T)(slice2) = ([N]T)(slice1)

If my understanding is correct (at least what I would expect) is that:

	*(*[N]T)(slice2) = *(*[N]T)(slice1)

could be rewritten as:

	left := (*[N]T)(slice2)  // points to original underlying memory
	right := (*[N]T)(slice1) // points to original underlying memory
	*left = *right           // copies slice2 memory to slice1 memory

and where

	([N]T)(slice2) = ([N]T)(slice1)

could be rewritten as:

	left := ([N]T)(slice2)  // Creates a full copy, doesn't point to original slice
	right := ([N]T)(slice1) // Creates a full copy, doesn't point to original slice
	left = right

The two are nor semantically the same.

Loading

@go101
Copy link

@go101 go101 commented Aug 24, 2021

In my understanding, the conversions in the two lines just copy the data directly from the slices to the arrays.
There are no intermediate copies.

	left := ([N]T)(slice2)  // Creates a full copy, doesn't point to original slice
	right := ([N]T)(slice1) // Creates a full copy, doesn't point to original slice

[append]: the semantics should be like *p1 = *p2.

Loading

@zephyrtronium
Copy link
Contributor

@zephyrtronium zephyrtronium commented Aug 24, 2021

So *(*[N]T)(slice2) = *(*[N]T)(slice1) couldn't be written as ([N]T)(slice2) = ([N]T)(slice1)?

*(*[N]T)(slice2) = *(*[N]T)(slice1) could be rewritten as *(*[N]T)(slice2) = ([N]T)(slice1).

Loading

@ianlancetaylor
Copy link
Contributor

@ianlancetaylor ianlancetaylor commented Aug 25, 2021

@go101

I think making an exception for this is reasonable and worthy

That is a different language change proposal that should be discussed separately. This proposal just adds the conversion.

Loading

@go101
Copy link

@go101 go101 commented Sep 1, 2021

Is it better to let such conversion never panic?
I mean by padding zero values if the array size is larger than the slice length.

Loading

@ianlancetaylor
Copy link
Contributor

@ianlancetaylor ianlancetaylor commented Sep 9, 2021

@go101

Is it better to let such conversion never panic?

That was discussed extensively over at #395. This proposal should be consistent with that one, and the decision was to panic if the slice is too short.

Loading

@go101
Copy link

@go101 go101 commented Nov 6, 2021

Will this be implemented in Go 1.18? It looks it is not in tip yet.

Loading

@ianlancetaylor
Copy link
Contributor

@ianlancetaylor ianlancetaylor commented Nov 7, 2021

We didn't get to this for 1.18.

Loading

@griesemer griesemer removed this from the Backlog milestone Nov 9, 2021
@griesemer griesemer added this to the Go1.19 milestone Nov 9, 2021
@griesemer
Copy link
Contributor

@griesemer griesemer commented Nov 9, 2021

Marking for 1.19 so it doesn't get dropped.

Loading

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Proposals
Accepted
Linked pull requests

Successfully merging a pull request may close this issue.

None yet