Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

spec: review/clarify uses of "slice of bytes" #23814

Open
griesemer opened this issue Feb 13, 2018 · 9 comments
Open

spec: review/clarify uses of "slice of bytes" #23814

griesemer opened this issue Feb 13, 2018 · 9 comments
Assignees
Milestone

Comments

@griesemer
Copy link
Contributor

@griesemer griesemer commented Feb 13, 2018

This is a follow-up issue for #23536: Comb through documentation (spec, builtins) to verify the use of the term "slice of bytes". The spec appears to mean by this any type whose underlying type is []byte. But not all implementations agree.

@mdempsky
Copy link
Member

@mdempsky mdempsky commented Sep 27, 2019

I just ran into this issue. I think it's valuable to at least come to a shared agreement on what we think the right behavior is, so we can start aligning the implementations.

I think the spec cases relevant here are:

  1. Conversions: string -> []byte, string -> []rune, []byte -> string, []rune -> string
  2. copy([]byte, string)
  3. append([]byte, string...)

My 2c is that the spec should allow this compilation unit:

package p

type (
	String string
	Byte   byte
	Bytes  []Byte
	Rune   rune
	Runes  []Rune
)

var (
	s  String
	bs Bytes
	rs Runes
)

func f() {
	bs = Bytes(s)
	rs = Runes(s)

	s = String(bs)
	s = String(rs)

	copy(bs, s)
	bs = append(bs, s...)
}

That is, I think we should liberally interpret "slice of bytes" and "slice of runes" to mean "slice type with element type of underlying type byte/rune". I.e., defined slice types and (user-)defined element type should be allowed.

gccgo accepts this.

cmd/compile and go/types currently reject the copy call, claiming that Bytes's element type (Byte) is different from byte.

go/types currently additionally rejects the append call too.

(Also notably, cmd/compile's typechecker allows String(Bytes(nil)) and String(Runes(nil)), but then walk.go fails to generate the appropriate conversions when calling runtime.slicebytetostring and runtime.slicerunetostring, so compilation fails. It has a similar problem with the append call too.)

@mdempsky
Copy link
Member

@mdempsky mdempsky commented Sep 27, 2019

@gopherbot gopherbot removed the NeedsFix label Oct 22, 2019
@griesemer griesemer modified the milestones: Unplanned, Backlog Oct 22, 2019
@griesemer
Copy link
Contributor Author

@griesemer griesemer commented Oct 22, 2019

@rsc, @ianlancetaylor, @robpike Do you have any opinions here?

We're inconsistent at the moment and different implementations (cmd/compile, gccgo, go/types) differ for operations where this matters.

@bradfitz
Copy link
Contributor

@bradfitz bradfitz commented Oct 23, 2019

Personally, if there's any ambiguity that needs to be specified, I'd prefer to be a permissive as possible, letting copies/appends/conversions work as expected if the underlying representation is the same.

@ianlancetaylor
Copy link
Contributor

@ianlancetaylor ianlancetaylor commented Oct 31, 2019

I'm inclined to agree with @mdempsky 's comment #23814 (comment) . I don't see any particular reason to restrict operations on a "slice of bytes" to only the type []byte.

@josharian
Copy link
Contributor

@josharian josharian commented Feb 2, 2020

I just rediscovered this in #36965.

Is there sufficient consensus here that I can fix cmd/compile’s implementation to use a maximally liberal interpretation of “byte slices?

Other things to check on once the desired direction is clear: is the spec clear enough? Do go/types, package reflect, cmd/compile, and gccgo agree?

@josharian
Copy link
Contributor

@josharian josharian commented Feb 2, 2020

(The reason I’d like to fix cmd/compile now/soon is that I’m touching this code anyway for #36890.)

@griesemer
Copy link
Contributor Author

@griesemer griesemer commented Feb 3, 2020

@josharian The reason for this issue is that not all implementations agree. @mdempsky 's comment explains how they disagree.

I think we should first get the spec to be crystal clear before making changes. It sounds like we're leaning towards making the interpretation more liberal, so this should not break future code, but enable more. This is not urgent, either.

@griesemer
Copy link
Contributor Author

@griesemer griesemer commented Feb 3, 2020

Marking for Go 1.15.

@griesemer griesemer modified the milestones: Backlog, Go1.15 Feb 3, 2020
@ianlancetaylor ianlancetaylor modified the milestones: Go1.15, Go1.16 May 19, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
6 participants
You can’t perform that action at this time.