Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

go/types, cmd/compile/internal/types2: disallow type parameters as RHS of type declarations #45639

Closed
findleyr opened this issue Apr 19, 2021 · 39 comments
Labels
FrozenDueToAge generics Issue is related to generics NeedsFix The path to resolution is known, but the work has not been done.
Milestone

Comments

@findleyr
Copy link
Contributor

findleyr commented Apr 19, 2021

Consider the following snippet, with annotated type checking error.

type xer interface {
        x() string
}

func Print[T xer](s []T) {
        type Y T
        for _, v := range s {
                fmt.Println(v.x())
        }
        var y Y
        y.x() // <-- ERROR "y.x undefined (interface xer has no method x)"
}

This error is both inaccurate and (as a secondary concern) fails to mention the named type Y.

Update: per discussion below, let's disallow such declarations (as well as type Y = T) for now.

CC @griesemer

@findleyr findleyr added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Apr 19, 2021
@findleyr findleyr added this to the Go1.18 milestone Apr 19, 2021
@gopherbot
Copy link
Contributor

Change https://golang.org/cl/311455 mentions this issue: go/types: support type parameters in NewMethodSet

@griesemer griesemer self-assigned this Apr 19, 2021
@griesemer
Copy link
Contributor

Our proposals do not explain what Y should be. Should it be a type parameter? Or an interface? Maybe it shouldn't be permitted.

@findleyr
Copy link
Contributor Author

Yes, sorry to be clear the problem here may only be the misleading error message. I wasn't sure.

I think we should either allow Y to have a type parameter as its underlying type (and remove the guard against type parameters in rawLookupFieldOrMethod), or make it an error to declare such a Y. I think it would be more confusing to make Y an interface, as then we'd lose convertibility: var y Y; t := T(y) would be invalid.

Perhaps in the absence of any reason to write this code, we should err on the side of caution and disallow such a declaration (at least for now).

@griesemer
Copy link
Contributor

Let's disallow it for now. Same for local aliases of type parameters (even though those shouldn't be a problem, I think).

@findleyr findleyr changed the title go/types, cmd/compile/internal/types2: bad method lookup for named type with type parameter underlying go/types, cmd/compile/internal/types2: disallow local declaration and aliasing of type parameters Apr 20, 2021
@findleyr findleyr changed the title go/types, cmd/compile/internal/types2: disallow local declaration and aliasing of type parameters go/types, cmd/compile/internal/types2: disallow local declarations and aliases of type parameters Apr 20, 2021
@findleyr
Copy link
Contributor Author

Let's disallow it for now. Same for local aliases of type parameters

Sounds good. Updated the issue accordingly.

gopherbot pushed a commit that referenced this issue Apr 20, 2021
Add handling for TypeParams in NewMethodSet, to bring it in sync with
lookupFieldOrMethod. Also add a test, since we had none. I wanted this
fix to get gopls completion working with type params, but due to the
subtlety of lookupFieldOrMethod, I left a TODO to confirm that there are
no behavioral differences between the APIs.

Updates #45639

Change-Id: I16723e16d4d944ca4ecb4d87fc196815abb6fcff
Reviewed-on: https://go-review.googlesource.com/c/go/+/311455
Trust: Robert Findley <rfindley@google.com>
Trust: Robert Griesemer <gri@golang.org>
Run-TryBot: Robert Findley <rfindley@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
@findleyr findleyr self-assigned this May 30, 2021
@findleyr
Copy link
Contributor Author

findleyr commented Jun 8, 2021

As discussed in #43621, the case of type P[T any] T is equivalently problematic to the local declaration type Y T. Renaming this issue to track both cases.

@findleyr findleyr changed the title go/types, cmd/compile/internal/types2: disallow local declarations and aliases of type parameters go/types, cmd/compile/internal/types2: disallow type parameters as RHS of type declarations Jun 8, 2021
@griesemer griesemer added NeedsDecision Feedback is required from experts, contributors, and/or the community before a change can be made. and removed NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels Jul 1, 2021
@ianlancetaylor ianlancetaylor added the generics Issue is related to generics label Jul 1, 2021
@griesemer
Copy link
Contributor

Per discussion with @iant, we do want to permit (non-local) generic type declarations of the form

type T[P C] P

because those are useful, for instance to add methods to a type P.

But we don't want to permit (local) type declarations of the form

type T P

where P is a type parameter of an enclosing function.

@gopherbot
Copy link
Contributor

Change https://golang.org/cl/332411 mentions this issue: [dev.typeparams] cmd/compile/internal/types2: disallow type parameters as RHS of type declarations

gopherbot pushed a commit that referenced this issue Jul 7, 2021
…rameter as RHS of a type declaration

For #45639.

Change-Id: I20e331b04f464db81e916af75f70ec8ae73eb989
Reviewed-on: https://go-review.googlesource.com/c/go/+/332411
Trust: Robert Griesemer <gri@golang.org>
Reviewed-by: Robert Findley <rfindley@google.com>
@mdempsky
Copy link
Member

mdempsky commented Jul 14, 2021

Per discussion with @iant, we do want to permit (non-local) generic type declarations of the form

type T[P C] P

because those are useful, for instance to add methods to a type P.

As noted at #43621 (comment), it's not safe/possible to add methods to pointer or interface types. So C needs to at least be constrained to not allow either of those in its type set (edit:) to allow declaring methods on T[P].

What are some of the examples of where this is useful? Would they not be equally useful/applicable if P was replaced with either struct { p P } or struct { p *P }?

@mdempsky
Copy link
Member

/cc @ianlancetaylor (apologies to @ iant)

@ianlancetaylor
Copy link
Contributor

At top level defining a type as its own parameter is useful as seen, for example, in the code (still using old syntax) at https://go.googlesource.com/proposal/+/refs/heads/master/design/43651-type-parameters.md#absolute-difference .

@mdempsky
Copy link
Member

At top level defining a type as its own parameter is useful as seen, for example, in the code (still using old syntax) at https://go.googlesource.com/proposal/+/refs/heads/master/design/43651-type-parameters.md#absolute-difference .

Thanks.

It seems like the consequence of disallowing type P[T ...] T would be that:

type ComplexAbs[T Complex] T

func (a ComplexAbs[T]) Abs() ComplexAbs[T] {
	d := math.Hypot(float64(real(a)), float64(imag(a)))
	return ComplexAbs[T](complex(d, 0))
}

func ComplexAbsDifference[T Complex](a, b T) T {
	return T(AbsDifference(ComplexAbs[T](a), ComplexAbs[T](b)))
}

needs to be written instead as:

type ComplexAbs[T Complex] struct{ X T }

func (a ComplexAbs[T]) Abs() ComplexAbs[T] {
	d := math.Hypot(float64(real(a.X)), float64(imag(a.X)))
	return ComplexAbs[T](complex(d, 0))
}

func ComplexAbsDifference[T Complex](a, b T) T {
	return T(AbsDifference(ComplexAbs[T]{a}, ComplexAbs[T]{b}))
}

That doesn't seem substantively different to me.

I'm still currently leaning for blanket disallowing type P[T C] T, at least in Go 1.18. Otherwise, I think we'll need to specify a lot of corner cases around handling/preventing promoted methods and also reflection API issues like #43621 (comment). The cases that make type P[T C] struct { T } tricky also apply to type P[T C] T.

@ianlancetaylor
Copy link
Contributor

I don't feel strongly about it.

@mdempsky
Copy link
Member

mdempsky commented Jul 19, 2021

On CL 294469, @ianlancetaylor wrote:

Yes, for "type L io.Writer", L has a method. But I am arguing that if we write

   func F[P io.Writer]() {
       type L P
   }

then L does not have a method. In this case, P has a Write method, and so does a value of type P. But if F is instantiated with a type like bytes.Buffer, then although a value of type P has a Write method, a value of type L should not have a Write method. Or so it seems to me.

My point is if you instantiate F[io.Writer] then regardless how we statically treat L at compile-time, we need to reconcile with the fact that at run-time it's going to be an interface type with a non-empty method set. In particular, I worry about how this affects users of package reflect, esp. in the presence of method promotion.

I think a reasonable rule could be something like "it's invalid to embed a type into a struct if its method set depends upon the type arguments used for instantiation," but that's fairly subtle both to specify and implement. I'd rather defer to post-1.18 to worry about specifying a rule like that, assuming there's user demand.

@randall77
Copy link
Contributor

@danscales

@ianlancetaylor
Copy link
Contributor

I'm not quite following you, sorry. My way of thinking is that compile-time and execution-time handling of type parameters are two different things. At compile time we have various complexities where we are trying to work through exactly which type arguments are permitted and exactly which operations are permitted on those type arguments. I don't think we're there yet when it comes to things like promoted methods.

At execution time, though, we know what we want: the function should act as though the type parameters are consistently replaced throughout by the type arguments. We know here are some cases there where we have to be different, as in unsafe.Sizeof can't always be a constant, and type switches must permit duplicate cases. But those are also by and large compilation-time details.

With this view, I'm not sure where confusion about method promotion arises. I'm not saying that it doesn't arise, I just don't see it yet.

It's worth noting that it may be the case that some code is too complicated for dictionaries. For some code we may have to fall back to stenciling. I think that's fine, assuming that that code is not common. It's also of course fine to postpone handling some cases entirely.

But I don't know why we would want a rule like "it's invalid to embed a type into a struct if its method set depends upon the type arguments used for instantiation." Maybe we do. And it's quite possible that we want a compile-time rule like "if the method set of an embedded type depends on type arguments, then those methods may not be called." But I'm not sure why we need a rule like this at execution time.

adamhassel pushed a commit to adamhassel/stringer that referenced this issue Nov 19, 2021
Per the issue below, a lone type parameter is not permitted
on the the RHS of a type declaration, at least for Go 1.18.

Slightly modified the generics tests so that they pass for
now.

For golang/go#45639.

Change-Id: I3c5dc0ff65bfdc268c372e5e3fdfe00a8547f17e
Reviewed-on: https://go-review.googlesource.com/c/tools/+/359274
Trust: Robert Griesemer <gri@golang.org>
Run-TryBot: Robert Griesemer <gri@golang.org>
gopls-CI: kokoro <noreply+kokoro@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Peter Weinberger <pjw@google.com>
@ianlancetaylor
Copy link
Contributor

ianlancetaylor commented Nov 29, 2021

I'm reopening this issue for more consideration for the 1.19 release.

There won't be any further change for 1.18.

The specific question is whether to permit

type P[T C] T

and then permit defining methods on P. I think we agree that we should not permit embedding type parameters. There are clearly difficulties with type P[T C] T, but as it seems like a very convenient mechanism for some cases, we should see whether we can work them out.

@ianlancetaylor ianlancetaylor modified the milestones: Go1.18, Go1.19 Nov 29, 2021
@ianlancetaylor ianlancetaylor added NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. and removed NeedsFix The path to resolution is known, but the work has not been done. labels Nov 29, 2021
@mdempsky

This comment has been minimized.

@ianlancetaylor

This comment has been minimized.

@carlverge

This comment has been minimized.

@griesemer

This comment has been minimized.

@matthewmueller
Copy link

matthewmueller commented Dec 30, 2021

I arrived at this issue trying the following:

package main

import (
	"bytes"
	"io"
)

func main() {
	Show(bytes.NewBufferString(""))
}

type Body[T any] = T

func Show(body Body[io.Reader]) {
  buf, err := ioutil.ReadAll(body)
}

And getting this error:

./prog.go:11:6: generic type cannot be alias
./prog.go:11:20: cannot use a type parameter as RHS in type declaration

I thought I'd share my use case for wanting alias support, though I admit, it might be rather niche.

I'd like to (ab)use the Body[T] type as an annotation to be able to generate a function that reads from the request body and passes you an io.Reader. You could also have Body[*SignupParams], which would generate code that unmarshals the request body and passes you *SignupParams.

@perillo
Copy link
Contributor

perillo commented Jan 22, 2022

Here is an implementation of absolute difference from the Type Parameters Proposal that does not use type parameters as RHS of type:
https://gotipplay.golang.org/p/DYwfTfAvfnT

The code does not compile:

./prog.go:63:22: OrderedAbs[T] does not implement NumericAbs[OrderedAbs[T]]
./prog.go:69:24: ComplexAbs[T] does not implement NumericAbs[ComplexAbs[T]]

@vsivsi
Copy link

vsivsi commented Mar 23, 2022

Here's a type of case I've run into that would clearly benefit from allowing declarations of type P[T C] T

What I want to do is:

type VarLenBits interface {
	~[1]uint64 | ~[2]uint64 | ~[3]uint64 | ~[4]uint64 | ~[]uint64
}

// ERROR: cannot use a type parameter as RHS in type declaration
type bitVectorV[T VarLenBits] T

// This doesn't work because of invalid type bitVectorV:
func (b bitVectorV[T]) hammingD(a bitVectorV[T]) (d int) {
	for i := 0; i < len(a); i++ {
		d += bits.OnesCount64(a[i] ^ b[i])
	}
	return d
}

Nope... How about make it a pointer instead?

// Valid! 
type bitVectorP[T VarLenBits] *T

// ERROR: invalid receiver type bitVectorP[T] (pointer or interface type)
func (bp bitVectorP[T]) hammingD(ap bitVectorP[T]) (d int) {
	a, b := *ap, *bp
	for i := 0; i < len(a); i++ {
		d += bits.OnesCount64(a[i] ^ b[i])
	}
	return d
}

Hmmm... Maybe this will work!

type bitVectorA[T VarLenBits] [1]*T

func newBVA[T bitVectorA[A], A VarLenBits](in *A) (out T) {
	out[0] = in
	return
}

// Works! 
func (bp bitVectorA[T]) hammingD(ap bitVectorA[T]) (d int) {
	a, b := *ap[0], *bp[0]
	for i := 0; i < len(a); i++ {
		d += bits.OnesCount64(a[i] ^ b[i])
	}
	return d
}

The above kludge (using [1]*T as the carrier type) is the only way I've found to make this work as a generic method. During 1.18 beta it also worked with something like type bitVectorS[T VarLenBits] struct { val *T }, but in 1.18 it is no longer possible to access x.val generically.

More elaboration of this case with working code here: https://go.dev/play/p/fl9rBkCrMNi

@griesemer
Copy link
Contributor

This restriction (disallow type parameters as RHS of type declarations) has been implemented for 1.18. There's nothing left to do here for now. If we want to change this behavior we need a detailed proposal. Closing.

@dmitshur dmitshur added NeedsFix The path to resolution is known, but the work has not been done. and removed NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels Apr 29, 2022
@dmitshur dmitshur modified the milestones: Go1.19, Go1.18 Apr 29, 2022
@EricRabil
Copy link

Playing devils advocate, having the ability to do this would make it easier to marshal multiple structs into a flat namespace.

Consider the following:

You are writing an HTTP client library. You have methods wrapping various API endpoints, each with a struct outlining the body for the associated API. Users of your library are requesting the ability to supply additional fields for the body beyond what is explicitly defined in the struct.

You have two options:

  • https://github.com/mautrix/go/blob/master/event/content.go#L120-L158
    Here, if the user supplies a body to serialize and supplies custom fields to merge, you have to do the following:

    • Marshal your typed body,
    • Unmarshal that result into a map[string]interface{},
    • Deeply merge your custom body into said map[string]interface{},
    • And finally marshal your map[string]interface{} to get your merged JSON result
  • https://go.dev/play/p/gfRek36wXfJ
    Here, if the user supplies a body to serialize and supplies custom fields to merge, all you do is:

    • Create a struct with two embedded values one being the body and one being the custom fields,
    • Marshal it

While the second approach doesn't support a custom body whose type is map[string]interface{} it gets you a lot closer to dynamically merged serialization.

VojtechVitek added a commit to golang-cz/gospeak that referenced this issue Aug 3, 2023
The generic Enum type didn't work, as it failed with:
"cannot use a type parameter as RHS in type declaration"

golang/go#45639
@golang golang locked and limited conversation to collaborators Dec 5, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge generics Issue is related to generics NeedsFix The path to resolution is known, but the work has not been done.
Projects
None yet
Development

No branches or pull requests