go/types, cmd/compile/internal/types2: disallow type parameters as RHS of type declarations

[findleyr]

Consider the following snippet, with annotated type checking error.

type xer interface {
        x() string
}

func Print[T xer](s []T) {
        type Y T
        for _, v := range s {
                fmt.Println(v.x())
        }
        var y Y
        y.x() // <-- ERROR "y.x undefined (interface xer has no method x)"
}

This error is both inaccurate and (as a secondary concern) fails to mention the named type Y.

Update: per discussion below, let's disallow such declarations (as well as type Y = T) for now.

CC @griesemer

[gopherbot]

Change https://golang.org/cl/311455 mentions this issue: go/types: support type parameters in NewMethodSet

[griesemer]

Our proposals do not explain what Y should be. Should it be a type parameter? Or an interface? Maybe it shouldn't be permitted.

[findleyr]

Yes, sorry to be clear the problem here may only be the misleading error message. I wasn't sure.

I think we should either allow Y to have a type parameter as its underlying type (and remove the guard against type parameters in rawLookupFieldOrMethod), or make it an error to declare such a Y. I think it would be more confusing to make Y an interface, as then we'd lose convertibility: var y Y; t := T(y) would be invalid.

Perhaps in the absence of any reason to write this code, we should err on the side of caution and disallow such a declaration (at least for now).

[griesemer]

Let's disallow it for now. Same for local aliases of type parameters (even though those shouldn't be a problem, I think).

[findleyr]

Let's disallow it for now. Same for local aliases of type parameters

Sounds good. Updated the issue accordingly.

[findleyr]

As discussed in #43621, the case of type P[T any] T is equivalently problematic to the local declaration type Y T. Renaming this issue to track both cases.

[griesemer]

Per discussion with @iant, we do want to permit (non-local) generic type declarations of the form

type T[P C] P

because those are useful, for instance to add methods to a type P.

But we don't want to permit (local) type declarations of the form

type T P

where P is a type parameter of an enclosing function.

[gopherbot]

Change https://golang.org/cl/332411 mentions this issue: [dev.typeparams] cmd/compile/internal/types2: disallow type parameters as RHS of type declarations

[mdempsky]

Per discussion with @iant, we do want to permit (non-local) generic type declarations of the form

type T[P C] P

because those are useful, for instance to add methods to a type P.

As noted at #43621 (comment), it's not safe/possible to add methods to pointer or interface types. So C needs to at least be constrained to not allow either of those in its type set (edit:) to allow declaring methods on T[P].

What are some of the examples of where this is useful? Would they not be equally useful/applicable if P was replaced with either struct { p P } or struct { p *P }?

[mdempsky]

/cc @ianlancetaylor (apologies to @ iant)

[ianlancetaylor]

At top level defining a type as its own parameter is useful as seen, for example, in the code (still using old syntax) at https://go.googlesource.com/proposal/+/refs/heads/master/design/43651-type-parameters.md#absolute-difference .

[mdempsky]

At top level defining a type as its own parameter is useful as seen, for example, in the code (still using old syntax) at https://go.googlesource.com/proposal/+/refs/heads/master/design/43651-type-parameters.md#absolute-difference .

Thanks.

It seems like the consequence of disallowing type P[T ...] T would be that:

type ComplexAbs[T Complex] T

func (a ComplexAbs[T]) Abs() ComplexAbs[T] {
	d := math.Hypot(float64(real(a)), float64(imag(a)))
	return ComplexAbs[T](complex(d, 0))
}

func ComplexAbsDifference[T Complex](a, b T) T {
	return T(AbsDifference(ComplexAbs[T](a), ComplexAbs[T](b)))
}

needs to be written instead as:

type ComplexAbs[T Complex] struct{ X T }

func (a ComplexAbs[T]) Abs() ComplexAbs[T] {
	d := math.Hypot(float64(real(a.X)), float64(imag(a.X)))
	return ComplexAbs[T](complex(d, 0))
}

func ComplexAbsDifference[T Complex](a, b T) T {
	return T(AbsDifference(ComplexAbs[T]{a}, ComplexAbs[T]{b}))
}

That doesn't seem substantively different to me.

I'm still currently leaning for blanket disallowing type P[T C] T, at least in Go 1.18. Otherwise, I think we'll need to specify a lot of corner cases around handling/preventing promoted methods and also reflection API issues like #43621 (comment). The cases that make type P[T C] struct { T } tricky also apply to type P[T C] T.

[ianlancetaylor]

I don't feel strongly about it.

[mdempsky]

On CL 294469, @ianlancetaylor wrote:

Yes, for "type L io.Writer", L has a method. But I am arguing that if we write

   func F[P io.Writer]() {
       type L P
   }

then L does not have a method. In this case, P has a Write method, and so does a value of type P. But if F is instantiated with a type like bytes.Buffer, then although a value of type P has a Write method, a value of type L should not have a Write method. Or so it seems to me.

My point is if you instantiate F[io.Writer] then regardless how we statically treat L at compile-time, we need to reconcile with the fact that at run-time it's going to be an interface type with a non-empty method set. In particular, I worry about how this affects users of package reflect, esp. in the presence of method promotion.

I think a reasonable rule could be something like "it's invalid to embed a type into a struct if its method set depends upon the type arguments used for instantiation," but that's fairly subtle both to specify and implement. I'd rather defer to post-1.18 to worry about specifying a rule like that, assuming there's user demand.

[randall77]

@danscales

[ianlancetaylor]

I'm not quite following you, sorry. My way of thinking is that compile-time and execution-time handling of type parameters are two different things. At compile time we have various complexities where we are trying to work through exactly which type arguments are permitted and exactly which operations are permitted on those type arguments. I don't think we're there yet when it comes to things like promoted methods.

At execution time, though, we know what we want: the function should act as though the type parameters are consistently replaced throughout by the type arguments. We know here are some cases there where we have to be different, as in unsafe.Sizeof can't always be a constant, and type switches must permit duplicate cases. But those are also by and large compilation-time details.

With this view, I'm not sure where confusion about method promotion arises. I'm not saying that it doesn't arise, I just don't see it yet.

It's worth noting that it may be the case that some code is too complicated for dictionaries. For some code we may have to fall back to stenciling. I think that's fine, assuming that that code is not common. It's also of course fine to postpone handling some cases entirely.

But I don't know why we would want a rule like "it's invalid to embed a type into a struct if its method set depends upon the type arguments used for instantiation." Maybe we do. And it's quite possible that we want a compile-time rule like "if the method set of an embedded type depends on type arguments, then those methods may not be called." But I'm not sure why we need a rule like this at execution time.

[ianlancetaylor]

I'm reopening this issue for more consideration for the 1.19 release.

There won't be any further change for 1.18.

The specific question is whether to permit

type P[T C] T

and then permit defining methods on P. I think we agree that we should not permit embedding type parameters. There are clearly difficulties with type P[T C] T, but as it seems like a very convenient mechanism for some cases, we should see whether we can work them out.

[matthewmueller]

I arrived at this issue trying the following:

package main

import (
	"bytes"
	"io"
)

func main() {
	Show(bytes.NewBufferString(""))
}

type Body[T any] = T

func Show(body Body[io.Reader]) {
  buf, err := ioutil.ReadAll(body)
}

And getting this error:

./prog.go:11:6: generic type cannot be alias
./prog.go:11:20: cannot use a type parameter as RHS in type declaration

I thought I'd share my use case for wanting alias support, though I admit, it might be rather niche.

I'd like to (ab)use the Body[T] type as an annotation to be able to generate a function that reads from the request body and passes you an io.Reader. You could also have Body[*SignupParams], which would generate code that unmarshals the request body and passes you *SignupParams.

[perillo]

Here is an implementation of absolute difference from the Type Parameters Proposal that does not use type parameters as RHS of type:
https://gotipplay.golang.org/p/DYwfTfAvfnT

The code does not compile:

./prog.go:63:22: OrderedAbs[T] does not implement NumericAbs[OrderedAbs[T]]
./prog.go:69:24: ComplexAbs[T] does not implement NumericAbs[ComplexAbs[T]]

[vsivsi]

Here's a type of case I've run into that would clearly benefit from allowing declarations of type P[T C] T

What I want to do is:

type VarLenBits interface {
	~[1]uint64 | ~[2]uint64 | ~[3]uint64 | ~[4]uint64 | ~[]uint64
}

// ERROR: cannot use a type parameter as RHS in type declaration
type bitVectorV[T VarLenBits] T

// This doesn't work because of invalid type bitVectorV:
func (b bitVectorV[T]) hammingD(a bitVectorV[T]) (d int) {
	for i := 0; i < len(a); i++ {
		d += bits.OnesCount64(a[i] ^ b[i])
	}
	return d
}

Nope... How about make it a pointer instead?

// Valid! 
type bitVectorP[T VarLenBits] *T

// ERROR: invalid receiver type bitVectorP[T] (pointer or interface type)
func (bp bitVectorP[T]) hammingD(ap bitVectorP[T]) (d int) {
	a, b := *ap, *bp
	for i := 0; i < len(a); i++ {
		d += bits.OnesCount64(a[i] ^ b[i])
	}
	return d
}

Hmmm... Maybe this will work!

type bitVectorA[T VarLenBits] [1]*T

func newBVA[T bitVectorA[A], A VarLenBits](in *A) (out T) {
	out[0] = in
	return
}

// Works! 
func (bp bitVectorA[T]) hammingD(ap bitVectorA[T]) (d int) {
	a, b := *ap[0], *bp[0]
	for i := 0; i < len(a); i++ {
		d += bits.OnesCount64(a[i] ^ b[i])
	}
	return d
}

The above kludge (using [1]*T as the carrier type) is the only way I've found to make this work as a generic method. During 1.18 beta it also worked with something like type bitVectorS[T VarLenBits] struct { val *T }, but in 1.18 it is no longer possible to access x.val generically.

More elaboration of this case with working code here: https://go.dev/play/p/fl9rBkCrMNi

[griesemer]

This restriction (disallow type parameters as RHS of type declarations) has been implemented for 1.18. There's nothing left to do here for now. If we want to change this behavior we need a detailed proposal. Closing.

[EricRabil]

Playing devils advocate, having the ability to do this would make it easier to marshal multiple structs into a flat namespace.

Consider the following:

You are writing an HTTP client library. You have methods wrapping various API endpoints, each with a struct outlining the body for the associated API. Users of your library are requesting the ability to supply additional fields for the body beyond what is explicitly defined in the struct.

You have two options:

• https://github.com/mautrix/go/blob/master/event/content.go#L120-L158
  Here, if the user supplies a body to serialize and supplies custom fields to merge, you have to do the following:

  • Marshal your typed body,
  • Unmarshal that result into a map[string]interface{},
  • Deeply merge your custom body into said map[string]interface{},
  • And finally marshal your map[string]interface{} to get your merged JSON result

• https://go.dev/play/p/gfRek36wXfJ
  Here, if the user supplies a body to serialize and supplies custom fields to merge, all you do is:

  • Create a struct with two embedded values one being the body and one being the custom fields,
  • Marshal it

While the second approach doesn't support a custom body whose type is map[string]interface{} it gets you a lot closer to dynamically merged serialization.