types2: Generics allow creating values of unexported or internal types

[Merovius]

I'm not sure if this is a bug or should be a proposal, or if it is just a curiosity. But I noticed a thing and I think it's weird.

Say package a wants to make sure that of some type, only well-controlled values are ever used (in my use case, its an interface which I want to share between packages but be able to change without breaking compatibility). It uses an internal package to do so:

-- a/a.go --
package a

import "a/internal"

func MakeX() internal.X {
	return internal.Make(42)
}

func F(x internal.X) {
	if x.V() != 42 {
		panic(fmt.Errorf("invalid %#v", x))
	}
}
-- a/internal/internal.go --
package internal

type X struct { v int }

func Make(v int) X { return X{v} }

func (x X) V() int { return x.v }

-- b/b.go --
package main

import (
	"a"
	// illegal: use of internal package a/internal not allowed
	// "a/internal"
)

func main() {
	// doesn't work
	// a.F(internal.X{})
	a.F(a.MakeX())
}

As a/internal cannot be imported, there is no way to get an internal.X that isn't sanctioned by a. reflect can be used to construct an any with dynamic type internal.X, but that can't be type-asserted, so it can't be passed to a.F.

However, using generics, we can do this trick:

package main

import "x/a"

func main() {
	a.F(trick(a.MakeX))
}

func trick[T any](f func() T) T {
	return *new(T)
}

The same also applies if the type is not from an internal package, but an unexported type. I suspect a similar issue would apply to #21498 (there is currently no way for a different package to write a func(unexported), but #21498 would allow it by spelling it (x) => { … }).

I'm not sure how important it is to prevent this. But I don't think this is how it should work. I think if an inferred type argument is from an internal package or is a defined type with unexported name from a different package, it should fail. But doing that would technically be a breaking change.

Just thought I'd bring this up, at least.

[mknyszek]

CC @golang/compiler

[randall77]

You can also do this workaround using reflect:

	f := reflect.ValueOf(a.F)
	argType := f.Type().In(0)
	arg := reflect.New(argType).Elem()
	f.Call([]reflect.Value{arg})

[Merovius]

True, the call still works using reflect. Hm.

[griesemer]

The handling of internal packages is "external" to the language, it's really the build system that prevents invalid internal imports. So I'd leave that out from the discussion.

But as you say, it doesn't require an internal package. Here's a playground link for the "trick" and reflection-based scenario simply using an unexported type.

It's always been possible to create invalid values of unexported types, without using generics, and without reflection, at least for basic types. Here's an (admittedly pathological) example which creates a zero value for an unexported numeric type because operations such as - are still available.

The upcoming clear built-in (#56351) will zero the elements of a slice, even if the slice elements are of unexported type and there's no other mechanism to create that zero value.

I wonder if the zero value for a type is the only value that can be created this way (ignoring numeric types, where one probably can create a 1, and then all the values, using suitable operations).

And I wonder whether clear is making the hole bigger.

(I think we could arguably fix the type inference rules by treating this as a bug: one should not be able to infer an unimportable type. I don't think there's a substantial amount of code dependent on this.)

I think we should probably fix the type inference issue: it should not be possible to infer a type that is not otherwise accessible in a package. We have carved out a provision in our backward-compatibility guarantees for errors in the generics implementation.

[zephyrtronium]

And I wonder whether clear is making the hole bigger.

clear only applies when there is a function that returns specifically a map, slice, or array of some unexported element type, otherwise you can't mention b.unexported to construct []b.unexported or map[K]b.unexported. Obtaining the zero value from a map is trivial: delete a key and then access that key (which works even if the key type is also unexported by using a range loop). Non-empty slices and arrays can be used to obtain the zero value by repeatedly appending the first element to the slice (starting with s := array[:] for the latter) until cap > len. The last case to consider is a map with an unexported key type, but there is no (non-reflective, non-generic) way to get the zero value from that.

If clear could zero any arbitrary pointer type, as was at one point proposed, then it would introduce a new way to obtain zero values of unexported types without reflection or generics. Since that capability has been retracted, I don't see any way that clear makes it possible to access zero values that could not be accessed before.

[griesemer]

Good point about maps: one simply needs to access an element with a key that doesn't exist, using comma-ok form, and one gets a zero value. Same for type assertions, etc. Assuming that the map element type is unexported.

I don't follow your slices example: if the slice is not empty, all we can append are the element (or other) values that already exist. What am I missing?

There's also the case of a channel with elements of unexported type. Once the channel is closed, one receives zero values.

But in all cases a package defining the unexported type can chose to not export a map, channel, slice, or array with that unexported type as element type.

[zephyrtronium]

I don't follow your slices example: if the slice is not empty, all we can append are the element (or other) values that already exist. What am I missing?

We can append the values that already exist until append grows the slice, at which point we have zero values:

for len(s) == cap(s) {
	s = append(s[:cap(s)], s[0])
}
fmt.Printf("%#v\n", s[:cap(s)][cap(s)-1])

https://go.dev/play/p/QrXkeXqvMfn

But in all cases a package defining the unexported type can chose to not export a map, channel, slice, or array with that unexported type as element type.

I agree. But then clear doesn't widen the hole. That's what I meant to show – perhaps more as a defense of #56351 than anything relating to this issue – prior to your edit. 🙂

[griesemer]

Tricky! It seems that the spec doesn't even say that the values past the length of a slice are zero values in case of growth. Hm. (Filed #56684.)

But yes - it looks like clear is in the clear with respect to adding unexpected capabilities. It does not.

[neild]

As a concrete impact, this does make it quite a bit easier to evade the constant string checks in github.com/google/safehtml.

type stringConstant string

// The parameter to ScriptFromConstant must be an untyped string constant.
func ScriptFromConstant(script stringConstant) Script

func callWithInconstantString[S ~string, R any](f func(S) R, s string) R {
  return f(S(s))
}

s := "this is not a constant string"
script := callWithInconstantString(ScriptFromConstant, s)

There are other, less convenient, ways to elude these checks, so this isn't a tremendous problem; the goal of the package is to avoid accidental error, not defend against malice on the part of the programmer. Still, it might support a change to avoid inferring types that the instantiating package can't name.

[DmitriyMV]

Will this code https://go.dev/play/p/ASrjalg4xdr continue to work? I mean such usage of type aliases was not exactly correct, but it worked even before generics.