Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

proposal: spec: permit referring to a field shared by all elements of a type set #48522

Open
beoran opened this issue Sep 21, 2021 · 45 comments
Open
Assignees
Labels
generics Issue is related to generics Proposal
Milestone

Comments

@beoran
Copy link

beoran commented Sep 21, 2021

What version of Go are you using (go version)?

$ go version
/tmp/golang-tip/bin/go version                                                                                                                      
go version devel go1.18-986f8ea6b4 Tue Sep 21 00:59:42 2021 +0000 linux/amd64

Does this issue reproduce with the latest release?

No, it is a generics issue, therefore tested with a recent tip only.

What operating system and processor architecture are you using (go env)?

linux/amd64


go env Output
$ go env
GO111MODULE="" 
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/me/.cache/go-build"
GOENV="/home/me/.config/go/env"            
GOEXE=""                                                                                                                                                                             GOEXPERIMENT=""                                                                                                                                                                      GOFLAGS=""                                                                                                                                                                           GOHOSTARCH="amd64"                                                                                                                                                                   GOHOSTOS="linux"                                                                                                                                                                     GOINSECURE=""                                                                                                                                                                        GOMODCACHE="/home/me/src/go/pkg/mod"                                                                                                                                           GONOPROXY="k8s.io/*"
GONOSUMDB=""                                                                                                                                                                         GOOS="linux"                                                                                                                                                                         GOPATH="/home/me/src/go"                                                                                                                                                       GOPRIVATE=""                                                                                                                                                                         GOPROXY=""                                                                                                                                 
GOROOT="/tmp/golang-tip"                                                                                                                                                             GOSUMDB="off"                                                                                                                                                                        GOTMPDIR=""                                                                                                                                                                          GOTOOLDIR="/tmp/golang-tip/pkg/tool/linux_amd64"                                                                                                                                     GOVCS=""                                                                                                                                                                             GOVERSION="devel go1.18-986f8ea6b4 Tue Sep 21 00:59:42 2021 +0000"                                                                                                                   GCCGO="gccgo" 
GOAMD64="v1"                                                                                                                                                                         AR="ar"                                                                                                                                                                              
CC="gcc"                                                                                                                                                                             CXX="g++"                                                                                                                                                                            CGO_ENABLED="1"                                                                                                                                                                      GOMOD="/home/me/src/gocrtp/go.mod"                                                                                                                                             CGO_CFLAGS="-g -O2"                                                                                                                                                                  CGO_CPPFLAGS=""                                                                                                                                                                      CGO_CXXFLAGS="-g -O2"                                                                                                                                                                CGO_FFLAGS="-g -O2"                                                                                                                                                                  CGO_LDFLAGS="-g -O2"                                                                                                                                                                 PKG_CONFIG="pkg-config"                                                                                                                                                              GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build1474680903=/tmp/go-build -gno-record-gcc-switches"

What did you do?

I tried to compile this program (crtp.go) with generics:

package main

import "fmt"

type Point struct {
	X, Y int
}

type Rect struct {
	X, Y, W, H int
}

type Elli struct {
	X, Y, W, H int
}

func GetX[P interface { Point | Rect | Elli }] (p P) int {
	return p.X
}

func main() {
	p := Point { 1, 2}
	r := Rect {2, 3, 7, 8}
	e := Elli {4, 5, 9, 10}
	fmt.Printf("X: %d %d %d\n", GetX(p), GetX(r), GetX(e))
}

with tmp/golang-tip/bin/go build

What did you expect to see?

Program compiles, runs and outputs X: 1 2 4

What did you see instead?

./crtp.go:19:11: p.X undefined (type bound for P has no method X)

All three structs in the type bound have an identical X /field/, so I think this is wrong. Of course there is no method but I don't think that matters here. I feel I should be able to use the public field X of p since p can only be one of the three Point, Rect, or Elli.

@beoran
Copy link
Author

beoran commented Sep 21, 2021

@gopherbot, please add label generics

@gopherbot gopherbot added the generics Issue is related to generics label Sep 21, 2021
@dr2chase dr2chase added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Sep 21, 2021
@dr2chase
Copy link
Contributor

@griesemer

Reading "Composite types in constraints" in https://go.googlesource.com/proposal/+/refs/heads/master/design/43651-type-parameters.md I get the impression that this ought to work; the example there says "this doesn't work because the field types for X don't all match" which implies that if the field types did match, as they do here, then it would work.

package main

import "fmt"

type hasX interface {
	struct {
		X, Y int
	} | struct {
		X, Y, H int
	} | struct {
		X, Y, W, H int
	}
}

func GetX[P hasX](p *P) int {
	return p.X
}

func main() {
	p := struct {
		X, Y int
	}{1, 2}
	r := struct {
		X, Y, H int
	}{2, 3, 8}
	e := struct {
		X, Y, W, H int
	}{4, 5, 9, 10}
	fmt.Printf("X: %d %d %d\n", GetX(&p), GetX(&r), GetX(&e))
}

@griesemer
Copy link
Contributor

We don't currently support field accesses of this kind even though the proposal says that this should/could work. We may not support this for Go1.18 as it doesn't seem like an essential feature. There's a trivial work-around that uses a method:

package main

import "fmt"

type Point struct {
	X, Y int
}

func (p Point) GetX() int { return p.X }

type Rect struct {
	X, Y, W, H int
}

func (r Rect) GetX() int { return r.X }

type Elli struct {
	X, Y, W, H int
}

func (e Elli) GetX() int { return e.X }


func GetX[P interface { Point | Rect | Elli; GetX() int }] (p P) int {
	return p.GetX()
}

func main() {
	p := Point { 1, 2}
	r := Rect {2, 3, 7, 8}
	e := Elli {4, 5, 9, 10}
	fmt.Printf("X: %d %d %d\n", GetX(p), GetX(r), GetX(e))
}

Of course, then you don't need generic code in the first place because you could just use dynamic method dispatch. And maybe you should.

@beoran
Copy link
Author

beoran commented Sep 21, 2021

The reason I tried this out is for #48499, wherr the OP directly wants to get fields or pointers to fields from a set of similar structs. In that use case, the overhead of an accessor function and dynamic method dispatch would be not acceptable.

So while it may not be essential, #48499, which is for use with Go language databases, goes to show that it is not academic, but actually would be very useful for existing code, in stead of the feature proposed in that issue. Furthermore it is more consistent and easier to learn to also allow it.

If there is not enough time left to implement this for 1.18, then please consider this for 1.19.

@griesemer griesemer added NeedsDecision Feedback is required from experts, contributors, and/or the community before a change can be made. and removed NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels Nov 17, 2021
@griesemer griesemer added this to the Go1.19 milestone Nov 17, 2021
@Canadadry
Copy link

Open question here.

I feel this is a bit restrictive because we have to explicitly say which struct statisfy the constraint. Like for the interface we may want something that will allow us to tell what is the constraint not who can pass. For example :

package main

import "fmt"

type Point struct {
	X, Y int
}

type Rect struct {
	X, Y, W, H int
}

type Elli struct {
	X, Y, W, H int
}

func GetX[P struct { X int }] (p P) int {
        // here p is known to have only the field X anything more
	return p.X
}

func main() {
	p := Point { 1, 2}
	r := Rect {2, 3, 7, 8}
	e := Elli {4, 5, 9, 10}
	fmt.Printf("X: %d %d %d\n", GetX(p), GetX(r), GetX(e))
}

With this example we can pass any struct that has an int field name X, our function is not coupled with some defined type.

@ghostsquad
Copy link

ghostsquad commented Feb 9, 2022

func GetX[P struct { X int }] (p P) int {

possibly another way to describe this is..

type Xer struct {
   X int
}

func GetX[P ~Xer](p P) int {
    return p.X
}

Just stopping by to say that this is exactly something that I was looking for. Basically, how can I avoid explicitly defining the accessors (via an interface), especially in the case when I fully intend on allowing both Get & Set to occur.

I feel that gymnastics are a totally reasonable expectation when I either want only 1 of Get/Set, OR if the Get/Set needs extra explicit handling. In that case, an interface makes sense. Otherwise, it's just boilerplate.

@davidmdm
Copy link

davidmdm commented Feb 16, 2022

I believe that this kind of generic structural typing would be extremely useful, and necessary for systems that have performance requirements that dynamic dispatch cannot handle.

That being said the above suggestions for type sets with struct members only being structural seems to be against the grain of the type set. If I were to write the follow:

type Named interface {
  string |  struct { Name string }
}

I expect the set of types to be the set that consists of string and the struct literal that Has a Name property of type string.
Not the set of string and any struct with property Name. Adding the ~ operator does help express perhaps that we want structural typing for the struct, but it is inconsistent with what came before in 1.18 where the meaning of the ~ operator is to mean any type's who's underlying type is what follows the operator.

Therefore, and this might be extremely unpopular, would it not make more sense to extend the constraint/interface syntax to allow for struct members?

type Xer interface {
  X int
}

In the same way that type XGetter interface{ GetX() int } represents the set of types that implement the method GetX, Xer would be the set of types that have an accessor X ?

This way we don't need to touch what the tilde operator means, or how to interpret type unions? Otherwise I think we would be taking two steps backwards in regards to eventually supporting sum types (if ever).

EDIT:

To add a more concrete scenario suppose:

type Point1D { X int }
type Point2D { X, Y int }
type Point3D { X, Y, Z int }

// type constraint
type TwoDimensional interface { X, Y int }

// Works for any struct with X and Y of type int, including Point2D and Point3D, etc.
func SomePlanar2DOperation[T TwoDimensional](value T) { ... }

@beoran
Copy link
Author

beoran commented Feb 17, 2022

Many interesting ideas here. But, if the use case of my original examples could work somehow, I don't mind the implementation details.

@davidmdm
Copy link

I think my idea might need to be rewritten as a proposal. As far as whether common fields of a struct union should usable from the function body, I agree. But somebody who knows if it is implementable should comment.

@yulrizka
Copy link

yulrizka commented Feb 18, 2022

I have a real scenario where I'm working on ETL jobs that work nicely with generics.
It removes the necessity to do type checking with switch.

The only thing that prevents it to work is assigning a value of a Field to the struct. And since there is a lot of struct object and each has a lot of properties, adding setter / getter for each struct seems to defeat the whole purpose.

@go101
Copy link

go101 commented Feb 24, 2022

It is strange that if all the structs have the same underlying type, then the code works.

package main

import "fmt"

type Point struct {
	X, Y, W, H int
}

type Rect struct {
	X, Y, W, H int
}

type Elli struct {
	X, Y, W, H int
}

func GetX[P interface { Point | Rect | Elli }] (p P) int {
	return p.X
}

func main() {
	p := Point { 1, 2, 0, 0}
	r := Rect {2, 3, 7, 8}
	e := Elli {4, 5, 9, 10}
	fmt.Printf("X: %d %d %d\n", GetX(p), GetX(r), GetX(e))
}

However, I didn't find the ~ sign in the code.

[Edit], maybe the reason why it works is the constraint has a core type: https://tip.golang.org/ref/spec#Core_types

@go101
Copy link

go101 commented Feb 24, 2022

Promoted fields also work (if their underlying types are the same):

package main

import "fmt"

type Base struct {
	X, Y, W, H int
}

type Point struct {
	Base
}

type Rect struct {
	Base
}

type Elli struct {
	Base
}

func GetX[P interface { Point | Rect | Elli }] (p P) int {
	return p.X // okay
}

func main() {
	p := Point {}
	r := Rect {}
	e := Elli {}
	fmt.Printf("X: %d %d %d\n", GetX(p), GetX(r), GetX(e))
}

But hybrid promoted and direct fields don't work:

package main

import "fmt"

type Point struct {
	X, Y, W, H int
}

type Rect struct {
	Point
}

type Elli struct {
	Point
}

func GetX[P interface { Point | Rect | Elli }] (p P) int {
	return p.X // error: p.X undefined (type P has no field or method X)
}

func main() {
	p := Point { 1, 2, 0, 0}
	r := Rect {}
	e := Elli {}
	fmt.Printf("X: %d %d %d\n", GetX(p), GetX(r), GetX(e))
}

@go101
Copy link

go101 commented Feb 25, 2022

Different but with some similarity:

package main

func bar[F func(int) | func(int)int] (f F) {
	f(1) // invalid operation: cannot call non-function f (variable of type F constrained by func(int)|func(int) int)
}

func main() {}

@blackgreen100
Copy link

blackgreen100 commented Mar 7, 2022

@griesemer

We don't currently support field accesses of this kind even though the proposal says that this should/could work.

It doesn't support method access either.

Playground: https://gotipplay.golang.org/p/aEwSDTXYOlL

Of course method access can be solved by adding the common method to the constraint:

type AB interface {
    *A | *B
    Foo() bool
}

I would like to ask

  1. does method access not work for the same reason as field access?
  2. Is there a more formal explanation for why this doesn't work beside "it wasn't implemented"? It'd be useful when explaining this issue to others. It seems to me that the only relevant quote in the Go 1.18 specifications is "A type constraint [...] controls the operations supported by values of that type parameter." and then adding that selector expressions are not operations. Is there anything else to go by?

EDIT:
Unlike for field access, the type parameter proposal seemed to be much more explicit about this Method sets of constraint elements:

The method set of a union element is the intersection of the method sets of the elements of the union. These rules are implied by the definition of type sets, but they are not needed for understanding the behavior of constraints.

This quote has then been ported to the Go 1.18 specs:

The method set of an interface type is the intersection of the method sets of each type in the interface's type set (the resulting method set is usually just the set of declared methods in the interface).

Emphasis on usually, which doesn't exclude pushing into the interface's method set the intersection of the methods of the type terms. So... on a second thought, method access not working seems a bug (maybe a documentation bug). Am I misreading something?

@go101
Copy link

go101 commented Mar 7, 2022

@blackgreen100 That is #51183

@griesemer
Copy link
Contributor

Is there a more formal explanation for why this doesn't work beside "it wasn't implemented"? It'd be useful when explaining this issue to others. It seems to me that the only relevant quote in the Go 1.18 specifications is "A type constraint [...] controls the operations supported by values of that type parameter." and then adding that selector expressions are not operations. Is there anything else to go by?

The Go 1.18 release is a huge release and we simply haven't had the bandwidth to get everything done.

@ianlancetaylor
Copy link
Contributor

@blackgreen100 Note that the method set limitation is explicitly called out in the release notes (https://go.dev/doc/go1.18#generics).

@mdempsky
Copy link
Member

This isn't going to happen in Go 1.19.

@randall77
Copy link
Contributor

Whatever happens here, will not happen for 1.20. Bumping to 1.21.

@diamondhands0
Copy link

diamondhands0 commented Dec 24, 2022

Just wanted to add one more anecdote to this thread.

We have a few dozen types that all have a common field isDeleted, like you can see here.

A common check we do is if entry != nil && !entry.isDeleted {.

We wanted to simplify this check into a simple IsEntryDeleted(entry), which would be much less error-prone. But we couldn't because the following code requires accessing a member :(

// A generic utility function to check whether an Entry has been deleted from the view.
func IsEntryDeleted[E any](entry *E) bool {
	if entry == nil || entry.isDeleted {
		return true
	}
	return false
}

Of course, we could define an interface with a common getter for every single type that we have. But this would be highly redundant at this point, and the compiler should clearly be smart enough to pick up on this member being common across all types.

I imagine many such use-cases exist where someone wants to do a complicated check on a struct's members, with multiple structs all having the same common members. Libraries that deal with vector math come to mind, where there may be multiple different structs that rely on a common "core" of members that require similar operations to be run on them.

It does also ultimately feel like it would be more "elegant" if [E any] were to allow one to use the E type exactly as if the struct's actual type were substituted in its place.

Anyway, just wanted to say that it would be amazing if struct members could be accessible via generic functions in 1.21 :)

@beoran
Copy link
Author

beoran commented Dec 24, 2022

@griesemer Looking at this thread, there seems to be a wide consensus amongst the people in this thread that implementing this feature is a good idea. However it has been postponed several times now. It would be great if this was accepted, so this could be implemented for 1.21

@ziposcar
Copy link

I found a description in the go1.18 release note https://go.dev/doc/go1.18 : The Go compiler does not support accessing a struct field x.f where x is of type parameter type even if all types in the type parameter's type set have a field f. We may remove this restriction in a future release. I'm looking forward to this future release coming soon.

@crazyoptimist

This comment was marked as duplicate.

@snutiise

This comment was marked as duplicate.

@subtle-byte
Copy link
Contributor

subtle-byte commented Mar 25, 2023

btw Original example can be fixed using embedded fields:

Code
package main

import "fmt"

type xField int

func (x xField) X() int         { return int(x) }
func (x *xField) SetX(newX int) { *x = xField(newX) }

type yField int

func (y yField) Y() int         { return int(y) }
func (y *yField) SetY(newY int) { *y = yField(newY) }

type wField int

func (w wField) W() int         { return int(w) }
func (w *wField) SetW(newW int) { *w = wField(newW) }

type hField int

func (h hField) H() int         { return int(h) }
func (h *hField) SetH(newH int) { *h = hField(newH) }

type Point struct {
	xField
	yField
}

type Rect struct {
	xField
	yField
	wField
	hField
}

type Elli struct {
	xField
	yField
	wField
	hField
}

func GetX[P interface{ X() int }](p P) int {
	return p.X()
}

func main() {
	p := Point{1, 2}
	r := Rect{2, 3, 7, 8}
	r.SetW(8)
	e := Elli{4, 5, 9, 10}
	fmt.Printf("X: %d %d %d\n", GetX(p), GetX(r), GetX(e))
}

It is not very beautiful, but can reduce the amount of code if you need to write getters/setters. But I think the syntax type withX interface {X int} would be much cleaner.

@avivdolev
Copy link

Just another example for a use case and another workaround, this time without an interface.

  • There's a common pattern of chaining many "withX" calls, usually implemented as func (foo Foo) withX(x) Foo
  • When there are many foos that need to make some common code before setting X, e.g. check if a map is initialized before setting a value
  • I think it could be cleaner to use generics here, because interfaces would force the caller to assert the returned value back to the concrete type and won't be chain-able
  • We can't make generic "methods", so passing the generic type as argument instead (func withX(foo Foo, x) Foo)
  • Adding an example workaround for passing "generic" func literals instead of implementing interfaces for getters/setters - not sure regarding performance, but IMHO it feels weird to implement getters/setter in Go for this sole purpose
package options

import "io"

type GetOpts struct {
	url     string
	headers map[string]string
}

type PostOpts struct {
	url     string
	headers map[string]string
	body    io.Reader
}

type opts interface{ GetOpts | PostOpts }

// Desired implementation, for now it's a compile-time error
func WithHeader[Opts opts](o Opts, k, v string) Opts {
	if o.headers == nil {
		o.headers = make(map[string]string)
	}
	o.headers[k] = v
	return o
}

// With current generics, without getter/setter interfaces
// `headers` param is defined as generic only to make it easier to change
func withHeader[Opts opts, headers map[string]string](o Opts, k, v string,
	getter func(Opts) headers,
	setter func(*Opts, headers)) Opts {
	h := getter(o)
	if h == nil {
		h = make(headers)
	}
	h[k] = v
	setter(&o, h)
	return o
}
// Publishing a cleaner API usage without generics, that benefits from the single implementation
// Imagine a real-world use case where the verification is more complicated (e.g. appending a value to existing key instead of overriding)
func (o PostOpts) WithHeader(k, v string) PostOpts {
	return withHeader(o, k, v,
		func(o PostOpts) map[string]string { return o.headers },
		func(o *PostOpts, h map[string]string) { o.headers = h })
}

func (o GetOpts) WithHeader(k, v string) GetOpts {
	return withHeader(o, k, v,
		func(o GetOpts) map[string]string { return o.headers },
		func(o *GetOpts, h map[string]string) { o.headers = h })
}

@griesemer
Copy link
Contributor

On the typechecker side, the 1.21 release focuses on (much better) type inference. Didn't get to this. Moving along to 1.22.

@zaxiom13
Copy link

Completely new to go but here would be my use case for field access on generics:

type User struct {
	name string
	password string
}

type Admin struct {
	name string
	level string
	password string
}

func redactPassword[T User | Admin](someUser T) (T){
	someUser.password = "*****"
	return someUser
}



func main() {
	fmt.Println("hello world?")

	user := User{"apple","banana"};

	fmt.Println(redactPassword(user))

}

lets me use the same function wherever i need to log the details of a struct without exposing one of the fields

@marmiha
Copy link

marmiha commented Sep 22, 2023

Promoted fields also work (if their underlying types are the same):

package main

import "fmt"

type Base struct {
	X, Y, W, H int
}

type Point struct {
	Base
}

type Rect struct {
	Base
}

type Elli struct {
	Base
}

func GetX[P interface { Point | Rect | Elli }] (p P) int {
	return p.X // okay
}

func main() {
	p := Point {}
	r := Rect {}
	e := Elli {}
	fmt.Printf("X: %d %d %d\n", GetX(p), GetX(r), GetX(e))
}

The above example does not work, or am I missing something obvious.

@sirockin
Copy link

sirockin commented Sep 26, 2023

@marmiha: The above doesn't work but you can work around this by creating an interface to get/set the common base fields then using the interface as the constraint for your generic:

package main

import "fmt"

type IBase interface {
	GetX() int
}

type Base struct {
	X, Y, W, H int
}

func (b Base) GetX() int {
	return b.X
}

type Point struct {
	Base
}

type Rect struct {
	Base
}

type Elli struct {
	Base
}

func GetX[T IBase](v T) int {
	return v.GetX()
}

func main() {
	p := Point{}
	r := Rect{}
	e := Elli{}
	fmt.Printf("X: %d %d %d\n", GetX(p), GetX(r), GetX(e))

}

@ianlancetaylor ianlancetaylor changed the title Generics: cannot use common struct fields of a type set. proposal: spec: permit referring to a field shared by all elements of a type set Sep 26, 2023
@ianlancetaylor ianlancetaylor added Proposal and removed NeedsDecision Feedback is required from experts, contributors, and/or the community before a change can be made. labels Sep 26, 2023
@ianlancetaylor ianlancetaylor modified the milestones: Go1.22, Proposal Sep 26, 2023
@piroux

This comment was marked as resolved.

@ianlancetaylor

This comment was marked as resolved.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
generics Issue is related to generics Proposal
Projects
Status: Incoming
Development

No branches or pull requests