Background
With Go 1.18, we are adding a large number of new APIs in go/* packages such as go/ast and go/types to represent the new languages features introduced with generics (see proposals #47781 and #47916). Those proposals describe the new types and functions, but do not comprehensively discuss their semantics nor offer guidance on how to update existing tools. Recently, we've updated many tools in x/tools, such as gopls and vet analyzers, and it would be great to share what we learned from that work with the larger community, to help others update their tools.
There were two notably difficult aspects of updating our tools to support generics:
- We need to handle the new syntax and types while continuing to build at older Go versions.
- For analyzers in particular, we often needed to compute a representation of the structural restrictions on a type parameter's type set. For example, this is necessary in the
printf analyzer to report a diagnostic if any members of the type set to not match the corresponding printf verb. go/types does not yet expose an API for this, because at the time we weren't sure what the correct interface for a type set should be.
Our solutions to both of those problems are contained in the x/tools/internal/typeparams package. This package exposes a transitional API, which at Go 1.18 is just a proxy for the corresponding go/* APIs, and at earlier Go versions is a stub. For example, it exports a typeparams.Union type, which is an alias for go/types.Union at Go 1.18 but a placeholder at Go 1.17. Furthermore the typeparams.StructuralTerms function computes the 'structural' terms of a type parameter, i.e. the normalized list of structural restrictions, after reducing unions and intersections.
Proposal
I propose that we to add a new module to x/exp, golang.org/x/exp/typeparams, which contains the functionality currently exposed by x/tools/internal/typeparams (modulo improved documentation and other superficial improvements). By being publicly importable, this module would provide the same functionality to other tools as x/tools/internal/typeparams has provided x/tools. Furthermore, this module could serve as a home (or entrypoint) for more detailed documentation on how to update tools to support generics. As an initial pass, it could include guidance in its package documentation or in an associated README.
I considered instead suggesting that tools simply copy x/tools/internal/typeparams, but particularly with the typeparams.StructuralTerms API it is possible that we will release bug-fixes, so we should make updating easy.
As for why a new module, and why x/exp, consider the following:
- This package will be small and have no dependencies.
- At some point we may want to tag a final version and delete it. This could possibly occur upon the release of Go 1.21 (two versions after 1.19, which will probably also contain API additions).
Putting this code in a new module in x/exp identifies it as experimental/transient, and means that we won't break builds depending on x/tools for other reasons, if/when it is deleted.
Caveats
If we add x/exp/typeparams, it makes sense to use it to replace the existing usage of x/tools/internal/typeparams. In that case, x/exp/typeparams would not only be required by x/tools, it would be vendored into cmd by way of cmd/vet. It might not be a good precedent to include code from x/exp in the go distribution, though I'm not aware of a practical problem with this. We could alternatively create a new submodule of x/tools. Since this is disallowed, we'll have to keep using x/tools/internal/typeparams, and sync it with x/exp via a script.
Aside: suggestions for improved APIs are certainly welcome, but I'd like to keep the initial focus of the discussion here about whether or not to create such a module, and where to put it. If this proposal is accepted, we can shift discussion to the details.
Background
With Go 1.18, we are adding a large number of new APIs in
go/*packages such asgo/astandgo/typesto represent the new languages features introduced with generics (see proposals #47781 and #47916). Those proposals describe the new types and functions, but do not comprehensively discuss their semantics nor offer guidance on how to update existing tools. Recently, we've updated many tools inx/tools, such asgoplsandvetanalyzers, and it would be great to share what we learned from that work with the larger community, to help others update their tools.There were two notably difficult aspects of updating our tools to support generics:
printfanalyzer to report a diagnostic if any members of the type set to not match the corresponding printf verb.go/typesdoes not yet expose an API for this, because at the time we weren't sure what the correct interface for a type set should be.Our solutions to both of those problems are contained in the
x/tools/internal/typeparamspackage. This package exposes a transitional API, which at Go 1.18 is just a proxy for the correspondinggo/*APIs, and at earlier Go versions is a stub. For example, it exports atypeparams.Uniontype, which is an alias forgo/types.Unionat Go 1.18 but a placeholder at Go 1.17. Furthermore thetypeparams.StructuralTermsfunction computes the 'structural' terms of a type parameter, i.e. the normalized list of structural restrictions, after reducing unions and intersections.Proposal
I propose that we to add a new module to
x/exp,golang.org/x/exp/typeparams, which contains the functionality currently exposed byx/tools/internal/typeparams(modulo improved documentation and other superficial improvements). By being publicly importable, this module would provide the same functionality to other tools asx/tools/internal/typeparamshas providedx/tools. Furthermore, this module could serve as a home (or entrypoint) for more detailed documentation on how to update tools to support generics. As an initial pass, it could include guidance in its package documentation or in an associatedREADME.I considered instead suggesting that tools simply copy
x/tools/internal/typeparams, but particularly with thetypeparams.StructuralTermsAPI it is possible that we will release bug-fixes, so we should make updating easy.As for why a new module, and why
x/exp, consider the following:Putting this code in a new module in
x/expidentifies it as experimental/transient, and means that we won't break builds depending onx/toolsfor other reasons, if/when it is deleted.Caveats
If we add
x/exp/typeparams, it makes sense to use it to replace the existing usage ofx/tools/internal/typeparams. In that case,x/exp/typeparamswould not only be required byx/tools, it would be vendored intocmdby way ofcmd/vet.It might not be a good precedent to include code fromSince this is disallowed, we'll have to keep usingx/expin the go distribution, though I'm not aware of a practical problem with this. We could alternatively create a new submodule ofx/tools.x/tools/internal/typeparams, and sync it withx/expvia a script.Aside: suggestions for improved APIs are certainly welcome, but I'd like to keep the initial focus of the discussion here about whether or not to create such a module, and where to put it. If this proposal is accepted, we can shift discussion to the details.