how to update APIs for generics #48287
Replies: 45 comments 211 replies
-
Using new major versions for stdlib packages, e.g. |
Beta Was this translation helpful? Give feedback.
-
Another approach is to observe that the language supports type aliases to support transitions. So let's use type aliases. type Pool[T any] ...
type Pool = Pool[interface{}] The rules here would be:
This alias rule does not work for functions. We can either say there is no transition for functions, or we can introduce another rule. A function may be defined both with and without type parameters. A reference to the function (not a call) with no type arguments is permitted, and gets the version without type parameters. A call of the function with no type arguments gets the version with type parameters and does type inference as usual. If type inference fails, the call is instead made to the version without type parameters (and may fail if the arguments type are not assignable). That gives us func Min[T constraints.Ordered](a, b T) T { ... }
func Min(a, b float64) float64 { return Min[float64](a, b) } |
Beta Was this translation helpful? Give feedback.
-
The idea clearly exists already, as it was mentioned in this thread but I think it's worth its own suggestion as I think it aligns pretty nicely with what we're asking package maintainers to do in general: Don't introduce generics in most of these libraries. /v2 them:
The normal packages should be implementable in terms of the generic version, so if it doesn't compromise performance we could do the |
Beta Was this translation helpful? Give feedback.
-
Personally, I don't like the asymmetry this would create between type-parameters and regular parameters. Default values for arguments is something that has often been asked about, but we never did it. To me, the case for default type arguments isn't really stronger than for default regular arguments though. I also don't like introducing a new language feature just for a one-time migration. If the motivator for this feature is just the migration of pre-generics APIs to post-generics APIs, it will become de-facto obsolete in a year or so. I don't like that idea. |
Beta Was this translation helpful? Give feedback.
-
Would definitely be confusing. Especially if you converted from the former to the latter while debugging something. |
Beta Was this translation helpful? Give feedback.
-
I support this suggestion, particularly as I see it as having the potential to be generally useful into the future given a minor tweak to the rules. I'd question the need for the parentheses. I understand that they're there to suggest the "maybe" aspect of the default but I don't think they pull their weight, and there's considerable precedent from other languages (e.g. Rust's default type parameters, Python's default function parameters) that they could be omitted without much confusion. That is, I think it would be fine if the examples were written thus:
|
Beta Was this translation helpful? Give feedback.
-
Another thought:
This gets the new generic-based API, old sort lives where it does now for historical reasons, and if for some reason you need both, you can specify a prefix. I like this slightly better than On the other hand, this works poorly for cases, like I'm also unsure what the right type restrictions would be for |
Beta Was this translation helpful? Give feedback.
-
I think this is a good time to declare Go 2, and let Go1 Rest In Peace.It served its purpose well but it is starting to show its age and hiding all the worts is getting hard. We could and should have done this awhile ago eg when modules were introduced which obsoleted lots of the premodule packages books and tools. I don’t see much benefit of keeping the same major version number if the style and idioms change so significantly. If planned correctly the drawbacks of some older code not compiling due to some needed minor changes is far less than the consequences of having to learn maintain and debug complex code. Tools can deal with minor code breakage. Humans pay the price for complexity. |
Beta Was this translation helpful? Give feedback.
-
I went through the list of packages in the standard library to see what I think should be done with them. Going through the list, it seems pretty clear to me that you're either going to add some /v2 packages or be stuck with really ugly docs forever. Changing how aliasing works or adding default types can only help in limited situations. Regexp and math/big in particular really need v2 to be viable. However, I think a lot of them would be fine with just adding a generic Of type and some back references. It varies a quite bit depending on the specifics of what needs to be added or removed and why. container/heap
container/list
container/ring
regex
sort
sync
sync/atomic
testing
math
math/big
strconv
Packages which could benefit from a v2 cleanup unrelated to generics:
|
Beta Was this translation helpful? Give feedback.
-
I think most comments here are focusing on the standard library, but the convention for other packages is probably the question which is going to tease out the most interesting problems. Per the OP:
For "everyone else", I'm thinking there's a handful of cases:
It's also reasonable that this problem will still occur in various forms well past 1.18's release. Go package authors should feel safe not making every function parameter generic just in case they might want to in the future. This rules out any magic related to package version. Some sort of syntax for defining a default type when it cannot be inferred seems to be the best solution here. If there is a mechanism for using defaults, it should likely only be used for transition (like type aliasing today). I propose nice way to do this would be by using I think then the question becomes syntax, but also how to encourage users to run |
Beta Was this translation helpful? Give feedback.
-
I would rather prefer to keep track of math.MinOF() vs math.Min() as opposed to math.Min() vs v2/math.Min()
Adding a default notation in the declaration signature makes an already too loaded syntax even harder to glance. A solution along the lines of using aliases for handling the defaults for the transition, as suggested by |
Beta Was this translation helpful? Give feedback.
-
These are straightforward, but I think there's a lot of not-straightforward stuff, too. For example, package net is oriented around the net.Conn interface, which reasonably successfully models the different underlying connection types with a behavioral contract. But it's always been a bit of an impedance mismatch — anytime you see interface upgrades, you know something's not quite right. Had generics been available on day one, I'm sure package net would have a much different API surface area and implementation. And so I don't think you can feasibly update it in-place, type-by-type. I think you're gonna need a second revision. And I think, actually, most APIs are more like package net than package math. Is this discussion about only the simpler cases, or is it about generalizable patterns? If it's the former, cool; if it's the latter, I think the only viable option is e.g. |
Beta Was this translation helpful? Give feedback.
-
The new generic variations of these packages can live under |
Beta Was this translation helpful? Give feedback.
-
I think the main problem with, and simultaneously the main advantage of, creating a v2 for this is that it'll open the door to v2s in general, and proposals will start flowing in for more than just generic packages. For instance, any package that added a lot of WithContext methods could get a v2 to make the non-contextual ones the harder thing to type in, I'd like a v2 of every marshaling package to take a Context for the decoder (another proposal around here), there's any number of proposals than are going to get fresh bursts of activity if there's a realistic possibility of a v2. Arguably this is more a "Go 2.0" sort of thing than generics themselves. At the moment I'm not advocating for or against this, just making an observation that creating v2s will inevitably snowball. |
Beta Was this translation helpful? Give feedback.
-
I'm going to go out on a limb and suggest that we use the Constructor functions for generic containers get the
I think If we do the above, what are the remaining problematic names? |
Beta Was this translation helpful? Give feedback.
-
Also, I'm not sure this is the best idea, but one option is to claim the same exemption that map and slice do, and put hard-coded exceptions into the compiler for the standard library, and make all current standard library APIs generic in-place. Along with the fact that Go has sort of reserved the right to do this for itself with map & slice, there is justification in treating the standard library a bit differently because it, uniquely, has the high bar of backwards compatibility that no other Go code has. So, arguably, the feature used to upgrade it doesn't have to necessarily work or be available for anything else. This comes with non-zero levels of confusion, but I'm not sure it'll be that high. It could be documented with a link to some central explanation in each instance with too much doc overhead. |
Beta Was this translation helpful? Give feedback.
-
I wonder how much compilation time it would take if the function signatures would also be distinguished by their parameter lists. Preferring a generic implementation over the interface one. |
Beta Was this translation helpful? Give feedback.
-
Why can’t you just make Min generic today without any default type arguments? What existing code would that break? Shouldn’t all existing usages be inferrable as float64? |
Beta Was this translation helpful? Give feedback.
-
I suspect it is far too late to introduce this comment, but it may be useful to record it for later when we look back at this. Long and short: the reason we are talking about default types here is that we aren't doing type inference. In languages where type inference is adopted, none of the issues being raised here exist because the type parameters are auto-populated by inference. This can definitely be done in a go-like language, as we showed in BitC. I definitely wouldn't adopt everything we tried - if only because the result wouldn't be go. But the heart of the problem here is that type variables are getting instantiated explicitly. Circling back to an early example in this thread:
It appears to me that this is a type error, because the first invocation of
If the promotion is required to be explicit, there is neither ambiguity (to the compiler) nor surprise (to the user). We went to significant lengths in BitC to use type classes to arrange for this type of implicit promotion on binary operators. At the time, I was pushing the type classes idea to see what kinds of things could be expressed with them. We ended up with something similar to go's untyped constants, but we got there using type classes. The problem, ultimately, is that type class constraints accumulate in much the way that explicitly declared exception types accumulate: incomprehensibly. Jonathan Shapiro (the BitC architect). |
Beta Was this translation helpful? Give feedback.
-
If we had a Go equivalent of the absl::StatusOr, let's call it func ParseNumber[T constraints.Number](s string, base int) ErrorOr[T] The typical
But only states 1 and 2 are sensible in most cases. The Another advantage of func WithFunc[T any](myfunc func()T) T
WithFunc(func()int { ... })
WithFunc(func()ErrorOr[int] { ... }) instead of having 2 versions of func WithFunc[T any](myfunc func()T) T
func WithFuncErr[T any](myfunc func()(T, error)) (T, error) This would make it easier to write generic libraries and APIs. |
Beta Was this translation helpful? Give feedback.
-
How about just break, since there is no promise to keep compatible of stdlib. It's simple and for users, the changes are small. And of course, some people will develop tools to migrate them automatically. |
Beta Was this translation helpful? Give feedback.
-
We could teach As an example:
Benefits:
|
Beta Was this translation helpful? Give feedback.
-
I may be “under-thinking” the problem, it feels like too much work on the language to come up with what feels like a workaround. The use cases I can think of feel like they are handled with existing constructs. This is my use case summary, what am I missing?: |
Beta Was this translation helpful? Give feedback.
-
how about suffix 'X'? |
Beta Was this translation helpful? Give feedback.
-
At the risk of adding one more proposal to this already lengthy discussion:
This approach is intended to keep the standard library as similar as possible to what public packages need to do, and step 3 is the only new piece required for this to work. The warts of this approach are also warts any open-source library author will face, and the (optional) additional changes below will help with community migrations as well. Each of these would need to go through its own proposal process.
|
Beta Was this translation helpful? Give feedback.
-
Thinking further about constraint accumulation, it will be interesting to see how type constraints evolve in Go. At some point I suspect the language will need to consider intersection and union types to deal with them comprehensively. |
Beta Was this translation helpful? Give feedback.
-
A possible awful idea for rejection: implementation-specific suffixes akin to This builds on the observation:
|
Beta Was this translation helpful? Give feedback.
-
@rsc:
I don't think we'd want to make math.Abs, Min, or Max generic anyway, since they take into account NaN, infinities, and signed zeroes. Are there any other examples of wanting to abstract a non-interface type to a type variable in the stdlib? I think we should follow Java's approach, with raw types. The Go equivalent would be:
So We would get:
Due to the basic interface constraint limitation, we wouldn't be able to convert, say, func Max(x, y int64) int64 {
if x > y {
return x
}
return y
}
var max func(int64, int64) int64 = Max to func Max[T constraints.Ordered](x, y T) T {
if x > y {
return x
}
return y
}
var max func(?, ?) ? = Max because there's nothing we can put for For the cases where we'd need to add a second, generic, variant of a function, I suggest appending the constraint to the name, rather than "Of". E.g. |
Beta Was this translation helpful? Give feedback.
-
I've been holding off commenting here, because from the discussion it doesn't sound like this is a very popular choice, but since nobody mentioned it this way yet and this issue has resurfaced, I figured I would. One obvious thing for me to consider when it's about a single or a few symbols that needs a new version is to version those symbols. That's how I've been solving the Setup function in one of my packages, without generics being involved. I renamed it to SetupV2, changing the signature, then made a new Setup with the old signature calling the new version, also indicating that's what it does in the documentation. I think that's concise and delivers the message, while still being backwards compatible. It's possible to extend it slightly based on some comments in this thread. For example, one could make sure to also create a SetupV1 when there need for SetupV2 arise. SetupV1 and the old Setup would be identical, both calling SetupV2. One could then start thinking about go.mod maneuvering to change the signature of Setup to that of SetupV2 instead of SetupV1 based on the version number in there, but it would be a later problem. Or perhaps to drop the versionless one altogether. |
Beta Was this translation helpful? Give feedback.
-
@ianlancetaylor, @griesemer, and I are wondering about different possible plans for updating APIs that would have used generics but currently use interface{}, and we wanted to cast a wider net for ideas.
There exist types and functions that clearly would use generics if we wrote them today. For example:
There are certainly more of these, both in Go repos and everyone else's code.
The question is what should be the established convention for updating them.
One suggestion is to adopt an "Of" suffix, as in PoolOf[T], MinOf[T], and so on.
For types, which require the parameter list, that kind of works.
For functions, which will often infer it, the result is strange:
there is no obvious difference between math.MinOf(1, 2) and math.Min(1, 2).
Any new, from-scratch, post-generics API would presumably not include the Of - Tree[K,V], not TreeOf[K,V] -
so the "Of" in these names would be a persistent awkward reminder of our pre-generics past.
Another possibility is to adopt some kind of notation for default type parameters.
For example, suppose you could write
The (= ...) sets the default for a type parameter.
The rule for types could be that the bracketed type parameter list may be omitted when all parameters have defaults,
so saying List would be equivalent to List[interface{}], which, if we are careful, would be identical to the current code,
making the introduction of a generic List as list.List not a backwards-incompatible change.
The rule for functions could be that when parameters have defaults,
those defaults are applied just before the application of default constant types in the type inference algorithm.
That way, Min(1, 2) stays a float64, while Min(i, j) for i, j of type int, infers int instead.
The downside of this is a little bit more complexity in the language spec,
while the upside is potentially smoother migration for users.
Like the "Of" suffix, an API with type defaults would be a persistent awkward reminder of our pre-generics past,
but it would remind mainly the author of the code rather than all the users.
And users would not need to remember which APIs need an "Of" suffix added.
Are there other options we haven't considered?
Are there arguments for or against these options that haven't been mentioned?
Thanks very much.
Beta Was this translation helpful? Give feedback.
All reactions