Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

proposal: spec: generic programming facilities #15292

Open
adg opened this issue Apr 14, 2016 · 816 comments
Open

proposal: spec: generic programming facilities #15292

adg opened this issue Apr 14, 2016 · 816 comments

Comments

@adg
Copy link
Contributor

@adg adg commented Apr 14, 2016

This issue proposes that Go should support some form of generic programming.
It has the Go2 label, since for Go1.x the language is more or less done.

Accompanying this issue is a general generics proposal by @ianlancetaylor that includes four specific flawed proposals of generic programming mechanisms for Go.

The intent is not to add generics to Go at this time, but rather to show people what a complete proposal would look like. We hope this will be of help to anyone proposing similar language changes in the future.

@adg adg added Go2 Proposal labels Apr 14, 2016
@gopherbot
Copy link

@gopherbot gopherbot commented Apr 14, 2016

CL https://golang.org/cl/22057 mentions this issue.

gopherbot pushed a commit to golang/proposal that referenced this issue Apr 14, 2016
This change proposes the notion of generics in Go, and includes four
proposals from years' past that are each flawed in their own ways.
These historical documents are included to show what a complete
generics proposal looks like.

Updates golang/go#15292

Change-Id: Ice1c35af618a043d55ceec54f182d45a4de780e1
Reviewed-on: https://go-review.googlesource.com/22057
Reviewed-by: Ian Lance Taylor <iant@golang.org>
@ianlancetaylor ianlancetaylor added this to the Proposal milestone Apr 14, 2016
@bradfitz
Copy link
Contributor

@bradfitz bradfitz commented Apr 14, 2016

Let me preemptively remind everybody of our https://golang.org/wiki/NoMeToo policy. The emoji party is above.

@egonelbre
Copy link
Contributor

@egonelbre egonelbre commented Apr 14, 2016

There is Summary of Go Generics Discussions, which tries to provide an overview of discussions from different places. It also provides some examples how to solve problems, where you would want to use generics.

@tamird
Copy link
Contributor

@tamird tamird commented Apr 14, 2016

There are two "requirements" in the linked proposal that may complicate the implementation and reduce type safety:

  • Define generic types based on types that are not known until they are instantiated.
  • Do not require an explicit relationship between the definition of a generic type or function and its use. That is, programs should not have to explicitly say type T implements generic G.

These requirements seem to exclude e.g. a system similar to Rust's trait system, where generic types are constrained by trait bounds. Why are these needed?

@sbunce
Copy link

@sbunce sbunce commented Apr 14, 2016

It becomes tempting to build generics into the standard library at a very low level, as in C++ std::basic_string<char, std::char_traits, std::allocator >. This has its benefits—otherwise nobody would do it—but it has wide-ranging and sometimes surprising effects, as in incomprehensible C++ error messages.

The problem in C++ arises from type checking generated code. There needs to be an additional type check before code generation. The C++ concepts proposal enables this by allowing the author of generic code to specify the requirements of a generic type. That way, compilation can fail type checking before code generation and simple error messages can be printed. The problem with C++ generics (without concepts) is that the generic code is the specification of the generic type. That's what creates the incomprehensible error messages.

Generic code should not be the specification of a generic type.

@ianlancetaylor
Copy link
Contributor

@ianlancetaylor ianlancetaylor commented Apr 14, 2016

@tamird It is an essential feature of Go's interface types that you can define a non-interface type T and later define an interface type I such that T implements I. See https://golang.org/doc/faq#implements_interface . It would be inconsistent if Go implemented a form of generics for which a generic type G could only be used with a type T that explicitly said "I can be used to implement G."

I'm not familiar with Rust, but I don't know of any language that requires T to explicitly state that it can be used to implement G. The two requirements you mention do not mean that G can not impose requirements on T, just as I imposes requirements on T. The requirements just mean that G and T can be written independently. That is a highly desirable feature for generics, and I can not imagine abandoning it.

@alex
Copy link
Contributor

@alex alex commented Apr 14, 2016

@ianlancetaylor https://doc.rust-lang.org/book/traits.html explains Rust's traits. While I think they're a good model in general, they would be a bad fit for Go as it exists today.

@ianlancetaylor
Copy link
Contributor

@ianlancetaylor ianlancetaylor commented Apr 14, 2016

@sbunce I also thought that concepts were the answer, and you can see the idea scattered through the various proposals before the last one. But it is discouraging that concepts were originally planned for what became C++11, and it is now 2016, and they are still controversial and not particularly close to being included in the C++ language.

@joho
Copy link

@joho joho commented Apr 14, 2016

Would there be value on the academic literature for any guidance on evaluating approaches?

The only paper I've read on the topic is Do developers benefit from generic types? (paywall sorry, you might google your way to a pdf download) which had the following to say

Consequently, a conservative interpretation of the experiment
is that generic types can be considered as a tradeoff
between the positive documentation characteristics and the
negative extensibility characteristics. The exciting part of
the study is that it showed a situation where the use of a
(stronger) static type system had a negative impact on the
development time while at the same time the expected bene-
fit – the reduction of type error fixing time – did not appear.
We think that such tasks could help in future experiments in
identifying the impact of type systems.

I also see #15295 also references Lightweight, flexible object-oriented generics.

If we were going to lean on academia to guide the decision I think it would be better to do an up front literature review, and probably decide early if we would weigh empirical studies differently from ones relying on proofs.

@benjamingr
Copy link

@benjamingr benjamingr commented Apr 14, 2016

Please see: http://dl.acm.org/citation.cfm?id=2738008 by Barbara Liskov:

The support for generic programming in modern object-oriented programming languages is awkward and lacks desirable expressive power. We introduce an expressive genericity mechanism that adds expressive power and strengthens static checking, while remaining lightweight and simple in common use cases. Like type classes and concepts, the mechanism allows existing types to model type constraints retroactively. For expressive power, we expose models as named constructs that can be defined and selected explicitly to witness constraints; in common uses of genericity, however, types implicitly witness constraints without additional programmer effort.

I think what they did there is pretty cool - I'm sorry if this is the incorrect place to stop but I couldn't find a place to comment in /proposals and I didn't find an appropriate issue here.

@larsth
Copy link

@larsth larsth commented Apr 15, 2016

It could be interesting to have one or more experimental transpilers - a Go generics source code to Go 1.x.y source code compiler.
I mean - too much talk/arguments-for-my-opinion, and no one is writing source code that try to implement some kind of generics for Go.

Just to get knowledge and experience with Go and generics - to see what works and what doesn't work.
If all Go generics solutions aren't really good, then; No generics for Go.

@michael-schaller
Copy link
Contributor

@michael-schaller michael-schaller commented Apr 15, 2016

Can the proposal also include the implications on binary size and memory footprint? I would expect that there will be code duplication for each concrete value type so that compiler optimizations work on them. I hope for a guarantee that there will be no code duplication for concrete pointer types.

@mandolyte
Copy link

@mandolyte mandolyte commented Apr 16, 2016

I offer a Pugh Decision matrix. My criteria include perspicuity impacts (source complexity, size). I also forced ranked the criteria to determine the weights for the criteria. Your own may vary of course. I used "interfaces" as the default alternative and compared this to "copy/paste" generics, template based generics (I had in mind something like how D language works), and something I called runtime instantiation style generics. I'm sure this is a vast over simplification. Nonetheless, it may spark some ideas on how to evaluate choices... this should be a public link to my Google Sheet, here

@benjamingr
Copy link

@benjamingr benjamingr commented Apr 16, 2016

Pinging @yizhouzhang and @andrewcmyers so they can voice their opinions about genus like generics in Go. It sounds like it could be a good match :)

@andrewcmyers
Copy link

@andrewcmyers andrewcmyers commented Apr 16, 2016

The generics design we came up with for Genus has static, modular type checking, does not require predeclaring that types implement some interface, and comes with reasonable performance. I would definitely look at it if you're thinking about generics for Go. It does seem like a good fit from my understanding of Go.

Here is a link to the paper that doesn't require ACM Digital Library access:
http://www.cs.cornell.edu/andru/papers/genus/

The Genus home page is here: http://www.cs.cornell.edu/projects/genus/

We haven't released the compiler publicly yet, but we are planning to do that fairly soon.

Happy to answer any questions people have.

@andrewcmyers
Copy link

@andrewcmyers andrewcmyers commented Apr 16, 2016

In terms of @mandolyte's decision matrix, Genus scores a 17, tied for #1. I would add some more criteria to score, though. For example, modular type checking is important, as others such as @sbunce observed above, but template-based schemes lack it. The technical report for the Genus paper has a much larger table on page 34, comparing various generics designs.

@andrewcmyers
Copy link

@andrewcmyers andrewcmyers commented Apr 17, 2016

I just went through the whole Summary of Go Generics document, which was a helpful summary of previous discussions. The generics mechanism in Genus does not, to my mind, suffer from the problems identified for C++, Java, or C#. Genus generics are reified, unlike in Java, so you can find out types at run time. You can also instantiate on primitive types, and you don't get implicit boxing in the places you really don't want it: arrays of T where T is a primitive. The type system is closest to Haskell and Rust -- actually a bit more powerful, but I think also intuitive. Primitive specialization ala C# is not currently supported in Genus but it could be. In most cases, specialization can be determined at link time, so true run-time code generation would not be required.

@gopherbot
Copy link

@gopherbot gopherbot commented Apr 18, 2016

CL https://golang.org/cl/22163 mentions this issue.

mk0x9 pushed a commit to mk0x9/go that referenced this issue Apr 18, 2016
Updates golang#15292.

Change-Id: I229f66c2a41ae0738225f2ba7a574478f5d6d620
Reviewed-on: https://go-review.googlesource.com/22163
Reviewed-by: Andrew Gerrand <adg@golang.org>
gopherbot pushed a commit that referenced this issue Apr 18, 2016
Updates #15292.

Change-Id: I229f66c2a41ae0738225f2ba7a574478f5d6d620
Reviewed-on: https://go-review.googlesource.com/22163
Reviewed-by: Andrew Gerrand <adg@golang.org>
Reviewed-on: https://go-review.googlesource.com/22166
Reviewed-by: David Symonds <dsymonds@golang.org>
@jba
Copy link
Contributor

@jba jba commented Apr 18, 2016

A way to constrain generic types that doesn't require adding new language concepts: https://docs.google.com/document/d/1rX4huWffJ0y1ZjqEpPrDy-kk-m9zWfatgCluGRBQveQ/edit?usp=sharing.

@jimmyfrasche
Copy link
Member

@jimmyfrasche jimmyfrasche commented Apr 18, 2016

Genus looks really cool and it's clearly an important advancement of the art, but I don't see how it would apply to Go. Does anyone have a sketch of how it would integrate with the Go type system/philosophy?

@sprstnd
Copy link

@sprstnd sprstnd commented Apr 27, 2016

The issue is the go team is stonewalling attempts. The title clearly states the intentions of the go team. And if that wasn't enough to deter all takers, the features demanded of such a broad domain in the proposals by ian make it clear that if you want generics then they don't want you. It is asinine to even attempt dialog with the go team. To those looking for generics in go, I say fracture the language. Begin a new journey- many will follow. I've already seen some great work done in forks. Organize yourselves, rally around a cause

@andrewcmyers
Copy link

@andrewcmyers andrewcmyers commented Apr 27, 2016

If anyone wants to try to work up a generics extension to Go based on the Genus design, we are happy to help. We don't know Go well enough to produce a design that harmonizes with the existing language. I think the first step would be a straw-man design proposal with worked-out examples.

@mandolyte
Copy link

@mandolyte mandolyte commented Apr 28, 2016

@andrewcmyers hoping that @ianlancetaylor will work with you on that. Just having some examples to look at would help a lot.

@ianlancetaylor
Copy link
Contributor

@ianlancetaylor ianlancetaylor commented Jun 26, 2020

@sebastien-rosset We have not considered default types for generic type parameters. The language does not have default values for function arguments, and it's not obvious why generics would be different. In my opinion, the ability to make existing code compatible with a package that adds generics is not a priority. If a package is rewritten to use generics, it's OK to require existing code to change, or to simply introduce the generic code using new names.

@ianlancetaylor
Copy link
Contributor

@ianlancetaylor ianlancetaylor commented Jun 26, 2020

@sighoya

Furthermore, are there any plans to allow for parametrizable finite interface bounds?

I'm sorry, I don't understand the question.

@ianlancetaylor
Copy link
Contributor

@ianlancetaylor ianlancetaylor commented Jun 26, 2020

I'd like to remind people that the blog post (https://blog.golang.org/generics-next-step) suggests that discussion about generics take place on the golang-nuts mailing list, not on the issue tracker. I'll keep reading this issue, but it has nearly 800 comments and is completely unwieldy, besides the other difficulties of the issue tracker such as not having comment threading. Thanks.

@tooolbox
Copy link

@tooolbox tooolbox commented Aug 1, 2020

Feedback: I listened to the most recent Go Time podcast, and I have to say that the explanation from @griesemer on the problem with angle brackets was the first time I really got it, i.e. what does "unbounded lookahead on the parser" actually mean for Go? Thanks very much for the additional detail there.

Also, I'm in favor of square brackets. 😄

@purpleidea
Copy link

@purpleidea purpleidea commented Aug 2, 2020

@ianlancetaylor

the blog post suggests that discussion about generics take place on the golang-nuts mailing list, not on the issue tracker

In a recent blog post [1], @ddevault points out that Google Group (where that mailing list is) requires a Google account. You need one to post, and apparently some groups even require an account to read. I have a Google account, so this isn't an issue for me (and I'm also not saying I agree with everything in that blog post), but I do agree that if we want to have a more just golang community, and if we want to avoid an echo chamber, that it might be better to not have this sort of requirement.

I didn't know this about Google groups, and if there's some exception for golang-nuts, then please accept my apologies and disregard this. For what it's worth, I've learned a lot from reading this thread, and I've also been fairly convinced (after using golang for well over six years) that generics are the wrong approach for the language. Just my personal opinion though, and thank you for bringing us the language which I enjoy as-is quite a lot!

Cheers!

[1] https://drewdevault.com/2020/08/01/pkg-go-dev-sucks.html

@ianlancetaylor
Copy link
Contributor

@ianlancetaylor ianlancetaylor commented Aug 2, 2020

@purpleidea Any Google Group can be used as a mailing list. You can join and participate without having a Google account.

@purpleidea
Copy link

@purpleidea purpleidea commented Aug 2, 2020

@ianlancetaylor

Any Google Group can be used as a mailing list. You can join and participate without having a Google account.

When I go to:

https://groups.google.com/forum/#!forum/golang-nuts

in a private browser window (to hide my google account that I'm logged into), and click "new topic" it redirects me to a google login page. How do I use it without a Google account?

@Merovius
Copy link

@Merovius Merovius commented Aug 2, 2020

@purpleidea By writing an E-Mail to golang-nuts@googlegroups.com. It's a mailing list. Only the web interface needs a Google account. Which seems fair - given that it's a mailing list, you need an E-Mail address and Groups can obviously only send mails from a gmail account.

@OneOfOne
Copy link
Contributor

@OneOfOne OneOfOne commented Aug 2, 2020

I think most people don't understand what a mailing list is.

Anyway you can use any public mailing list mirror as well, for example https://www.mail-archive.com/golang-nuts@googlegroups.com/

@SamWhited
Copy link
Member

@SamWhited SamWhited commented Aug 2, 2020

@Merovius
Copy link

@Merovius Merovius commented Aug 3, 2020

This is not really the place to have this discussion.

@Imperatorn
Copy link

@Imperatorn Imperatorn commented Sep 7, 2020

Any updates on this? 🤔

@TotallyGamerJet
Copy link

@TotallyGamerJet TotallyGamerJet commented Sep 7, 2020

@Imperatorn there have been, they just have not been discussed here. It was decided that square brackets [ ] would be the chosen syntax and the word "type" would not be required when writing generic types/functions. There is also a new alias "any" for the empty interface.

@griesemer
Copy link
Contributor

@griesemer griesemer commented Sep 7, 2020

The latest generics draft design is here.
See also this comment re: discussions on this topic. Thanks.

@tooolbox
Copy link

@tooolbox tooolbox commented Sep 8, 2020

I'd like to remind people that the blog post (https://blog.golang.org/generics-next-step) suggests that discussion about generics take place on the golang-nuts mailing list, not on the issue tracker. I'll keep reading this issue, but it has nearly 800 comments and is completely unwieldy, besides the other difficulties of the issue tracker such as not having comment threading. Thanks.

On this, while I respect that the Go Team would like to move such discussions out of an issue for practical reasons, it does seem like there are a lot of community members on GitHub who are not on golang-nuts. I wonder if GitHub's new Discussions feature would be a good fit? 🤔 It has threading, apparently.

@Merovius
Copy link

@Merovius Merovius commented Sep 8, 2020

@toolbox The argument can also be made in the other direction - there are people who don't have a github account (and refuse to get one). You also don't have to be subscribed to golang-nuts to be able to post and participate there.

@keean
Copy link

@keean keean commented Sep 8, 2020

@Merovius One of the features I really like about GitHub issues is that I can subscribe to notifications for just the issues I am interested in. I am not sure how to do that with Google Groups?

@Merovius
Copy link

@Merovius Merovius commented Sep 8, 2020

I'm sure there are good reasons to prefer one or the other. There certainly can be a discussion about what the preferred forum should be. However, again, I don't think that discussion should be here. This issue is noisy enough as it is.

@tooolbox
Copy link

@tooolbox tooolbox commented Sep 8, 2020

@toolbox The argument can also be made in the other direction - there are people who don't have a github account (and refuse to get one). You also don't have to be subscribed to golang-nuts to be able to post and participate there.

I get what you're saying, and it's true, but you're missing the mark. I'm not saying that golang-nuts users should be told to go to GitHub, (as is happening now in reverse) I'm saying it would be nice for the GitHub users to have a discussion forum.

I'm sure there are good reasons to prefer one or the other. There certainly can be a discussion about what the preferred forum should be. However, again, I don't think that discussion should be here. This issue is noisy enough as it is.

I agree that this is wildly off-topic for this issue, and I apologize for having brought it up, but I do hope you see the irony.

@changkun
Copy link
Contributor

@changkun changkun commented Sep 9, 2020

@keean @Merovius @tooolbox and folks in the future.

FYI: There is an open issue for this type of discussion, see #37469.

@chanced
Copy link

@chanced chanced commented Sep 19, 2020

Hello,

First of all, thank you for Go. The language is absolutely brilliant. One of the most amazing things about Go, for me, has been readability. I'm new to the language so I am still in the early stages of discovery but thus far, it has come across as incredibly clear, crisp, and to the point.

The one bit of feedback that I'd like to present is that from my initial scanning of the generics proposal, [T Constraint] is not easy for me to quickly parse, at least not as easy as a character set designated for generics. I understand that C++ style F<T Constraint> is not feasible due to the nature of go's multi-return paradigm. Any non-ascii characters would be an absolute chore so I'm really thankful you nixed that idea.

Please consider using a character combination. I'm not sure if bitwise operations could be misconstrued or muddy up the parsing waters but F<<T Constraint>> would be nice, in my opinion. Any symbol combination would suffice though. While it may add some initial eye-scanning tax, I think this can easily be remedied with font ligatures like FireCoda and Iosevka. There is not a whole lot that can be done to clearly and easily distinguish the difference between Map[T Constraint] and map[string]T.

I have no doubt that people will train their mind to distinguish between the two applications of [] based on context. I just suspect that it'll steepen the learning curve.

@ianlancetaylor
Copy link
Contributor

@ianlancetaylor ianlancetaylor commented Sep 19, 2020

Thanks for the note. Not to miss the obvious, but map[T1]T2 and Map[T1 Constraint] can be distinguished because the former has no constraint and the latter has a required constraint.

The syntax has been extensively discussed on golang-nuts and I think it's settled. We are happy to hear comments based on actual data such as parsing ambiguities. For comments based on feelings and preferences I think it's time to disagree and commit.

Thanks again.

@chanced
Copy link

@chanced chanced commented Sep 19, 2020

@ianlancetaylor Fair enough. I'm sure you're tired of hearing nitpicks on it :) For what it's worth, I meant easily differentiate scanning wise.

Regardless, I look forward to using it. Thank you.

@Julio-Guerra
Copy link

@Julio-Guerra Julio-Guerra commented Nov 23, 2020

A generic alternative to reflect.MakeFunc would be a huge performance win for Go instrumentation. But I see no way to decompose a function type with the current proposal.

@Merovius
Copy link

@Merovius Merovius commented Nov 23, 2020

@Julio-Guerra I'm not sure what you mean by "decompose a function type". You can, to a degree, parameterize over argument and return types: https://go2goplay.golang.org/p/RwU11S4gC59

package main

import (
	"fmt"
)

func Call[In, Out any](f func(In) Out, v In) Out {
	return f(v)
}

func main() {
	triple := func(i int) int {
		return 3 * i
	}
	fmt.Println(Call(triple, 23))
}

This only works if the number of both is constant though.

@Julio-Guerra
Copy link

@Julio-Guerra Julio-Guerra commented Nov 24, 2020

@Julio-Guerra I'm not sure what you mean by "decompose a function type". You can, to a degree, parameterize over argument and return types: https://go2goplay.golang.org/p/RwU11S4gC59

Indeed I am referring to what you did, but generalized to any function parameter and return type list (similarly to the array of parameter and return types of reflect.MakeFunc). That would allow to have generalized function wrappers (instead of using tooled code generation).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
You can’t perform that action at this time.