proposal: decide policy for sub-repositories #17244

Open
adg opened this Issue Sep 27, 2016 · 81 comments

Comments

Projects
None yet
@adg
Contributor

adg commented Sep 27, 2016

[this is not a proposal yet, just a problem statement]

There are several proposals to add new packages to the x/foo "sub-repositories". (See: #16353, #13432, #12332, #16257, #15985, #15406, and more.) The proposals are all different, but they circle around the same issue: what should and should not be in a sub-repository.

Currently, the sub-repositories contain x different things:

  1. Packages and tools that are part of a loose set of Go tools (godoc, goimports, benchcmp, x/debug, etc)
  2. Packages and tools that support the Go web services (godoc, present, x/playground, x/tour, etc)
  3. Tools that support Go project development (x/build, x/benchmarks, x/review, etc).
  4. Packages that augment the standard library (x/text, x/image, x/crypto, x/sys, etc).
  5. Other projects worked on by Go team members at Google (x/exp, x/mobile, etc).

To me, it's clear that 1-2 belong in the sub-repositories, and should be an official part of the Go project.

For 4-5, this code only lives in the sub-repositories either for historical reasons or mere convenience. The Go contributors are accustomed to the process (tools and reviews) of writing code in that style, so the sub-repos were just the obvious place to go.

(3 is a fringe case, as the tools could be worked on elsewhere, but the people working on them work on Go full time and work on that code solely to support the project, so I am inclined to ignore this code for the purposes of this issue.)

As the project grows, it attracts people who want to contribute new packages to these sub-repositories (see the issues cited at the beginning of this issue). The reasons for inclusion include:

  1. To provide an official package for doing X.
  2. To provide a supported package for doing X.
  3. To contribute to the Go project.
  4. To use the project's development processes (gerrit, etc).
  5. To have their code reviewed by other Go contributors and to benefit from their expertise.
  6. To use the same consistent license (with CLA) that the project uses.
  7. To provide a single package for a single purpose, and avoid duplicate effort.
  8. To provide an official package for interoperating about (but not necessarily doing) X.
  9. To act as a dependency for something else under x/.

[What are the other reasons? I'd like to enumerate them all here.]


My opinion begins here:

None of the above reasons are good arguments to add new packages to the sub-repositories. Taking each in turn:

  1. The "official" packages, such as they are, are generally of high quality because a small group of dedicated contributors have invested constant energy in them. But there are other high quality packages elsewhere in the Go ecosystem (many of higher quality than some of those in the sub-repos), it's just harder for people to find them and they don't have "prestigious" import paths. Rather than moving more stuff into the Go project, I'd think we should address this issue by making it easier to find and recognize high quality packages. (One way to do this is to improve godoc.org, but there are many other ways.)
  2. Putting something in the sub-repositories does not imply support. People are not more or less inclined to work on a package just because it's in a sub-repository. The current set of Go contributors are stretched pretty thin as it is. I think that by moving more stuff into the sub-repositories we set expectations of support where there is none. That's bad for our users.
  3. Moving more code into the project just for the sake of contributing is not a net win. There's plenty of work to be done on the code that is already part of the project (and the many, many open issues).
  4. If people want to use our processes, I think that a better approach is to find a way to use those processes outside of the project. That's a solution that scales much better.

[see discussion below for other points]

In general, I'd like to make the Go project smaller rather than bigger. (In the same sense that we would remove stuff from the standard library, if we could.)

As I said, this is just my opinion and perspective. I have created this issue to gather feedback from other [potential] contributors.

Personal opinion ends.


We need to set a policy for what belongs in the sub-repositories, and in doing so more clearly define the boundaries of the Go project.

@adg adg added the Proposal label Sep 27, 2016

@adg adg added this to the Proposal milestone Sep 27, 2016

@adg adg self-assigned this Sep 27, 2016

@bradfitz

This comment has been minimized.

Show comment
Hide comment
@bradfitz

bradfitz Sep 27, 2016

Member

Nice summary. Thanks for writing this up. I agree with your opinion too.

Member

bradfitz commented Sep 27, 2016

Nice summary. Thanks for writing this up. I agree with your opinion too.

@quentinmit

This comment has been minimized.

Show comment
Hide comment
@quentinmit

quentinmit Sep 27, 2016

Contributor

I have a very different opinion. I think there is significant value in having a set of "semi-blessed" packages, and random packages that can be found (more or less easily) on godoc.org do not meet that bar. I think one of the big reasons that languages like Python have taken off is that they are "batteries included"; their ~standard libraries include a wide swath of functionality, such that the average program can be written without adding any third-party libraries. If anything, I think more should be included than we have now.

I think the x repos today have already improved on the "batteries included" model, by making it clear that "these are good packages, but they're not great packages, and we don't make as many promises about them". I would actually like to see a future where we include a snapshot of the x repos in release tarballs, attaching the same disclaimer.

I see your proposal as directly advocating for fewer packages in x. I disagree that that is a worthy goal.

That said, I don't think that #2 and #3 necessarily have to live in the same place as #1 and #4. I would be happy to separate the "random part of the Go project" repos from the "semi-blessed packages" repos.

Contributor

quentinmit commented Sep 27, 2016

I have a very different opinion. I think there is significant value in having a set of "semi-blessed" packages, and random packages that can be found (more or less easily) on godoc.org do not meet that bar. I think one of the big reasons that languages like Python have taken off is that they are "batteries included"; their ~standard libraries include a wide swath of functionality, such that the average program can be written without adding any third-party libraries. If anything, I think more should be included than we have now.

I think the x repos today have already improved on the "batteries included" model, by making it clear that "these are good packages, but they're not great packages, and we don't make as many promises about them". I would actually like to see a future where we include a snapshot of the x repos in release tarballs, attaching the same disclaimer.

I see your proposal as directly advocating for fewer packages in x. I disagree that that is a worthy goal.

That said, I don't think that #2 and #3 necessarily have to live in the same place as #1 and #4. I would be happy to separate the "random part of the Go project" repos from the "semi-blessed packages" repos.

@ianlancetaylor

This comment has been minimized.

Show comment
Hide comment
@ianlancetaylor

ianlancetaylor Sep 27, 2016

Contributor

@quentinmit I think we absolutely do need a way to tell people "these are good packages" and I think that is what @adg was alluding to with his point 4 above. The question is what that mechanism should be.

The disadvantage I see with the current golang.org/x system is that these packages are effectively owned by the core Go developers. That isn't scalable or even desirable. If we can solve that problem within the golang.org/x name space, then, fine. But how? And who controls the golang.org/x name space?

I think it would be better to have a scalable mechanism for identifying and maintaining good packages, and I don't see why that should be, or even can be, associated with golang.org/x.

Contributor

ianlancetaylor commented Sep 27, 2016

@quentinmit I think we absolutely do need a way to tell people "these are good packages" and I think that is what @adg was alluding to with his point 4 above. The question is what that mechanism should be.

The disadvantage I see with the current golang.org/x system is that these packages are effectively owned by the core Go developers. That isn't scalable or even desirable. If we can solve that problem within the golang.org/x name space, then, fine. But how? And who controls the golang.org/x name space?

I think it would be better to have a scalable mechanism for identifying and maintaining good packages, and I don't see why that should be, or even can be, associated with golang.org/x.

@griesemer

This comment has been minimized.

Show comment
Hide comment
@griesemer

griesemer Sep 27, 2016

Contributor

Great summary of the issue. What @bradfitz said.

Contributor

griesemer commented Sep 27, 2016

Great summary of the issue. What @bradfitz said.

@cespare

This comment has been minimized.

Show comment
Hide comment
@cespare

cespare Sep 27, 2016

Contributor

As a consumer of many third-party packages, I think I agree with @quentinmit (or at least with @ianlancetaylor). If I need a package for $task and there is an /x package for $task, I've generally found that while it might be incomplete, it's usually of high quality (the scope and style matches the standard library; the test coverage is quite good; the documentation is thorough and concise). If I have to go browsing through $task packages on godoc.org and pick the best one, it's often of such poor quality that I either do an extensive cleanup/refactoring/testing/docs pass before bringing it into my repo or else I choose to write my own package instead.

@adg said:

I'd think we should address this issue by making it easier to find and recognize high quality packages. (One way to do this is to improve godoc.org, but there are many other ways.)

but I'm skeptical that a process outside the Go team will produce a quality bar that is usefully high for me.

I'd like to see [high-quality package identification mechanism] exist and be proven to work well before any trimming down of /x.

Contributor

cespare commented Sep 27, 2016

As a consumer of many third-party packages, I think I agree with @quentinmit (or at least with @ianlancetaylor). If I need a package for $task and there is an /x package for $task, I've generally found that while it might be incomplete, it's usually of high quality (the scope and style matches the standard library; the test coverage is quite good; the documentation is thorough and concise). If I have to go browsing through $task packages on godoc.org and pick the best one, it's often of such poor quality that I either do an extensive cleanup/refactoring/testing/docs pass before bringing it into my repo or else I choose to write my own package instead.

@adg said:

I'd think we should address this issue by making it easier to find and recognize high quality packages. (One way to do this is to improve godoc.org, but there are many other ways.)

but I'm skeptical that a process outside the Go team will produce a quality bar that is usefully high for me.

I'd like to see [high-quality package identification mechanism] exist and be proven to work well before any trimming down of /x.

@adg

This comment has been minimized.

Show comment
Hide comment
@adg

adg Sep 27, 2016

Contributor

If I have to go browsing through $task packages on godoc.org and pick the best one, it's often of such poor quality that I either do an extensive cleanup/refactoring/testing/docs pass before bringing it into my repo or else I choose to write my own package instead.

I'm advocating that we fix this problem.

I'm skeptical that a process outside the Go team will produce a quality bar that is usefully high for me.

The "Go team," such as it is, is only a handful of people, and they do not have the cycles to review any new packages. If you want more packages in x/ then they must be written, reviewed, and maintained by other people. That being the case, there's no reason why they should be in x/. They're not going to be any better or worse for it.

Contributor

adg commented Sep 27, 2016

If I have to go browsing through $task packages on godoc.org and pick the best one, it's often of such poor quality that I either do an extensive cleanup/refactoring/testing/docs pass before bringing it into my repo or else I choose to write my own package instead.

I'm advocating that we fix this problem.

I'm skeptical that a process outside the Go team will produce a quality bar that is usefully high for me.

The "Go team," such as it is, is only a handful of people, and they do not have the cycles to review any new packages. If you want more packages in x/ then they must be written, reviewed, and maintained by other people. That being the case, there's no reason why they should be in x/. They're not going to be any better or worse for it.

@adg

This comment has been minimized.

Show comment
Hide comment
@adg

adg Sep 27, 2016

Contributor

@quentinmit said:

I think one of the big reasons that languages like Python have taken off is that they are "batteries included"; their ~standard libraries include a wide swath of functionality, such that the average program can be written without adding any third-party libraries. If anything, I think more should be included than we have now.

Both Python and Go have suffered from having large standard libraries, too. In some cases, the standard libraries have become out of date and superseded by other packages in the ecosystem (or in the case of Python's urllib, by urllib2 in the same standard library!). Then we're left with prominent packages that are not suitable for common use. That's a net loss for our users.

If I understand your sentiment, you're advocating for including (a lot) more packages in the x/ repositories, as an extension to the standard library. Producing our current standard library (not the sub-repositories) was a massive concerted effort over many years, by people who are mostly domain experts in the functionality of those packages. I have two main questions:

  1. Who is going to do this work? How do you propose we maintain the same quality level across many more packages that our well outside our areas of expertise?
  2. Why does it need to go in x/? Why should we be the arbiters of which packages are prominent? Aren't we short-changing the ecosystem by neglecting third-party package support?
Contributor

adg commented Sep 27, 2016

@quentinmit said:

I think one of the big reasons that languages like Python have taken off is that they are "batteries included"; their ~standard libraries include a wide swath of functionality, such that the average program can be written without adding any third-party libraries. If anything, I think more should be included than we have now.

Both Python and Go have suffered from having large standard libraries, too. In some cases, the standard libraries have become out of date and superseded by other packages in the ecosystem (or in the case of Python's urllib, by urllib2 in the same standard library!). Then we're left with prominent packages that are not suitable for common use. That's a net loss for our users.

If I understand your sentiment, you're advocating for including (a lot) more packages in the x/ repositories, as an extension to the standard library. Producing our current standard library (not the sub-repositories) was a massive concerted effort over many years, by people who are mostly domain experts in the functionality of those packages. I have two main questions:

  1. Who is going to do this work? How do you propose we maintain the same quality level across many more packages that our well outside our areas of expertise?
  2. Why does it need to go in x/? Why should we be the arbiters of which packages are prominent? Aren't we short-changing the ecosystem by neglecting third-party package support?
@rakyll

This comment has been minimized.

Show comment
Hide comment
@rakyll

rakyll Sep 27, 2016

Member

From my perspective, being able to work under an x allows me to reach out to the go approvers and this is what I care when I am designing ecosystem-wide core libraries for a niche topic. I don't think it is possible once the experimentation is not considered a part of the Go project.

Member

rakyll commented Sep 27, 2016

From my perspective, being able to work under an x allows me to reach out to the go approvers and this is what I care when I am designing ecosystem-wide core libraries for a niche topic. I don't think it is possible once the experimentation is not considered a part of the Go project.

@adg

This comment has been minimized.

Show comment
Hide comment
@adg

adg Sep 27, 2016

Contributor

To expand a little on my goals here: by defining what does or does not belong in x/, I hope to remove bureaucracy.

Today, if someone wants to put something in /x/exp they need to ask permission. People shouldn't have to ask for permission to experiment; that has a chilling effect on experimentation.

Similarly, if something is to be included in an /x sub-repo, they need to ask permission. Why should they? Shouldn't they be able to put their high quality code in a repo somewhere else, and have it be regarded just as highly as anything in /x? Code should be judged on its own merits, not by its location.

Contributor

adg commented Sep 27, 2016

To expand a little on my goals here: by defining what does or does not belong in x/, I hope to remove bureaucracy.

Today, if someone wants to put something in /x/exp they need to ask permission. People shouldn't have to ask for permission to experiment; that has a chilling effect on experimentation.

Similarly, if something is to be included in an /x sub-repo, they need to ask permission. Why should they? Shouldn't they be able to put their high quality code in a repo somewhere else, and have it be regarded just as highly as anything in /x? Code should be judged on its own merits, not by its location.

@quentinmit

This comment has been minimized.

Show comment
Hide comment
@quentinmit

quentinmit Sep 27, 2016

Contributor

I'm not sure I think I would agree with "a lot", but yeah, I think x/ should perhaps be twice the number of packages it is now.

  1. Who is going to do this work? How do you propose we maintain the same quality level across many more packages that our well outside our areas of expertise?

Go project members will do the work, just as they do today. The people who are proposing these new packages are working in their area of expertise. But I also don't think it's "outside" our areas of expertise; our area of expertise is designing idiomatic Go APIs, and that can be equally applied to something like video compression as it can to XML or JSON marshalling.

  1. Why does it need to go in x/? Why should we be the arbiters of which packages are prominent? Aren't we short-changing the ecosystem by neglecting third-party package support?

That's a hard question. For me, I have a soft definition of "low-level" support packages that I think deserve to be centralized. To use an example I am personally familiar with, I think video and audio formats should be in x/video or x/media, ala the current image support we have in x/image. I think a streaming server built on top of those libraries, or a GIMP clone, does not belong in x/ because that is not something foundational.

I do think making high quality third-party packages easier to find solves a large part of this problem, but we are not there yet. And even if we do make them easy to find, packages that are separate from Go still have more adoption problems we can't possibly solve: for example, they may be under various licenses, and it may be cumbersome to import into a commercial codebase. That may be appropriate for a library to interface with a Siemens PLC from the 1990s, but I don't think it's appropriate for OpenGL.

Contributor

quentinmit commented Sep 27, 2016

I'm not sure I think I would agree with "a lot", but yeah, I think x/ should perhaps be twice the number of packages it is now.

  1. Who is going to do this work? How do you propose we maintain the same quality level across many more packages that our well outside our areas of expertise?

Go project members will do the work, just as they do today. The people who are proposing these new packages are working in their area of expertise. But I also don't think it's "outside" our areas of expertise; our area of expertise is designing idiomatic Go APIs, and that can be equally applied to something like video compression as it can to XML or JSON marshalling.

  1. Why does it need to go in x/? Why should we be the arbiters of which packages are prominent? Aren't we short-changing the ecosystem by neglecting third-party package support?

That's a hard question. For me, I have a soft definition of "low-level" support packages that I think deserve to be centralized. To use an example I am personally familiar with, I think video and audio formats should be in x/video or x/media, ala the current image support we have in x/image. I think a streaming server built on top of those libraries, or a GIMP clone, does not belong in x/ because that is not something foundational.

I do think making high quality third-party packages easier to find solves a large part of this problem, but we are not there yet. And even if we do make them easy to find, packages that are separate from Go still have more adoption problems we can't possibly solve: for example, they may be under various licenses, and it may be cumbersome to import into a commercial codebase. That may be appropriate for a library to interface with a Siemens PLC from the 1990s, but I don't think it's appropriate for OpenGL.

@adg

This comment has been minimized.

Show comment
Hide comment
@adg

adg Sep 27, 2016

Contributor

Go project members will do the work, just as they do today.

The reality is that there are hundreds of open CLs (some pending for more than a year) and thousands of open issues. The work is not getting done. The present structure does not scale. To encourage more development that we cannot support is irresponsible and unfair to the contributors.

I do think making high quality third-party packages easier to find solves a large part of this problem, but we are not there yet.

My argument is that we will never get there if we use the sub-repos as a crutch.

Contributor

adg commented Sep 27, 2016

Go project members will do the work, just as they do today.

The reality is that there are hundreds of open CLs (some pending for more than a year) and thousands of open issues. The work is not getting done. The present structure does not scale. To encourage more development that we cannot support is irresponsible and unfair to the contributors.

I do think making high quality third-party packages easier to find solves a large part of this problem, but we are not there yet.

My argument is that we will never get there if we use the sub-repos as a crutch.

@adg

This comment has been minimized.

Show comment
Hide comment
@adg

adg Sep 27, 2016

Contributor

From my perspective, being able to work under an x allows me to reach out to the go approvers and this is what I care when I am designing ecosystem-wide core libraries for a niche topic. I don't think it is possible once the experimentation is not considered a part of the Go project.

I think external packages will see more high quality reviews if they are more prominent. Everyone in the ecosystem has an interest in improving the most useful (and used) packages.

Contributor

adg commented Sep 27, 2016

From my perspective, being able to work under an x allows me to reach out to the go approvers and this is what I care when I am designing ecosystem-wide core libraries for a niche topic. I don't think it is possible once the experimentation is not considered a part of the Go project.

I think external packages will see more high quality reviews if they are more prominent. Everyone in the ecosystem has an interest in improving the most useful (and used) packages.

@rakyll

This comment has been minimized.

Show comment
Hide comment
@rakyll

rakyll Sep 27, 2016

Member

has an interest in improving the most useful packages

You are mistaking your experience on the high-level community-driven packages with fundamental packages. For example, no one in the community will find debating a low-level audio API on a personal project useful or productive given there is no arbiter or a person who has an authoritative final say. I support structure, your proposal suggests distributed chaos and blessing of the popular because it is more successful.

The role of the Go project should be to review and bless fundamental things, especially the APIs. And that's where the proposal system can be sufficient enough. If the Go project doesn't have bandwidth to maintain the proposed package, the proposal should be frozen for bandwidth reasons. The ideas can keep been communicated on the proposal even though multiple implementations can live in people's personal projects.

The current state of the x packages is not what I support. But, throwing people out of the project because the core people doesn't have bandwidth is not a solution. No one ever desires a language ecosystem dominated by broken third-party solutions like Node's or Python's, but languages evolve there. I worry that it is happening for Go so soon.

Member

rakyll commented Sep 27, 2016

has an interest in improving the most useful packages

You are mistaking your experience on the high-level community-driven packages with fundamental packages. For example, no one in the community will find debating a low-level audio API on a personal project useful or productive given there is no arbiter or a person who has an authoritative final say. I support structure, your proposal suggests distributed chaos and blessing of the popular because it is more successful.

The role of the Go project should be to review and bless fundamental things, especially the APIs. And that's where the proposal system can be sufficient enough. If the Go project doesn't have bandwidth to maintain the proposed package, the proposal should be frozen for bandwidth reasons. The ideas can keep been communicated on the proposal even though multiple implementations can live in people's personal projects.

The current state of the x packages is not what I support. But, throwing people out of the project because the core people doesn't have bandwidth is not a solution. No one ever desires a language ecosystem dominated by broken third-party solutions like Node's or Python's, but languages evolve there. I worry that it is happening for Go so soon.

@rakyll

This comment has been minimized.

Show comment
Hide comment
@rakyll

rakyll Sep 28, 2016

Member

Also people who are +1ing this proposal who has all the privilege to merge their stuff to Go should better be quieter on this one. This proposal is not going to affect your daily interaction with the project. It is about the contributors who are spending their weekends to contribute to the various parts of the project. We should ensure quality is preserved without discouragement. I would like to hear more from the actual part-time contributors and I am devastated by "+1, ship it" type one-line responses.

Member

rakyll commented Sep 28, 2016

Also people who are +1ing this proposal who has all the privilege to merge their stuff to Go should better be quieter on this one. This proposal is not going to affect your daily interaction with the project. It is about the contributors who are spending their weekends to contribute to the various parts of the project. We should ensure quality is preserved without discouragement. I would like to hear more from the actual part-time contributors and I am devastated by "+1, ship it" type one-line responses.

@jimmyfrasche

This comment has been minimized.

Show comment
Hide comment
@jimmyfrasche

jimmyfrasche Sep 28, 2016

Member

Never contributed, but use packages in /x. I'd rather it be well maintained than well stocked. Frozen packages in the stdlib are annoying enough.

Member

jimmyfrasche commented Sep 28, 2016

Never contributed, but use packages in /x. I'd rather it be well maintained than well stocked. Frozen packages in the stdlib are annoying enough.

@adg

This comment has been minimized.

Show comment
Hide comment
@adg

adg Sep 28, 2016

Contributor

no one in the community will find debating a low-level audio API on a personal project useful or productive given there is no arbiter or a person who has an authoritative final say.

I don't think that's true. For example, @peterbourgon has had great success with his go-kit project, attracting people to do high level design work. I could be wrong, but I don't think you'll see much difference between participation at "github.com/goaudio" or "golang.org/x/audio". I personally would be just as interested either way.

The role of the Go project should be to review and bless fundamental things, especially the APIs. And that's where the proposal system can be sufficient enough.

To repeat what I think you said: the equivalent of package io for audio or video could live in a sub-repository of the Go project. Those are small but require careful review and pride of place. The packages that use those interfaces must not necessarily live in the sub-repos.

I agree with this completely.

I would like to hear more from the actual part-time contributors and I am devastated by "+1, ship it" type one-line responses.

I am sensitive to your concerns about "throwing people out of the project". That's certainly a non-goal, here. But, for the record, nobody has said "ship it" because there is nothing to ship. Brad and Robert have agreed with what I said, but nobody has proposed any kind of policy yet.

I too would like to hear from more contributors (or potential contributors), particularly to expand upon the list of reasons that code should live in /x, and where people think the line should be drawn.

Contributor

adg commented Sep 28, 2016

no one in the community will find debating a low-level audio API on a personal project useful or productive given there is no arbiter or a person who has an authoritative final say.

I don't think that's true. For example, @peterbourgon has had great success with his go-kit project, attracting people to do high level design work. I could be wrong, but I don't think you'll see much difference between participation at "github.com/goaudio" or "golang.org/x/audio". I personally would be just as interested either way.

The role of the Go project should be to review and bless fundamental things, especially the APIs. And that's where the proposal system can be sufficient enough.

To repeat what I think you said: the equivalent of package io for audio or video could live in a sub-repository of the Go project. Those are small but require careful review and pride of place. The packages that use those interfaces must not necessarily live in the sub-repos.

I agree with this completely.

I would like to hear more from the actual part-time contributors and I am devastated by "+1, ship it" type one-line responses.

I am sensitive to your concerns about "throwing people out of the project". That's certainly a non-goal, here. But, for the record, nobody has said "ship it" because there is nothing to ship. Brad and Robert have agreed with what I said, but nobody has proposed any kind of policy yet.

I too would like to hear from more contributors (or potential contributors), particularly to expand upon the list of reasons that code should live in /x, and where people think the line should be drawn.

@rakyll

This comment has been minimized.

Show comment
Hide comment
@rakyll

rakyll Sep 28, 2016

Member

go-kit project

I don't think go-kit a foundational package. It is a very high-level library to do certain things in a certain way and is very successful in its domain. A foundational package is a core network protocol implementation, a core abstract (io package abstractions), an open standard implementation, an industry-wide standard, etc.

Those are small but require careful review and pride of place.

I totally agree with that.

I think one of the Go project's goals should be prioritization of requirements from the language and collaborating with experts to review designs. This proposal should address that aspect before being finalized.

Brad and Robert have agreed with what I said, but nobody has proposed any kind of policy yet.

Well, ok. I am just a bit worried when influential people are on the threads without arguments. Because it influences which side people lean towards in the end.

Member

rakyll commented Sep 28, 2016

go-kit project

I don't think go-kit a foundational package. It is a very high-level library to do certain things in a certain way and is very successful in its domain. A foundational package is a core network protocol implementation, a core abstract (io package abstractions), an open standard implementation, an industry-wide standard, etc.

Those are small but require careful review and pride of place.

I totally agree with that.

I think one of the Go project's goals should be prioritization of requirements from the language and collaborating with experts to review designs. This proposal should address that aspect before being finalized.

Brad and Robert have agreed with what I said, but nobody has proposed any kind of policy yet.

Well, ok. I am just a bit worried when influential people are on the threads without arguments. Because it influences which side people lean towards in the end.

@adg

This comment has been minimized.

Show comment
Hide comment
@adg

adg Sep 28, 2016

Contributor

I added a 5th bullet to the list of reasons in the first post.

Contributor

adg commented Sep 28, 2016

I added a 5th bullet to the list of reasons in the first post.

@ianlancetaylor

This comment has been minimized.

Show comment
Hide comment
@ianlancetaylor

ianlancetaylor Sep 28, 2016

Contributor

@rakyll I'm sorry, I don't understand the reference to throwing people out of the project. I hope that nobody is suggesting that.

I agree that we have a problem. I've been talking to people about this problem for years. I don't have a solution. I'm not aware of any language that has a good solution. Other languages have varying solutions of varying quality. None seem clearly better than what Go is doing, which is basically godoc.org.

I think that golang.org/x is not a solution. I think it's basically exclusionary. It privileges the packages that the core Go team happens to care about, and relegates the rest to limbo. That doesn't scale.

But letting all packages into golang.org/x also doesn't work. The end result of that is basically what we have today--an undifferentiated list of packages of varying and unknown quality, but with different package names.

What we need is some way to identify quality packages. That is the problem. It seems to me that golang.org/x is not the solution to that problem.

But I could of course be wrong. Perhaps golang.org/x is the solution. But not as it exists today; it has to be different. So how do we change it to make it work?

Contributor

ianlancetaylor commented Sep 28, 2016

@rakyll I'm sorry, I don't understand the reference to throwing people out of the project. I hope that nobody is suggesting that.

I agree that we have a problem. I've been talking to people about this problem for years. I don't have a solution. I'm not aware of any language that has a good solution. Other languages have varying solutions of varying quality. None seem clearly better than what Go is doing, which is basically godoc.org.

I think that golang.org/x is not a solution. I think it's basically exclusionary. It privileges the packages that the core Go team happens to care about, and relegates the rest to limbo. That doesn't scale.

But letting all packages into golang.org/x also doesn't work. The end result of that is basically what we have today--an undifferentiated list of packages of varying and unknown quality, but with different package names.

What we need is some way to identify quality packages. That is the problem. It seems to me that golang.org/x is not the solution to that problem.

But I could of course be wrong. Perhaps golang.org/x is the solution. But not as it exists today; it has to be different. So how do we change it to make it work?

@danp

This comment has been minimized.

Show comment
Hide comment
@danp

danp Sep 28, 2016

Contributor

Thinking of #16218, could CLA clarity as a reason for inclusion in golang.org/x use a callout here? Perhaps that falls under reasons 4-5 already.

Contributor

danp commented Sep 28, 2016

Thinking of #16218, could CLA clarity as a reason for inclusion in golang.org/x use a callout here? Perhaps that falls under reasons 4-5 already.

@griesemer

This comment has been minimized.

Show comment
Hide comment
@griesemer

griesemer Sep 28, 2016

Contributor

@rakyll Sorry for simply repeating @bradfitz 's and thus @adg 's words. I don't have a solution either, but (repeating what's been said before) as the sub-repos are used now, they imply some form of "approval" by the Go team, yet they have essentially historically grown. In fact, several packages from the std/lib landed in sub-repos (for lack of a better place) when we cleaned up the std repo a few years back. And I have "stuff" in there (x/tools) which perhaps should be elsewhere.

I'd love if our sub-repos were blessed somehow and of high quality and maybe even maintained by us (if we could). But it doesn't scale and it's not necessarily even good for the community.

I agree w/ @ianlancetaylor that some mechanism to qualify and identify high-quality packages would be a much better solution. The question is how to do that. The Go team has neither the manpower nor the expertise (we don't have domain experts in all areas) to do that at scale.

Perhaps part of the solution is to bless a location where such high-quality packages land in the first place, and then come up with a process to rate them.

Contributor

griesemer commented Sep 28, 2016

@rakyll Sorry for simply repeating @bradfitz 's and thus @adg 's words. I don't have a solution either, but (repeating what's been said before) as the sub-repos are used now, they imply some form of "approval" by the Go team, yet they have essentially historically grown. In fact, several packages from the std/lib landed in sub-repos (for lack of a better place) when we cleaned up the std repo a few years back. And I have "stuff" in there (x/tools) which perhaps should be elsewhere.

I'd love if our sub-repos were blessed somehow and of high quality and maybe even maintained by us (if we could). But it doesn't scale and it's not necessarily even good for the community.

I agree w/ @ianlancetaylor that some mechanism to qualify and identify high-quality packages would be a much better solution. The question is how to do that. The Go team has neither the manpower nor the expertise (we don't have domain experts in all areas) to do that at scale.

Perhaps part of the solution is to bless a location where such high-quality packages land in the first place, and then come up with a process to rate them.

@nigeltao

This comment has been minimized.

Show comment
Hide comment
@nigeltao

nigeltao Sep 28, 2016

Contributor

Also people who are +1ing this proposal who has all the privilege to merge their stuff to Go should better be quieter on this one. This proposal is not going to affect your daily interaction with the project.

I disagree. I spent a significant amount of my time, over a number of months, on design and code reviews for golang.org/x/mobile/audio (later moved to golang.org/x/mobile/exp/audio). This was time I wasn't spending, day to day, on my other work.

I spent my time this way because, as @cespare alluded to above, there is a community expectation that anything under golang.org/x is of high quality. I was also actively working on golang.org/x/mobile at the time, and a number of people have said that they'd like an audio or audio/visual or media Go package to have pluggable codecs the way the standard library's image and image/* packages works and, well, I designed and wrote those image packages.

You say that "being able to work under an x allows me to reach out to the go approvers" and sure, that's a benefit for you, but it is a cost for the Go approvers. If it's easier for everyone to demand @bradfitz's attention for anything potentially under golang.org/x/net, or @ianlancetaylor's attention for anything potentially under golang.org/x/sys, then that's less attention that they could spend on e.g. landing HTTP/2 support, or keeping gccgo up to date. It is not a free lunch, and as others have said above, it doesn't scale.

It's true that golang.org/x/exp/... (and similar paths like golang.org/x/mobile/exp/...) have lower quality and support expectations than the rest of golang.org/x, but my opinion is that the right response is to move those packages out of golang.org/x, rather than add more exp packages.

Contributor

nigeltao commented Sep 28, 2016

Also people who are +1ing this proposal who has all the privilege to merge their stuff to Go should better be quieter on this one. This proposal is not going to affect your daily interaction with the project.

I disagree. I spent a significant amount of my time, over a number of months, on design and code reviews for golang.org/x/mobile/audio (later moved to golang.org/x/mobile/exp/audio). This was time I wasn't spending, day to day, on my other work.

I spent my time this way because, as @cespare alluded to above, there is a community expectation that anything under golang.org/x is of high quality. I was also actively working on golang.org/x/mobile at the time, and a number of people have said that they'd like an audio or audio/visual or media Go package to have pluggable codecs the way the standard library's image and image/* packages works and, well, I designed and wrote those image packages.

You say that "being able to work under an x allows me to reach out to the go approvers" and sure, that's a benefit for you, but it is a cost for the Go approvers. If it's easier for everyone to demand @bradfitz's attention for anything potentially under golang.org/x/net, or @ianlancetaylor's attention for anything potentially under golang.org/x/sys, then that's less attention that they could spend on e.g. landing HTTP/2 support, or keeping gccgo up to date. It is not a free lunch, and as others have said above, it doesn't scale.

It's true that golang.org/x/exp/... (and similar paths like golang.org/x/mobile/exp/...) have lower quality and support expectations than the rest of golang.org/x, but my opinion is that the right response is to move those packages out of golang.org/x, rather than add more exp packages.

@mpvl

This comment has been minimized.

Show comment
Hide comment
@mpvl

mpvl Sep 28, 2016

Member

I largely agree with Quentin here. In my experience, having a solid base of fundamental libraries is key to being productive in a language. Look at Java, Objective-C/Swift, .Net. All have comprehensive standard libraries.

adg's reasons for inclusion have a lot of positive sides too:

  1. (1): Stability: having an "official" package gives assurances it won't be replaced by a new "blessed" one willy nilly. Go's adoption started taking off after the 1.0 guarantees were given. Stability of fundamental packages matters as well. urllib2 debacles should be rare and are an acceptable collateral damage.
  2. (1): Efforts: having "official" packages guides efforts towards such packages, instead of across competing standards.
  3. (4, 5): Consistency: having some standard review process and inclusion process is more likely to lead to uniformity of API than a selection of best-of-bread sources. It is easier to enforce consistencies on third-party contributors if a "golang.org" stamp is to be given.
  4. (2): Support: The golang.org stamp could assure hosting, an initial blessing of API and providing standard policies. It does not have to mean active support from the Go team. This can come from the community.
  5. (3): Branding/Trust: The golang.org is a brand. For a newcomer considering Go, core functionality consisting of a list of blessed package will not nearly instill as much confidence as a list of official packages providing the same.
  6. (1): Availability: Ever had a high-quality package disappear on you? I have.
  7. Commercial: I think many businesses will not adopt Go if fundamental functionality only exists in the form of no-guarantee, blessed packages with inconsistent licenses, guarantees of API stability, etc.

I understand that the Go team does not have enough cycles. But in my opinion, a blessed package model doesn't scale for users and will hamper Go adoption.

The solution, I think, should come from scaling the subrepo approach, maybe by using a hybrid form. Here are some thoughts:

  1. There would be 3 groups of subrepos:
    • 2 and 3: Go-team (and contributors) internal infrastructure.
    • 1 and 4: semi-blessed/ official tools and packages, what roughly would be part of a standard
      release for other languages.
    • Experimental, including mobile, exp, but possibly also counterparts to x repos with experimental packages.
  2. The golang.org/x "official" repos would give some guarantee of having gone through a process and quality control, without necessarily having support of Go-team members.
  3. To limit the number of proposals, the kind of repos/packages that can make it in should be reasonably well-defined. A guide could be what other languages/frameworks provide.
  4. The Go team would still have a list of blessed repos. Blessed repos with fundamental functionality could make it official after some trial. This also gives more weight to blessed repos.
  5. Proposals to include something in golang.org/x should be more of the from "please include this existing repo" rather than "please implement this repo". A reply to a proposal could be suggestions to API changes or other tweaks.
  6. Short of that, an answer to a proposal could be, "Sure we suggest someone implements it in experimental and if certain criteria are met it will be included in official".
  7. Each repo has a group of maintainers and approvers which may be outside of the Go team .
  8. There could be some way to review Important API changes by a mixed Go and external team. (Note that, as Quentin suggested, there would not be many more packages in x).
  9. Different repos may have different SLAs, but are required to have one.
  10. Repos in x could have some automated enforcement like ensuring a stable API surface, code coverage, linter, perhaps, not losing performance on benchmarks, etc.
  11. golang.org/x repos could be required to adhere to a stricter style guide than one would usually impose as a requirement on "blessed" packages.

To have a good quality list of "blessed" packages will either be also a lot of work (verifying APIs of probably more packages than you would normally include in an x subrepo), or it will inherently mean it will be of lesser quality than an official list. If it is of the same quality, it still can never instill as much trust as confidence as an "official" list, unless we include "blessed" packages in an official release, which seems even more work to me. Even worse, never having had "official" subrepos is one thing, but giving them up is a clear signal of throwing in the towel. So to me, trying to reduce/spread the burden of maintaining golang.org, rather then shedding it, seems crucial.

Member

mpvl commented Sep 28, 2016

I largely agree with Quentin here. In my experience, having a solid base of fundamental libraries is key to being productive in a language. Look at Java, Objective-C/Swift, .Net. All have comprehensive standard libraries.

adg's reasons for inclusion have a lot of positive sides too:

  1. (1): Stability: having an "official" package gives assurances it won't be replaced by a new "blessed" one willy nilly. Go's adoption started taking off after the 1.0 guarantees were given. Stability of fundamental packages matters as well. urllib2 debacles should be rare and are an acceptable collateral damage.
  2. (1): Efforts: having "official" packages guides efforts towards such packages, instead of across competing standards.
  3. (4, 5): Consistency: having some standard review process and inclusion process is more likely to lead to uniformity of API than a selection of best-of-bread sources. It is easier to enforce consistencies on third-party contributors if a "golang.org" stamp is to be given.
  4. (2): Support: The golang.org stamp could assure hosting, an initial blessing of API and providing standard policies. It does not have to mean active support from the Go team. This can come from the community.
  5. (3): Branding/Trust: The golang.org is a brand. For a newcomer considering Go, core functionality consisting of a list of blessed package will not nearly instill as much confidence as a list of official packages providing the same.
  6. (1): Availability: Ever had a high-quality package disappear on you? I have.
  7. Commercial: I think many businesses will not adopt Go if fundamental functionality only exists in the form of no-guarantee, blessed packages with inconsistent licenses, guarantees of API stability, etc.

I understand that the Go team does not have enough cycles. But in my opinion, a blessed package model doesn't scale for users and will hamper Go adoption.

The solution, I think, should come from scaling the subrepo approach, maybe by using a hybrid form. Here are some thoughts:

  1. There would be 3 groups of subrepos:
    • 2 and 3: Go-team (and contributors) internal infrastructure.
    • 1 and 4: semi-blessed/ official tools and packages, what roughly would be part of a standard
      release for other languages.
    • Experimental, including mobile, exp, but possibly also counterparts to x repos with experimental packages.
  2. The golang.org/x "official" repos would give some guarantee of having gone through a process and quality control, without necessarily having support of Go-team members.
  3. To limit the number of proposals, the kind of repos/packages that can make it in should be reasonably well-defined. A guide could be what other languages/frameworks provide.
  4. The Go team would still have a list of blessed repos. Blessed repos with fundamental functionality could make it official after some trial. This also gives more weight to blessed repos.
  5. Proposals to include something in golang.org/x should be more of the from "please include this existing repo" rather than "please implement this repo". A reply to a proposal could be suggestions to API changes or other tweaks.
  6. Short of that, an answer to a proposal could be, "Sure we suggest someone implements it in experimental and if certain criteria are met it will be included in official".
  7. Each repo has a group of maintainers and approvers which may be outside of the Go team .
  8. There could be some way to review Important API changes by a mixed Go and external team. (Note that, as Quentin suggested, there would not be many more packages in x).
  9. Different repos may have different SLAs, but are required to have one.
  10. Repos in x could have some automated enforcement like ensuring a stable API surface, code coverage, linter, perhaps, not losing performance on benchmarks, etc.
  11. golang.org/x repos could be required to adhere to a stricter style guide than one would usually impose as a requirement on "blessed" packages.

To have a good quality list of "blessed" packages will either be also a lot of work (verifying APIs of probably more packages than you would normally include in an x subrepo), or it will inherently mean it will be of lesser quality than an official list. If it is of the same quality, it still can never instill as much trust as confidence as an "official" list, unless we include "blessed" packages in an official release, which seems even more work to me. Even worse, never having had "official" subrepos is one thing, but giving them up is a clear signal of throwing in the towel. So to me, trying to reduce/spread the burden of maintaining golang.org, rather then shedding it, seems crucial.

@rakyll

This comment has been minimized.

Show comment
Hide comment
@rakyll

rakyll Sep 28, 2016

Member

disagree. I spent a significant amount of my time, over a number of months, on design and code reviews for golang.org/x/mobile/audio.

The mobile package is planned and executed by this team as an experiment. The entire thing should have never been under /x. As this proposal calls out, there are a few packages under this category and should have been bootstrapped elsewhere, e.g. how bradfitz bootstrapped http2 outside of the tree.

sure, that's a benefit for you, but it is a cost for the Go approvers.

I thought this discussion already acknowledged the bandwidth issue. What @adg asks for is how others are affected by such a policy and it is very critical to understand what a proper solution to community-driven packages might look like.

I mentioned "being able to work under an x allows me to reach out to the go approvers", because it is a necessity. If people were able to escalate a couple of API sanity questions to a group of people they trust, they wouldn't care where they bootstrap or maintain their packages. This is an actual requirement, please don't dismiss it by repeating what we already know.

And that's not a benefit for me. It is a benefit for the language to allow different people collaborate on a medium and escalate things about API review.

adg mentions "/x" being prestigious but I disagree. In my unique experience, /x is a very invisible point. There is no true benefit someone is contributing a package to /x if they don't care about the quality or design feedback.

Before I finish, I want to say I really appreciate what @ianlancetaylor and @griesemer suggests. We may need a successor plan or strategy and think about the indicator of quality. The ideal location where these packages bootstrapped or maintained might not be /x, but organizing the distributed effort towards enhancing the Go library-space should be a critical project on its own.

Member

rakyll commented Sep 28, 2016

disagree. I spent a significant amount of my time, over a number of months, on design and code reviews for golang.org/x/mobile/audio.

The mobile package is planned and executed by this team as an experiment. The entire thing should have never been under /x. As this proposal calls out, there are a few packages under this category and should have been bootstrapped elsewhere, e.g. how bradfitz bootstrapped http2 outside of the tree.

sure, that's a benefit for you, but it is a cost for the Go approvers.

I thought this discussion already acknowledged the bandwidth issue. What @adg asks for is how others are affected by such a policy and it is very critical to understand what a proper solution to community-driven packages might look like.

I mentioned "being able to work under an x allows me to reach out to the go approvers", because it is a necessity. If people were able to escalate a couple of API sanity questions to a group of people they trust, they wouldn't care where they bootstrap or maintain their packages. This is an actual requirement, please don't dismiss it by repeating what we already know.

And that's not a benefit for me. It is a benefit for the language to allow different people collaborate on a medium and escalate things about API review.

adg mentions "/x" being prestigious but I disagree. In my unique experience, /x is a very invisible point. There is no true benefit someone is contributing a package to /x if they don't care about the quality or design feedback.

Before I finish, I want to say I really appreciate what @ianlancetaylor and @griesemer suggests. We may need a successor plan or strategy and think about the indicator of quality. The ideal location where these packages bootstrapped or maintained might not be /x, but organizing the distributed effort towards enhancing the Go library-space should be a critical project on its own.

@bradfitz

This comment has been minimized.

Show comment
Hide comment
@bradfitz

bradfitz Sep 28, 2016

Member

Regarding the reachability of Go developers and comfort of using the typical Go tools for development, would it help if I finished up my @LetsUseGerrit bot so any repo on Github could use the Gerrit code review process?

Member

bradfitz commented Sep 28, 2016

Regarding the reachability of Go developers and comfort of using the typical Go tools for development, would it help if I finished up my @LetsUseGerrit bot so any repo on Github could use the Gerrit code review process?

@rakyll

This comment has been minimized.

Show comment
Hide comment
@rakyll

rakyll Sep 28, 2016

Member

I think Gerrit might not be critical given Github PRs are becoming better. How people reach out to quality reviewers might be a more interesting problem. Would it be a reviewers mailing list? If so, how to communicate it is entirely voluntary work? Should it come from the Go project or driven by the community? I have some ideas but don't have any concrete proposals at this point.

Member

rakyll commented Sep 28, 2016

I think Gerrit might not be critical given Github PRs are becoming better. How people reach out to quality reviewers might be a more interesting problem. Would it be a reviewers mailing list? If so, how to communicate it is entirely voluntary work? Should it come from the Go project or driven by the community? I have some ideas but don't have any concrete proposals at this point.

@zellyn

This comment has been minimized.

Show comment
Hide comment
@zellyn

zellyn Sep 28, 2016

My (subjective) impression is that the burden of being responsive and reviewing code is burning out core team members. Is that true? If so, I think it's a significant factor for consideration.

zellyn commented Sep 28, 2016

My (subjective) impression is that the burden of being responsive and reviewing code is burning out core team members. Is that true? If so, I think it's a significant factor for consideration.

@adg

This comment has been minimized.

Show comment
Hide comment
@adg

adg Sep 28, 2016

Contributor

My (subjective) impression is that the burden of being responsive and reviewing code is burning out core team members. Is that true?

I can't speak for everyone, but it's true for me and I know it's true for others on the team.

Contributor

adg commented Sep 28, 2016

My (subjective) impression is that the burden of being responsive and reviewing code is burning out core team members. Is that true?

I can't speak for everyone, but it's true for me and I know it's true for others on the team.

@bradfitz

This comment has been minimized.

Show comment
Hide comment
@bradfitz

bradfitz Sep 28, 2016

Member

@zellyn, yes. I'm starting to burn out. It's gotten exhausting the past year or two.

Member

bradfitz commented Sep 28, 2016

@zellyn, yes. I'm starting to burn out. It's gotten exhausting the past year or two.

@seh

This comment has been minimized.

Show comment
Hide comment
@seh

seh Sep 28, 2016

Contributor

@ianlancetaylor wrote:

I'm not aware of any language that has a good solution. Other languages have varying solutions of varying quality. None seem clearly better than what Go is doing, which is basically godoc.org.

For C++, the Boost project has served as a proving ground for libraries that might eventually make it into proposals for inclusion in the standard library. The libraries usually have to shrink (shedding features) and bend under the second phase of criticism from committee members. I've witnessed @davecheney preparing for this process with his pkg/errors library.

The downside of an external proving ground like Boost is that people start using the "outside" version of the library, which is a necessary part of vetting it, and may never be able to afford to port to the "standard" version that makes it in much later. The world winds up with two versions in use, even when the goal was to only have one.

Contributor

seh commented Sep 28, 2016

@ianlancetaylor wrote:

I'm not aware of any language that has a good solution. Other languages have varying solutions of varying quality. None seem clearly better than what Go is doing, which is basically godoc.org.

For C++, the Boost project has served as a proving ground for libraries that might eventually make it into proposals for inclusion in the standard library. The libraries usually have to shrink (shedding features) and bend under the second phase of criticism from committee members. I've witnessed @davecheney preparing for this process with his pkg/errors library.

The downside of an external proving ground like Boost is that people start using the "outside" version of the library, which is a necessary part of vetting it, and may never be able to afford to port to the "standard" version that makes it in much later. The world winds up with two versions in use, even when the goal was to only have one.

@dgryski

This comment has been minimized.

Show comment
Hide comment
@dgryski

dgryski Sep 28, 2016

Contributor

To scale kernel development, Linus named Trusted Lieutenants as guardians of different subsystems. @rakyll's idea for creating "Trusted Gophers" available for code review for external projects is an interesting one. However, it's hard to do code review without context. It's easy to comment on style and naming, but hard to discuss design without understanding the problem domain. Drive-by reviewers provide only superficial help here. Dealing with reviewing "yet another" web framework or whatever will also get tiring. I think this deserves more discussion, but possibly in a different forum that this issue.

Contributor

dgryski commented Sep 28, 2016

To scale kernel development, Linus named Trusted Lieutenants as guardians of different subsystems. @rakyll's idea for creating "Trusted Gophers" available for code review for external projects is an interesting one. However, it's hard to do code review without context. It's easy to comment on style and naming, but hard to discuss design without understanding the problem domain. Drive-by reviewers provide only superficial help here. Dealing with reviewing "yet another" web framework or whatever will also get tiring. I think this deserves more discussion, but possibly in a different forum that this issue.

@rakyll

This comment has been minimized.

Show comment
Hide comment
@rakyll

rakyll Sep 28, 2016

Member

Dealing with reviewing "yet another" web framework or whatever will also get tiring.

For the record, this is for the foundational packages. Not for yet another web framework but for a COAP or a USB package for Go. I am not proposing to establish a general-purpose support group.

Member

rakyll commented Sep 28, 2016

Dealing with reviewing "yet another" web framework or whatever will also get tiring.

For the record, this is for the foundational packages. Not for yet another web framework but for a COAP or a USB package for Go. I am not proposing to establish a general-purpose support group.

@ericlagergren

This comment has been minimized.

Show comment
Hide comment
@ericlagergren

ericlagergren Sep 29, 2016

Contributor

It's easy to comment on style and naming, but hard to discuss design without understanding the problem domain.

Then it seems to reason the "Trusted Gophers" should each be responsible for only one portion of whatever the Go "subsystem" is defined as.

Gopher A who has a knack for math can review all the new math libraries.

Gopher B who prefers interfacing with USB drives can review the USB device code.

Gopher C who really loves pre-historic protocols can review the new Gopher and Finger code.

I agree with others in that when I need to do X, I usually turn first to the stdlib, then x/, then godoc and Google. While not everything in x/ is usable (to my liking), I've found a lot less "surprises" inside x/ than I have casually browsing godoc or Google. Great post, @adg.

Contributor

ericlagergren commented Sep 29, 2016

It's easy to comment on style and naming, but hard to discuss design without understanding the problem domain.

Then it seems to reason the "Trusted Gophers" should each be responsible for only one portion of whatever the Go "subsystem" is defined as.

Gopher A who has a knack for math can review all the new math libraries.

Gopher B who prefers interfacing with USB drives can review the USB device code.

Gopher C who really loves pre-historic protocols can review the new Gopher and Finger code.

I agree with others in that when I need to do X, I usually turn first to the stdlib, then x/, then godoc and Google. While not everything in x/ is usable (to my liking), I've found a lot less "surprises" inside x/ than I have casually browsing godoc or Google. Great post, @adg.

@StabbyCutyou

This comment has been minimized.

Show comment
Hide comment
@StabbyCutyou

StabbyCutyou Sep 29, 2016

If the problem is burnout, then consider there is a hidden cost to the "trusted" approach. People need to be responsible to train, mentor, and guide newly "trusted" individuals into doing the right thing, at least to the existing standards, to the point where eventually they need no oversight. This will introduce some lag time initially as those who train the new folks will be pulling something of a double duty.

You might say that only folks who don't need oversight would be picked, but this is not realistically possible. The greatest engineer in the world still has ramp up moving into a new system or a new codebase.

Ultimately, I disagree with seeing Go get "smaller" as a result of these issues. I want to see Go get bigger. I want a standard library (even if it has different tiers of service or guarantees) that feels large and inclusive. Especially with Go, where the community ethos is always to do as much as possible with the stdlib before reaching out to a dependent library.

Solving the human cost issues and making it so these things aren't so taxing on the current team (whether that is a process change, the "trusted" approach, or some other solution) sound a lot more palatable in nature to me, someone who uses the language and wants to continue to see it grow, than the alternatives.

But i recognize that the reason why the core maintainers want it smaller is because it is likely exhausting to try and keep up with all of these changes / proposals / etc. But in my (likely uninformed) opinion, closing off or making it harder to put new things into the stdlib is a step backwards. The strongest part of Go is the community itself, and the community desperately wants to be involved in helping to shape the future of the language.

If the problem is burnout, then consider there is a hidden cost to the "trusted" approach. People need to be responsible to train, mentor, and guide newly "trusted" individuals into doing the right thing, at least to the existing standards, to the point where eventually they need no oversight. This will introduce some lag time initially as those who train the new folks will be pulling something of a double duty.

You might say that only folks who don't need oversight would be picked, but this is not realistically possible. The greatest engineer in the world still has ramp up moving into a new system or a new codebase.

Ultimately, I disagree with seeing Go get "smaller" as a result of these issues. I want to see Go get bigger. I want a standard library (even if it has different tiers of service or guarantees) that feels large and inclusive. Especially with Go, where the community ethos is always to do as much as possible with the stdlib before reaching out to a dependent library.

Solving the human cost issues and making it so these things aren't so taxing on the current team (whether that is a process change, the "trusted" approach, or some other solution) sound a lot more palatable in nature to me, someone who uses the language and wants to continue to see it grow, than the alternatives.

But i recognize that the reason why the core maintainers want it smaller is because it is likely exhausting to try and keep up with all of these changes / proposals / etc. But in my (likely uninformed) opinion, closing off or making it harder to put new things into the stdlib is a step backwards. The strongest part of Go is the community itself, and the community desperately wants to be involved in helping to shape the future of the language.

@kardianos

This comment has been minimized.

Show comment
Hide comment
@kardianos

kardianos Sep 29, 2016

Contributor

@adg Additional reasons for code in the x repos:

  • All packages in the x repos have a consistent license with a CLA to assure code ownership.
  • Because some level of permission is required, duplicate packages don't happen, so evolution of the current package is the default, rather than forking or doing my own thing.

I echo @griesemer

Perhaps part of the solution is to bless a location where such high-quality packages land in the first place, and then come up with a process to rate them.

Would the go team have the bandwidth to do high level API approval and package scope approval? Approval would have a certain scope, but then actual reviews and work would be community driven. If someone wanted to have a GUI widget library, a new package wouldn't get approved, rather they would be instructed to go talk to the ext/shiny contributors.

edit: but to be clear, I'm not directly advocating more packages go into an x like repo.

Contributor

kardianos commented Sep 29, 2016

@adg Additional reasons for code in the x repos:

  • All packages in the x repos have a consistent license with a CLA to assure code ownership.
  • Because some level of permission is required, duplicate packages don't happen, so evolution of the current package is the default, rather than forking or doing my own thing.

I echo @griesemer

Perhaps part of the solution is to bless a location where such high-quality packages land in the first place, and then come up with a process to rate them.

Would the go team have the bandwidth to do high level API approval and package scope approval? Approval would have a certain scope, but then actual reviews and work would be community driven. If someone wanted to have a GUI widget library, a new package wouldn't get approved, rather they would be instructed to go talk to the ext/shiny contributors.

edit: but to be clear, I'm not directly advocating more packages go into an x like repo.

@adg

This comment has been minimized.

Show comment
Hide comment
@adg

adg Sep 29, 2016

Contributor

@StabbyCutyou said:

But i recognize that the reason why the core maintainers want it smaller is because it is likely exhausting to try and keep up with all of these changes / proposals / etc.

Well, yeah. It's not just that we're exhausted, but also that we're falling behind, with no end in sight. Your proposed solution just creates more work for those people, therefore it is not a solution, as it will never happen.

The strongest part of Go is the community itself, and the community desperately wants to be involved in helping to shape the future of the language.

This isn't about the future of the language. It's about libraries. IMO if high quality libraries cannot thrive outside the project itself, then the language will have failed. How could any of today's major languages succeeded without high quality external libraries? We're not going to get there by gatekeeping.

Contributor

adg commented Sep 29, 2016

@StabbyCutyou said:

But i recognize that the reason why the core maintainers want it smaller is because it is likely exhausting to try and keep up with all of these changes / proposals / etc.

Well, yeah. It's not just that we're exhausted, but also that we're falling behind, with no end in sight. Your proposed solution just creates more work for those people, therefore it is not a solution, as it will never happen.

The strongest part of Go is the community itself, and the community desperately wants to be involved in helping to shape the future of the language.

This isn't about the future of the language. It's about libraries. IMO if high quality libraries cannot thrive outside the project itself, then the language will have failed. How could any of today's major languages succeeded without high quality external libraries? We're not going to get there by gatekeeping.

@adg

This comment has been minimized.

Show comment
Hide comment
@adg

adg Sep 29, 2016

Contributor

@kardianos thanks for those extra points. I've added them to the list. My opinion on them:

To use the same consistent license (with CLA) that the project uses.

I think we could solve this in a more scalable way by setting a community standard for code licensing. There could even be a little badge/icon with a gopher wearing a judge's wig.

To provide a single package for a single purpose, and avoid duplicate effort.

I'm concerned about what happens when better work is done elsewhere and then the package in /x atrophies. This already happened with websocket, where the package in /x/net is not the best implementation, but people use it because it's there. Efforts to replace it have stalled for years. IMO, this is more evidence that the centralized repo approach does not scale.

Contributor

adg commented Sep 29, 2016

@kardianos thanks for those extra points. I've added them to the list. My opinion on them:

To use the same consistent license (with CLA) that the project uses.

I think we could solve this in a more scalable way by setting a community standard for code licensing. There could even be a little badge/icon with a gopher wearing a judge's wig.

To provide a single package for a single purpose, and avoid duplicate effort.

I'm concerned about what happens when better work is done elsewhere and then the package in /x atrophies. This already happened with websocket, where the package in /x/net is not the best implementation, but people use it because it's there. Efforts to replace it have stalled for years. IMO, this is more evidence that the centralized repo approach does not scale.

@dlsniper

This comment has been minimized.

Show comment
Hide comment
@dlsniper

dlsniper Oct 4, 2016

Contributor

@nathany et. al. I've opened golang/gddo#445 as per suggestion you had so that we keep this one clean. Looking forward for comments there. Thank you.

Contributor

dlsniper commented Oct 4, 2016

@nathany et. al. I've opened golang/gddo#445 as per suggestion you had so that we keep this one clean. Looking forward for comments there. Thank you.

@SamWhited

This comment has been minimized.

Show comment
Hide comment
@SamWhited

SamWhited Oct 4, 2016

Member

Q: What should go in golang.org/x ?

As the proposer of #16257 (and other subrepo requests in the past, some accepted, and some not), my criteria for if I propose a project be moved into the x/ namespace is generally "do I think this would be useful in the standard library eventually, eg. context, or do I think it could be vendored in the standard library to support other packages, eg. that sasl proposal?)

If the answer to both of those questions is no, I probably would prefer to keep it to myself.

The only place I make a slight exception to this is when I consider security and crypto related packages. Eg. with the SASL proposal I feel that it's important to have a single, known-good, well-vetted implementation. Fair or not, having it under golang.org/x/ will make people assume it's the "recommended" solution, and the process (and intelligent developers that are a part of this community), would hopefuly ensure that nothing horrible slips through the cracks, so security packages get slightly different considerations for me.

Q: How to fix the Go team bandwidth issue?

I think the answer hear is to allow third parties and trusted domain experts to act as the maintainer of some packages in the subrepos. There's no reason the Go team has to manage every single package themselves. This doesn't mean all authors immediately get commit access or review access, but it might mean that eventually long-time contributors and others that have shown that they can be responsible and maintain the standards expected in the subrepos could take some of the review and merge burden off the core team.

Member

SamWhited commented Oct 4, 2016

Q: What should go in golang.org/x ?

As the proposer of #16257 (and other subrepo requests in the past, some accepted, and some not), my criteria for if I propose a project be moved into the x/ namespace is generally "do I think this would be useful in the standard library eventually, eg. context, or do I think it could be vendored in the standard library to support other packages, eg. that sasl proposal?)

If the answer to both of those questions is no, I probably would prefer to keep it to myself.

The only place I make a slight exception to this is when I consider security and crypto related packages. Eg. with the SASL proposal I feel that it's important to have a single, known-good, well-vetted implementation. Fair or not, having it under golang.org/x/ will make people assume it's the "recommended" solution, and the process (and intelligent developers that are a part of this community), would hopefuly ensure that nothing horrible slips through the cracks, so security packages get slightly different considerations for me.

Q: How to fix the Go team bandwidth issue?

I think the answer hear is to allow third parties and trusted domain experts to act as the maintainer of some packages in the subrepos. There's no reason the Go team has to manage every single package themselves. This doesn't mean all authors immediately get commit access or review access, but it might mean that eventually long-time contributors and others that have shown that they can be responsible and maintain the standards expected in the subrepos could take some of the review and merge burden off the core team.

@nathany nathany referenced this issue in Azure/go-pkcs12 Oct 4, 2016

Closed

revive project for pkcs12 write support #28

@namsral namsral referenced this issue in golang/gddo Oct 5, 2016

Open

Proposal to add API stability marker #446

@nigeltao

This comment has been minimized.

Show comment
Hide comment
@nigeltao

nigeltao Oct 16, 2016

Contributor

I don't think it's in your enumeration: another reason for something to live under x/ is being a dependency for something else under x/.

For example, Brotli is a compression format. A Go "package brotli" might or might not meet the bar for x/ inclusion on its own merits, but e.g. if we wanted WOFF2 (Web Open Font Format 2) support somewhere under x/, it would depend on a Brotli decoder, so I'd expect a "package brotli" to also have to live somewhere under x/.

Contributor

nigeltao commented Oct 16, 2016

I don't think it's in your enumeration: another reason for something to live under x/ is being a dependency for something else under x/.

For example, Brotli is a compression format. A Go "package brotli" might or might not meet the bar for x/ inclusion on its own merits, but e.g. if we wanted WOFF2 (Web Open Font Format 2) support somewhere under x/, it would depend on a Brotli decoder, so I'd expect a "package brotli" to also have to live somewhere under x/.

@mpvl

This comment has been minimized.

Show comment
Hide comment
@mpvl

mpvl Oct 16, 2016

Member

Data point: recently the dependencies of core on x/net, x/crypto, and x/text have been steadily increasing. There are now 9 vendored packages in core with more to come.

Member

mpvl commented Oct 16, 2016

Data point: recently the dependencies of core on x/net, x/crypto, and x/text have been steadily increasing. There are now 9 vendored packages in core with more to come.

@adg

This comment has been minimized.

Show comment
Hide comment
@adg

adg Oct 17, 2016

Contributor

@nigeltao I added that to the list. FWIW, I think we can use internal/ to side-step that issue.

Contributor

adg commented Oct 17, 2016

@nigeltao I added that to the list. FWIW, I think we can use internal/ to side-step that issue.

@nathany

This comment has been minimized.

Show comment
Hide comment
@nathany

nathany Oct 17, 2016

Contributor

@adg I am curious how licensing may impact the use of internal/ in core or current x/ tools. For example, if fsnotify wasn't under the Google CLA, could guru or godoc or go build still use fsnotify? Likewise with brotli or other core/low-level packages like USB.

Either way, (IMO) the CLA used should be orthogonal to the location of the code, e.g. grpc-go still has the Google CLA.

Contributor

nathany commented Oct 17, 2016

@adg I am curious how licensing may impact the use of internal/ in core or current x/ tools. For example, if fsnotify wasn't under the Google CLA, could guru or godoc or go build still use fsnotify? Likewise with brotli or other core/low-level packages like USB.

Either way, (IMO) the CLA used should be orthogonal to the location of the code, e.g. grpc-go still has the Google CLA.

@bradfitz

This comment has been minimized.

Show comment
Hide comment
@bradfitz

bradfitz Oct 17, 2016

Member

@nathany, yes, we can only use CLA-compliant code in the core. That's briefly addressed by the bullet at top:

To use the same consistent license (with CLA) that the project uses.

But it's not very explicit.

Member

bradfitz commented Oct 17, 2016

@nathany, yes, we can only use CLA-compliant code in the core. That's briefly addressed by the bullet at top:

To use the same consistent license (with CLA) that the project uses.

But it's not very explicit.

@nathany

This comment has been minimized.

Show comment
Hide comment
@nathany

nathany Oct 17, 2016

Contributor

The fsnotify codebase originated from code contributed to Go, then moved to x/exp, and now maintained in a separate GitHub organization. At one point we had all contributors sign the Google CLA, but I haven't been good about enforcing it for patches since. 🙁

If a project like gomobile were moved outside of x/, I assume the CLA would still be required. Is there tooling to help enforce it? Does it require a Gerrit workflow or is it independent of Gerrit?

Contributions to GitHub's repositories have a CLA check, so it is something that can be done on GitHub, though I don't know the implementation details.

github-cla

That would address the technical issue of enforcing the CLA -- independent of repository location.

The other issue I personally have is that the non-endorsement license and Google CLA feel awkward for fsnotify specifically. It isn't currently a Google-run project, but needs the CLA and license for a potential future use in core. Which ends up feeling like an endorsement. Heh.

I'm not sure if this applies to any other projects that may be moved out of the x/ subrepos? Projects that may no longer under the review of Go team members (that's kind've the point of all this), but are low-level enough that they still may be useful as internal/ packages in core.

Contributor

nathany commented Oct 17, 2016

The fsnotify codebase originated from code contributed to Go, then moved to x/exp, and now maintained in a separate GitHub organization. At one point we had all contributors sign the Google CLA, but I haven't been good about enforcing it for patches since. 🙁

If a project like gomobile were moved outside of x/, I assume the CLA would still be required. Is there tooling to help enforce it? Does it require a Gerrit workflow or is it independent of Gerrit?

Contributions to GitHub's repositories have a CLA check, so it is something that can be done on GitHub, though I don't know the implementation details.

github-cla

That would address the technical issue of enforcing the CLA -- independent of repository location.

The other issue I personally have is that the non-endorsement license and Google CLA feel awkward for fsnotify specifically. It isn't currently a Google-run project, but needs the CLA and license for a potential future use in core. Which ends up feeling like an endorsement. Heh.

I'm not sure if this applies to any other projects that may be moved out of the x/ subrepos? Projects that may no longer under the review of Go team members (that's kind've the point of all this), but are low-level enough that they still may be useful as internal/ packages in core.

@mdlayher

This comment has been minimized.

Show comment
Hide comment
@mdlayher

mdlayher Dec 14, 2016

Member

Hey all, apologies for the ping, but has there been any movement on this at all? I can see that this proposal has been referenced from several others (including one of my own).

Member

mdlayher commented Dec 14, 2016

Hey all, apologies for the ping, but has there been any movement on this at all? I can see that this proposal has been referenced from several others (including one of my own).

@bradfitz

This comment has been minimized.

Show comment
Hide comment
@bradfitz

bradfitz Dec 14, 2016

Member

No movement yet. Updates will be posted here.

Member

bradfitz commented Dec 14, 2016

No movement yet. Updates will be posted here.

@rakyll

This comment has been minimized.

Show comment
Hide comment
@rakyll

rakyll Jan 3, 2017

Member

Similar to my previous point earlier on this thread, @mattetti says on #18497 (comment), a central point to discuss ecosystem-wide APIs and libraries is a hard requirement for the success of anything beyond the core project.

I think regardless of what's our decision for the new policy regarding to the x repos, we need a migration/replacment plan for those who finds it productive to use Go mediums.

Member

rakyll commented Jan 3, 2017

Similar to my previous point earlier on this thread, @mattetti says on #18497 (comment), a central point to discuss ecosystem-wide APIs and libraries is a hard requirement for the success of anything beyond the core project.

I think regardless of what's our decision for the new policy regarding to the x repos, we need a migration/replacment plan for those who finds it productive to use Go mediums.

@mattetti

This comment has been minimized.

Show comment
Hide comment
@mattetti

mattetti Jan 3, 2017

Contributor

I'd like to resume this discussion since it seems to be critical for the evolution of the stdlib and the extension of the ecosystem. There are a lot of very valid points made in this thread and I believe everyone cares to keep the burden on core go developers as light as possible while also benefiting from a great ecosystem.

From what I read I saw two proposed ideas to address the expansion of "core" packages:

  1. improve godoc.org (@adg)
  2. create the equivalent of Linux subsystems / official interest groups (suggested by @rakyll / @jessfraz )

What's clear to me is that we need to scale and even though my proposal #18497 was to provide an official package for interoperating about (but not necessarily doing) X to quote @adg, I honestly don't care too much where it's going to live as long as it's clearly discoverable and have a thorough review process. Doing that outside of the Go project umbrella is extremely difficile and I think that's the heart of the issue.

At this point I think we could have 2 non exclusive strategies:

  1. The first one would look at what can we do to make godoc.org a more useful place to find high quality, non-std library packages, start on that work in an incremental manner.

  2. The other would be to create official interest groups which would have for mission to support "subsystems". Their role would go from maintaining and organizing a list of existing libraries. Defining standards and advocating for them. Code reviewing submitted 3rd party libraries and implementing interop solutions or actual features. These groups might benefit from current tooling and occasional access to core team members but wouldn't be restricted by release cycles and core team vetting.
    Off the top of my head, here are interest groups I can think of: media (audio/video), images, hardware, mobile, GUI, crypto, web services (similar to go-kit), dev tooling, machine intelligence... Having those groups defined, official and the process well documented should help us scale.
    That would obviously require a clearer delimitation between the standard library and non-standard packages. Really high quality packages that became standard don't ever have to "graduate" to the standard library but they have to be looked after by more than 1 owner (company or developer) and I believe those groups should be under the official Go umbrella.

If this idea sounds good, I would suggest to start by a couple special interest groups with momentum such as media and let them figure out how they want to get organized. I would assume an official team, repo, toolset and mission would be required. Then exposure with guidance are probably going to be the most valuable things that the project can provide to those teams.

What do you think?

Contributor

mattetti commented Jan 3, 2017

I'd like to resume this discussion since it seems to be critical for the evolution of the stdlib and the extension of the ecosystem. There are a lot of very valid points made in this thread and I believe everyone cares to keep the burden on core go developers as light as possible while also benefiting from a great ecosystem.

From what I read I saw two proposed ideas to address the expansion of "core" packages:

  1. improve godoc.org (@adg)
  2. create the equivalent of Linux subsystems / official interest groups (suggested by @rakyll / @jessfraz )

What's clear to me is that we need to scale and even though my proposal #18497 was to provide an official package for interoperating about (but not necessarily doing) X to quote @adg, I honestly don't care too much where it's going to live as long as it's clearly discoverable and have a thorough review process. Doing that outside of the Go project umbrella is extremely difficile and I think that's the heart of the issue.

At this point I think we could have 2 non exclusive strategies:

  1. The first one would look at what can we do to make godoc.org a more useful place to find high quality, non-std library packages, start on that work in an incremental manner.

  2. The other would be to create official interest groups which would have for mission to support "subsystems". Their role would go from maintaining and organizing a list of existing libraries. Defining standards and advocating for them. Code reviewing submitted 3rd party libraries and implementing interop solutions or actual features. These groups might benefit from current tooling and occasional access to core team members but wouldn't be restricted by release cycles and core team vetting.
    Off the top of my head, here are interest groups I can think of: media (audio/video), images, hardware, mobile, GUI, crypto, web services (similar to go-kit), dev tooling, machine intelligence... Having those groups defined, official and the process well documented should help us scale.
    That would obviously require a clearer delimitation between the standard library and non-standard packages. Really high quality packages that became standard don't ever have to "graduate" to the standard library but they have to be looked after by more than 1 owner (company or developer) and I believe those groups should be under the official Go umbrella.

If this idea sounds good, I would suggest to start by a couple special interest groups with momentum such as media and let them figure out how they want to get organized. I would assume an official team, repo, toolset and mission would be required. Then exposure with guidance are probably going to be the most valuable things that the project can provide to those teams.

What do you think?

@rakyll

This comment has been minimized.

Show comment
Hide comment
Member

rakyll commented Jan 3, 2017

/cc @spf13

@cep21

This comment has been minimized.

Show comment
Hide comment
@cep21

cep21 Jan 4, 2017

Contributor

Is there a longer, more abstract, article or general guidance around what code or libraries belong in Go's standard library?

Contributor

cep21 commented Jan 4, 2017

Is there a longer, more abstract, article or general guidance around what code or libraries belong in Go's standard library?

@nathany

This comment has been minimized.

Show comment
Hide comment
@nathany

nathany Jan 4, 2017

Contributor

"Once you have experience with the API being good, then it might make sense to promote to a subrepo or eventually the standard library" - @rsc #18497 (comment)

One big issue I see with this approach of "build it elsewhere, then promote it" is the licensing and CLA. If all contributors need to sign the Google CLA for inclusion into the subrepos/std, that not only involves a lot of administrative work, it could block promotion if some past contributors decide against it or are unreachable. (I don't know the remedy to that -- not a lawyer)

If the current system of subrepos is replaced with special interest groups (SIG) and the packages they support and recommend, this idea of promotion would only be relevant for standard library.

Then perhaps the licensing/CLA could be more relaxed (whatever each SIG chooses), with no expectation of the licensing or location of the code changing, or any of the overhead required to "promote" a library.

"all of these are on hold for #17244, which in turn is essentially blocked on understanding the long-term plan for package management. I intend to focus on that for the next six months." - @rsc

Thanks for putting so much time into this. I'm really looking forward to kicking the tires on the new dependency management tool once the initial version is made publicly available.

I ❤️ the Go standard library, especially net/http as compared to the myriad of http server and client options in other language communities I'm familiar with. Go makes it a lot easier to get started.

My long-term hope is that through a combination of processes (SIGs) and tooling, the Go community can reap many of the benefits of the "batteries included" (large std library) approach for a wide range of libraries developed by the community in a distributed fashion.

If it works extremely well, I could even imagine parts of the std moving to external SIG-supported libraries in Go 2.0 (i.e. image/*). An ambitious goal might be to make third-party package management so seamless and special interest groups so well organized that there no longer is a discernible advantage for a library to be in std vs. located elsewhere.

Contributor

nathany commented Jan 4, 2017

"Once you have experience with the API being good, then it might make sense to promote to a subrepo or eventually the standard library" - @rsc #18497 (comment)

One big issue I see with this approach of "build it elsewhere, then promote it" is the licensing and CLA. If all contributors need to sign the Google CLA for inclusion into the subrepos/std, that not only involves a lot of administrative work, it could block promotion if some past contributors decide against it or are unreachable. (I don't know the remedy to that -- not a lawyer)

If the current system of subrepos is replaced with special interest groups (SIG) and the packages they support and recommend, this idea of promotion would only be relevant for standard library.

Then perhaps the licensing/CLA could be more relaxed (whatever each SIG chooses), with no expectation of the licensing or location of the code changing, or any of the overhead required to "promote" a library.

"all of these are on hold for #17244, which in turn is essentially blocked on understanding the long-term plan for package management. I intend to focus on that for the next six months." - @rsc

Thanks for putting so much time into this. I'm really looking forward to kicking the tires on the new dependency management tool once the initial version is made publicly available.

I ❤️ the Go standard library, especially net/http as compared to the myriad of http server and client options in other language communities I'm familiar with. Go makes it a lot easier to get started.

My long-term hope is that through a combination of processes (SIGs) and tooling, the Go community can reap many of the benefits of the "batteries included" (large std library) approach for a wide range of libraries developed by the community in a distributed fashion.

If it works extremely well, I could even imagine parts of the std moving to external SIG-supported libraries in Go 2.0 (i.e. image/*). An ambitious goal might be to make third-party package management so seamless and special interest groups so well organized that there no longer is a discernible advantage for a library to be in std vs. located elsewhere.

@nathany nathany referenced this issue in fsnotify/fsnotify Jan 7, 2017

Open

Google bot for CLA checking #193

@glycerine

This comment has been minimized.

Show comment
Hide comment
@glycerine

glycerine Jan 12, 2017

Summary: a search engine that finds and ranks Go projects by code quality is being called for.

Summary: a search engine that finds and ranks Go projects by code quality is being called for.

@dgryski

This comment has been minimized.

Show comment
Hide comment
@dgryski

dgryski Jan 14, 2017

Contributor

Rust's bug tracking discoverability and ranking: rust-lang/rfcs#1824

Contributor

dgryski commented Jan 14, 2017

Rust's bug tracking discoverability and ranking: rust-lang/rfcs#1824

@rsc

This comment has been minimized.

Show comment
Hide comment
@rsc

rsc Jan 23, 2017

Contributor

As noted above, this issue is basically on hold for package management to settle. I think a lot of the pressure on the x repos is caused by bad broader support for package management.

Contributor

rsc commented Jan 23, 2017

As noted above, this issue is basically on hold for package management to settle. I think a lot of the pressure on the x repos is caused by bad broader support for package management.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment