Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

proposal: cmd/go: keep GOPATH mode #37755

Open
rcoreilly opened this issue Mar 9, 2020 · 11 comments
Open

proposal: cmd/go: keep GOPATH mode #37755

rcoreilly opened this issue Mar 9, 2020 · 11 comments

Comments

@rcoreilly
Copy link

@rcoreilly rcoreilly commented Mar 9, 2020

This proposal is simply to retain the existing GOPATH functionality, along with the go.mod modules.

The rationale is that GOPATH provides an ideal mode of operation for developing across multiple different repositories at the same time. It is otherwise cumbersome to do so with go.mod, requiring e.g., temporary insertion and removal of replace directives (see discussion in #26640).

A major design priority for Go is simplicity, and GOPATH is simple. It provides a great way for new users to explore lots of different code packages easily, browse, modify, experiment, etc.

There are many different types of Go users, but it would be great to retain support for the "hobbyist" / non-corporate hacker who just wants to do things simply and doesn't care about living on the bleeding edge.

It should not be so hard for the two modes to coexist, as the tooling etc is currently supporting both, and I personally find it most productive to develop in GOPATH mode and then have some Makefile targets that ensure go.mod still works for those who want to use that. And all official releases are done in go.mod.

Thus, it may actually be optimal to have both modes available, instead of trying to complicate go.mod so that it does what GOPATH already does so well.

@gopherbot gopherbot added this to the Proposal milestone Mar 9, 2020
@gopherbot gopherbot added the Proposal label Mar 9, 2020
@dmitshur dmitshur changed the title Proposal: keep GOPATH proposal: cmd/go: keep GOPATH mode Mar 9, 2020
@JAicewizard
Copy link

@JAicewizard JAicewizard commented Mar 9, 2020

although I do disagree, the editing repos locally is really nice

@dmitshur
Copy link
Member

@dmitshur dmitshur commented Mar 9, 2020

It should be helpful to identify more precisely and write down what types of workflows are currently easier to achieve in GOPATH mode than in module mode. That would make it possible to investigate if the module mode can be improved so that those workflows do not have more overhead in module mode compared to GOPATH mode.

@jayconrod
Copy link
Contributor

@jayconrod jayconrod commented Mar 9, 2020

I don't think we should maintain GOPATH indefinitely. Ensuring that tools work in both modes isn't free. It also requires time and effort from project authors to test their code and support users in both modes.

I'd much prefer we improve modules so that this workflow is better supported. Editing and releasing multiple modules has been a problem, and we should figure out a solution to.

As @dmitshur said, it would be helpful to explain your workflow and exactly why it doesn't work well with modules. Please consider adding an experience report.

@rcoreilly
Copy link
Author

@rcoreilly rcoreilly commented Mar 9, 2020

I think the issues are well documented in #26640 -- here's my summary:

  • Say you have 3 different repos, each depending on the other: A -> B -> C

  • And you are in the early development stage, so the API in A & B is potentially changing all the time.

  • With go.mod, you have to add a replace directive in B and C to use the current live source for A and B, but then for each commit (if others are to be able to use the code) you have to remove those replace directives and do the update to current, and remember to do the commit and push in the proper sequential order (A then B then C), and then re-insert the replace directives before continuing to edit.

  • This discourages frequent, well-documented commits, and encourages errors due to forgetting any of these steps. If you forget to re-insert the replaces, then you can't figure out why changes in A or B are not showing up in C, etc. It is frustrating.

So, while this is not the absolute worst situation in the world, it is a lot of seemingly unnecessary overhead, given the pure transparent simplicity of GOPATH.

With GOPATH, all core developers could just use that mode, and either work in a branch and do nothing at all at commits, or require users to check out tagged releases to get reliable behavior (or use GOPATH when using the latest HEAD), or have a simple Makefile target that updates go.mod to latest revision.

In short: there are different modes of working, one where you're relying on stable existing package API's and want reliability and stability, and another where everything is changing and the "live source" model makes more sense, so supporting both modes separately may make more sense than trying to find one solution that works for both, when each has very different fundamental requirements.

@chewxy
Copy link

@chewxy chewxy commented Mar 9, 2020

I would like to chime in with a real life example:

Context

I develop a family of machine learning and deep learning libraries - gorgonia. Roughly speaking the libraries function as such:

  • gorgonia.org/tensor - Generic multidimensional array
  • gorgonia.org/gorgonia - symbolic version of (i.e. AST of the expressions in) gorgonia.org/tensor
  • gorgonia.org/cu - CUDA.

These libraries each do similar but different things. For example, a MatMul in tensor performs the matrix multiplication. In gorgonia it may perform a matrix multiplication, or it may generate a symbolic representation of the matrix multiplication. In cu, the matrix multiplication is done in the graphics card.

Since they do similar but different things, it would be useful to have similar APIs in the packages. The versions are kept in sync, so all the packages have similar APIs at any given version (well, that's the goal).

The dependencies look like this:

gorgonia -> tensor
         -> cu

Despite gorgonia being dependent on tensor, it's often gorgonia that determines the APIs of tensor. This is done, factoring developer userfriendliness.

The Problem

Currently with Go modules, I have found it quite difficult to simultaneously work on all three libraries at the same time. My solution thus far has been to just use GO111MODULE=off.

Now I think Go modules are one of the more brilliant things about Go. However, it isn't very clear how to simultaneously work on newer versions of the libraries.

Other methods I have tried:

Work using Go modules. Say I am working on A and B at the same time, and B depends on A. I first add the APIs to A, publish, then add the latest version as a dependency of B.

This approach works but generates a lot of superfluous version numbers. This is especially true when working to discover the best most userfriendly APIs. Often it's the API of B that determines the API of A, despite the inversion of library dependencies.

Conclusion

It's difficult to work simultaneously on multiple co-dependent library (co-dependent in the abstract sense - the actual dependencies are unidirectional). GOPATH helps with that.

I view GOPATH as the "working bench" before publishing

@kortschak
Copy link
Contributor

@kortschak kortschak commented Mar 10, 2020

It seems to me that the local go.mod idea described in #26640 would address this.

@bcmills
Copy link
Member

@bcmills bcmills commented Mar 10, 2020

This proposal does not seem actionable: it is proposing that we not-do something (removing support for GOPATH) which we have not yet proposed to do anyway (#4719 notwithstanding).

Moreover, I think the emphasis on #26640 is telling: note that that issue is still open. It seems premature to prescribe a course of action based on this assumption that that issue, or its underlying use-case, will not be addressed.

(But #26640 itself seems like a bit of a red herring: the real issue to focus on here is the use-case, not the mechanism. #26640 describes one possible mechanism, but I believe the underlying use-case is #27542, and it may turn out that we can better address it by some other mechanism.)

@rcoreilly
Copy link
Author

@rcoreilly rcoreilly commented Mar 11, 2020

Wouldn't it be good to have a clear plan going forward? If there is a clear plan to retain GOPATH, then it seems like that would change the way that these other issues are addressed?

According to the oft-cited principle of feature-orthogonality, wouldn't it make sense to not try to shoe-horn GOPATH into go.mod, as in #27542 and #26640, and instead focus on doing whatever might be necessary to make the two modes of operation (GOPATH and go.mod) more cleanly supported and well-documented etc (e.g., the GO111MODULE env variable is likely a suboptimal way of switching between modes -- perhaps a command-line switch instead or at least in addition? And at least a better name for the env variable?)

From this perspective, the discussion in #27542 seems to support this approach, as it was largely focused on essentially just replicating GOPATH under go.mod. Again, the proposal here is to just embrace these two modes as serving largely different use-cases, and adopt the principle of orthogonality, etc. Also, again repeating myself, GOPATH is really really simple, and that is a major advantage over trying to replicate its functionality in some more complicated way on top of go.mod. Let go.mod do what it does best, and GOPATH do what it does best, and forget about all the replace directives etc.

@bitfield
Copy link

@bitfield bitfield commented Mar 12, 2020

I think this would be a big mistake. Go is often criticised for having more than one way to do certain things (such as declaring variables), and I take that point. For beginners, it's much less confusing if there's one straightforward, standard, official way to do X.

Once upon a time, the standard way was GOPATH. Now it's modules. Don't look back, look forward. If a few people decide to stop using Go because they loved GOPATH so much, I personally can live with that.

@komuw
Copy link
Contributor

@komuw komuw commented Mar 12, 2020

2017-survey

2016-survey

Users have expressed directly to the Go team, more than once[1][2], that they are unhappy with GOPATH. Go modules were put in place, in part, to solve that 'unhappiness'.
I do not think we should be keeping GOPATH mode once modules are deemed stable & complete.
If there's any deficiency in modules(that is currently been met by GOPATH); we should fix that rather than keeping GOPATH.

reference:

  1. https://blog.golang.org/survey2016-results
  2. https://blog.golang.org/survey2017-results
@ohir
Copy link

@ohir ohir commented Mar 12, 2020

@komuw

Users have expressed directly to the Go team, more than once that they are unhappy with GOPATH.

Oh! An astounding one percent of users were unhappy. I would like to see how many users with over a year of Go experience who used Go at work complained about GOPATH. If there was more than a single such person, ofc.

Many expressed a need for better dependency management tools. But not one I know had wished for GOPATH to be fierce slayed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Proposals
Awaiting triage
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
You can’t perform that action at this time.