Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

proposal: Better dependency management #41510

Closed
shirshendubhowmick opened this issue Sep 20, 2020 · 6 comments
Closed

proposal: Better dependency management #41510

shirshendubhowmick opened this issue Sep 20, 2020 · 6 comments

Comments

@shirshendubhowmick
Copy link

@shirshendubhowmick shirshendubhowmick commented Sep 20, 2020

Even after the introduction of Go modules, the dependency management is complex and not so developer friendly. For beginners it takes a good amount of time to understand what's going on behind the scene.

Some of this may be because of lack of well written documentation about dependency management.

  • There are some confusing syntax, for example
    github.com/myorganzation/mypackage/pkg

    This url results in 404 in browsers but somehow go resolves it, so it seems like depending on hosting providers such as GitHub, BitBucket etc go have different mechanisms of resolving the URL.

    Which is somewhat described in here

  • Upgrading to major version with a suffix like vX

  • Though i am not sure about this but i didn't find any way by which i can tell that this indirect dependency x is from the dependency y, just by looking into go.sum or go.mod files.

  • Error messages not being so helpful
    If i try to go get a package in a directory which is not a module i get an error message which is not so helpful for beginners

go get github.com/gofiber/fiber/v2

cannot find package "github.com/gofiber/fiber/v2" in any of:
	/usr/local/go/src/github.com/gofiber/fiber/v2 (from $GOROOT)
	/Users/shirshendubhowmick/go/src/github.com/gofiber/fiber/v2 (from $GOPATH)
  • There is no easy way of adding dev only dependencies.

Gophers please do let me know your thoughts about this, also request you to think about these issues from a beginners perspective especially if someone is coming from JavaScript or Python background.

I feel like there is a steep learning curve for go dependency management, which can be made easy with little changes in go and it's documentation.

Edit: Adding some suggestion to deal with the problems I mentioned above
Some high level suggestion to deal with the current problems

  • Problem 1: The current way of downloading, using & maintaining a package

    I like the idea of not having a central registry like npm or pip. However using repo URLs (that is also some modified URL) everywhere in the codebase to import the package doesn't seem to be the best way.

    Instead what we can do is use git URLs git+ssh://git@github.com/myorganization/mypackage (HTTPS url works too), only at one place, i.e. our dependency file (currently go.mod)
    With this, we can also refer to a particular branch, tag or commit.

    Now our dependency file (currently go.mod) will have a mapping of module name and its URL to create an alias for the module, for example
    mypackage git+ssh://git@github.com/myorganization/mypackage

    Everywhere in the code base a consumer will use the alias name instead of a URL to import the package, for example

    import "mypackage"

    Now how do we know where the module is located inside the repo ? Right now we add a go.mod file in every module root directory. With this change maybe we can add a single file in the repo root which will tell where the modules are located relative to the repo root. I guess this will give more flexibility to both the module developer and consumer.

    For updating version, we can either do it manually by changing the URL or maybe via some tool like go update mypackage

  • Problem 2: Improving the error messages
    This might be a simpler problem to solve compared to the above one. There is no specific solution to this. I think the best way is to run an audit to figure out point of failures while working with modules in go. And try to have as much meaningful error message as possible with some detailed log.

This is very high level solution proposal, happy to discuss more on it and also pros & cons, gotchas, bottlenecks etc.

@gopherbot gopherbot added this to the Proposal milestone Sep 20, 2020
@gopherbot gopherbot added the Proposal label Sep 20, 2020
@mvdan
Copy link
Member

@mvdan mvdan commented Sep 20, 2020

Proposals need to propose a very specific change to Go, and be detailed in doing so. You seem to be proposing a very large set of changes (entirely replacing modules?), but I don't see what you're actually suggesting that we do. The proposal reads more like a list of problems than any proposed changes.

Personally, I think this is better suited for one of the forums like golang-nuts. A proposal should only be filed once you have a very specific suggestion in mind.

@shirshendubhowmick
Copy link
Author

@shirshendubhowmick shirshendubhowmick commented Sep 20, 2020

Sure will i edit my original post to include some change proposal with detailed description.

For now, I am not in favour replacing modules entirely but making some changes in the existing module system to make it more developer friendly.

@shirshendubhowmick
Copy link
Author

@shirshendubhowmick shirshendubhowmick commented Sep 20, 2020

@mvdan Added some some suggestions to deal with problems i mentioned

@ianlancetaylor
Copy link
Contributor

@ianlancetaylor ianlancetaylor commented Sep 21, 2020

The issue tracker is not the right place to develop a new approach to dependencies in Go, because the issue tracker is not a good place for complex conversations. I strongly suggest taking this to the golang-nuts group.

@mvdan
Copy link
Member

@mvdan mvdan commented Sep 21, 2020

Thanks. Closing this in favor of that thread, for now.

@mvdan mvdan closed this Sep 21, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
4 participants
You can’t perform that action at this time.