Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: track tools/tooling updates to support modules #24661

Open
flibustenet opened this Issue Apr 3, 2018 · 63 comments

Comments

Projects
None yet
@flibustenet
Copy link

flibustenet commented Apr 3, 2018

Many Go tools that previously worked with GOPATH need to be upgraded in order to work with modules.

This is a tracking issue for the conversion of tools to work with modules

@gopherbot gopherbot added this to the vgo milestone Apr 3, 2018

@rsc rsc changed the title x/vgo how to works with tools like guru ? x/vgo: integrate with guru, goimports, etc Apr 3, 2018

@rsc

This comment has been minimized.

Copy link
Contributor

rsc commented Apr 3, 2018

Yes, this is an issue we're very aware of. Will leave this open to track it.

@myitcv

This comment has been minimized.

Copy link
Member

myitcv commented Apr 7, 2018

@rsc - just to check this issue effectively encapsulates what we discussed in golang-dev?

@myitcv myitcv changed the title x/vgo: integrate with guru, goimports, etc x/vgo: integrate with guru, goimports, and other tools Apr 9, 2018

@myitcv myitcv changed the title x/vgo: integrate with guru, goimports, and other tools x/vgo: integrate with guru, goimports, and other tools/tooling Apr 9, 2018

@myitcv

This comment has been minimized.

Copy link
Member

myitcv commented Apr 9, 2018

Changed the title so that this issue matches on use of the words "tool", "tools" or "tooling"

Will keep a list of tools here too (please edit):

  • Requiring build output:
    • guru
    • gocode
    • stringer
  • Requiring source code
@gopherbot

This comment has been minimized.

Copy link

gopherbot commented Apr 9, 2018

Change https://golang.org/cl/105855 mentions this issue: x/vgo: add deplist command to get build information about (test) deps

myitcv added a commit to myitcv/vgo that referenced this issue Apr 30, 2018

x/vgo: add deplist command to get build information about (test) deps
This CL is a work in progress for golang/go#24661. Following the pattern laid
down by the vet command, we want to pass the build details of the
transitive (test) deps of a list of packages onto vetters/linters etc
that need to load imports of said packages for go/types (and similar)
analysis.

Fixes golang/go#24661

Change-Id: If1496dd6a3ed501ad6f124226a05f5d57284c57d

myitcv added a commit to myitcv/vgo that referenced this issue May 20, 2018

x/vgo: add deplist command to get build information about (test) deps
This CL is a work in progress for golang/go#24661. Following the pattern laid
down by the vet command, we want to pass the build details of the
transitive (test) deps of a list of packages onto vetters/linters etc
that need to load imports of said packages for go/types (and similar)
analysis.

Fixes golang/go#24661

Change-Id: If1496dd6a3ed501ad6f124226a05f5d57284c57d
@rsc

This comment has been minimized.

Copy link
Contributor

rsc commented Jun 6, 2018

@rsc rsc modified the milestones: vgo, Go1.11 Jul 12, 2018

@rsc rsc added the modules label Jul 12, 2018

@rsc rsc changed the title x/vgo: integrate with guru, goimports, and other tools/tooling cmd/go: integrate with guru, goimports, and other tools/tooling Jul 12, 2018

@gopherbot

This comment has been minimized.

Copy link

gopherbot commented Jul 20, 2018

Change https://golang.org/cl/125296 mentions this issue: go/build: invoke go command to find modules during Import, Context.Import

gopherbot pushed a commit that referenced this issue Jul 28, 2018

go/build: invoke go command to find modules during Import, Context.Im…
…port

The introduction of modules has broken (intentionally) the rule
that the source code for a package x/y/z is in GOPATH/src/x/y/z
(or GOROOT/src/x/y/z). This breaks the code in go/build.Import,
which uses that rule to find the directory for a package.

In the long term, the fix is to move programs that load packages
off of go/build and onto golang.org/x/tools/go/packages, which
we hope will eventually become go/packages. That code invokes
the go command to learn what it needs to know about where
packages are.

In the short term, though, there are lots of programs that use go/build
and will not be able to find code in module dependencies.
To help those programs, go/build now runs the go command to
ask where a package's source code can be found, if it sees that
modules are in use. (If modules are not in use, it falls back to the
usual lookup code and does not invoke the go command, so that
existing uses are unaffected and not slowed down.)

Helps #24661.
Fixes #26504.

Change-Id: I0dac68854cf5011005c3b2272810245d81b7cc5a
Reviewed-on: https://go-review.googlesource.com/125296
Reviewed-by: Michael Matloob <matloob@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
@myitcv

This comment has been minimized.

Copy link
Member

myitcv commented Aug 1, 2018

@alandonovan @ianthehat - is it worth creating a number of issues, specific to each tool that's being worked on?

I keep directing people to this umbrella issue... but maybe there's merit in getting a bit more granularity on things? Equally, maybe not.

So I'm definitely wouldn't push back if you said "let's keep things here" 👍

@sam3d

This comment has been minimized.

Copy link

sam3d commented Aug 2, 2018

How about the case of goimports importing packages that are both:

  • Within the module's local directory
  • Outside of the system scoped $GOPATH

Assuming the following directory structure:

./
├── go.mod
├── main.go
└── test
    └── test.go

With ./go.mod containing:

module example.com

goimports will happily leave alone and even format ./main.go:

package main

import (
	"log"

	"example.com/test"
)

func main() {
	log.Print("Hello, world!")
	test.Start()
}

With ./test/test.go:

package test

import "fmt"

func Start() {
	fmt.Println("Started properly")
}

However, if the "example.com/test" import is removed from main.go, it wont get added or found again. If its import path is changed in any way to render it incorrect (say to "example.com/tes"), goimports simply deletes the line.

It's odd - goimports can clearly tell that package test exists when it's been provided already, but it can't actively seek and add it if it hasn't been.

Is this something that requires looking into the module cache for, or is this a different issue?

@ianthehat

This comment has been minimized.

Copy link

ianthehat commented Aug 2, 2018

goimports will assume that the import line "example.com/test" imports a package called test if it cannot find any sources for that package, and it knows the test symbol is used, so it won't delete the line. It does not mean that it knew the package existed...

goimports scans your GOPATH and GOROOT, anything not there it will not find right now.

It will be made to use x/tools/go/packages plus some extra stuff that is beyond the scope of that package. When it does, this case will work (does not involve the module cache, just the list of packages in the current module, which you can already get with go mod -packages

@sam3d

This comment has been minimized.

Copy link

sam3d commented Aug 2, 2018

Thank you for explaining @ianthehat! I've misunderstood how goimports handles symbols.

@metakeule

This comment has been minimized.

Copy link

metakeule commented Aug 11, 2018

I have two questions:

  1. will x/tools/go/packages move to the standard library with Go 1.12?
  2. I can't see any information about the version of a plugin, so how is a tool that assists in importing (like GoSublime) supposed to get a list of possible imports (with versions)?
@metakeule

This comment has been minimized.

Copy link

metakeule commented Aug 11, 2018

Also: How to prevent that import completion will get really slow with larger projects? (see mdempsky/gocode#32)

@myitcv

This comment has been minimized.

Copy link
Member

myitcv commented Aug 11, 2018

@metakeule

will x/tools/go/packages move to the standard library with Go 1.12?

The plan is for it to move to the standard library at some point, yes, once it has stabilised (ref).

I can't see any information about the version of a plugin

This is similar to the goimports "problem"; see #26717

How to prevent that import completion will get really slow with larger projects?

Many of the problems describe in mdempsky/gocode#32 relate to speed problems brought about by the use of the -source flag, which drives the use of the source importer. go/packages will work using the underlying build cache and so should not suffer these problems.

But in case lag does become an issue for gocode with its current architecture, go/packages will be offering support for long-running programs. So it's conceivable that gocode's cache will switch to be module-based, where it effectively "watches" for relevant changes and refreshes its cache accordingly. Its cache will then be primed for fast responses to ad-hoc queries.

cc @dominikh (in case there's any overlap of interest here with your work)

@tbarbugli

This comment has been minimized.

Copy link

tbarbugli commented Jan 3, 2019

errcheck added support for go modules on v1.2.0 🎉

@mdempsky

This comment has been minimized.

Copy link
Member

mdempsky commented Jan 11, 2019

FYI, something that caught me by surprise in the transition of mdempsky/unconvert from go/loader to go/packages was that the go/token.File.Name() for cgo-rewritten sources now points into the go-build cache instead of pointing to the pre-cgo source file, which meant that unconvert -apply could corrupt the build cache instead of rewriting the original source files.

Heads up to any other authors of automated-rewrite tools to be careful about this.

On the upside, I can confirm that unconvert -apply will fully work with cgo-processed source files starting with Go 1.12, thanks to the fix for #26745.

@andig

This comment has been minimized.

Copy link

andig commented Jan 12, 2019

Is there any update on the stringer issue? The patchset is from august?

@npgm

This comment has been minimized.

Copy link

npgm commented Jan 16, 2019

Noticed that golang/mock is not being tracked here: golang/mock#230

@F21

This comment has been minimized.

Copy link

F21 commented Feb 7, 2019

The latest master of stringer works with modules now.

@josharian

This comment has been minimized.

Copy link
Contributor

josharian commented Feb 23, 2019

I’m working on migrating go-fuzz to use go/packages. It is slow going. :)

Once that is done, there will probably be some further steps to properly support modules; I’m not sure yet.

@josharian

This comment has been minimized.

Copy link
Contributor

josharian commented Mar 1, 2019

go-fuzz PR for go/packages: dvyukov/go-fuzz#211. As anticipated, I believe there’s still more to do for module support. I don’t currently plan to work on that.

@ptone

This comment has been minimized.

Copy link

ptone commented Mar 10, 2019

adding a reference to src-d/proteus#107 to track proteus

@ramya-rao-a

This comment has been minimized.

Copy link

ramya-rao-a commented Mar 12, 2019

@ianthehat Can we add https://github.com/godoctor/godoctor to the list being maintained here?

@atombender

This comment has been minimized.

Copy link

atombender commented Mar 14, 2019

go-sumtype works with modules now.

@kush-patel-hs

This comment has been minimized.

Copy link

kush-patel-hs commented Mar 23, 2019

Any update for goreturns? so VSCode can work out of box way it used to in $GOPATH mode instead of having to change the config to point to goimports

@wingyplus

This comment has been minimized.

Copy link
Contributor

wingyplus commented Mar 24, 2019

How about gomvpkg? Is there still support on module?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.