cmd/go: track tools/tooling updates to support modules

[flibustenet]

Many Go tools that previously worked with GOPATH need to be upgraded in order to work with modules.

This is a tracking issue for the conversion of tools to work with modules

• aligncheck - Merged - opennota/check#43
• counterfeiter - maxbrunsfeld/counterfeiter#90
• deadcode
• delve - Working
• dlv
• dupl
• errcheck - Beta - kisielk/errcheck#148
• fillstruct
• gas
• go-langserver
• go-outline - Working
• go-symbols
• gocode - Beta - https://github.com/stamblerre/gocode
• goconst
• gocyclo
• godef - Beta - https://github.com/ianthehat/godef
• godoc
• gofmt - Working
• go-fuzz
• gogetdoc - Merged zmb3/gogetdoc#40
• goimports - #27287
• golint - Beta - https://go-review.googlesource.com/c/lint/+/127695
• gometalinter
• gomodifytags
• gopkgs
• goplay
• gorename - (#27571)
• goreturns
• gosimple
• gotags
• gotests
• gotype
• gotype-live
• govet
• guru
• impl
• ineffassign
• interfacer
• keyify
• lll - Working
• maligned
• megacheck
• misspell
• motion
• nakedret
• safesql
• staticcheck
• stringer - Beta - https://go-review.googlesource.com/c/tools/+/126535
• structcheck - Merged - opennota/check#43
• unconvert
• unparam
• unused
• varcheck - Merged - opennota/check#43

[rsc]

Yes, this is an issue we're very aware of. Will leave this open to track it.

[rsc]

Yes, this is an issue we're very aware of. Will leave this open to track it.

[myitcv]

@rsc - just to check this issue effectively encapsulates what we discussed in golang-dev?

[myitcv]

@rsc - just to check this issue effectively encapsulates what we discussed in golang-dev?

[myitcv]

Changed the title so that this issue matches on use of the words "tool", "tools" or "tooling"

Will keep a list of tools here too (please edit):

• Requiring build output:
  • guru
  • gocode
  • stringer
• Requiring source code
  • goimports
  • godef
  • godoc.org (golang/gddo#567)

[myitcv]

Changed the title so that this issue matches on use of the words "tool", "tools" or "tooling"

Will keep a list of tools here too (please edit):

• Requiring build output:
  • guru
  • gocode
  • stringer
• Requiring source code
  • goimports
  • godef
  • godoc.org (golang/gddo#567)

[gopherbot]

Change https://golang.org/cl/105855 mentions this issue: x/vgo: add deplist command to get build information about (test) deps

[gopherbot]

Change https://golang.org/cl/105855 mentions this issue: x/vgo: add deplist command to get build information about (test) deps

[rsc]

See https://go-review.googlesource.com/c/tools/+/116359 for a sketch of a start.
/cc @alandonovan

[rsc]

See https://go-review.googlesource.com/c/tools/+/116359 for a sketch of a start.
/cc @alandonovan

[gopherbot]

Change https://golang.org/cl/125296 mentions this issue: go/build: invoke go command to find modules during Import, Context.Import

[gopherbot]

Change https://golang.org/cl/125296 mentions this issue: go/build: invoke go command to find modules during Import, Context.Import

[myitcv]

@alandonovan @ianthehat - is it worth creating a number of issues, specific to each tool that's being worked on?

I keep directing people to this umbrella issue... but maybe there's merit in getting a bit more granularity on things? Equally, maybe not.

So I'm definitely wouldn't push back if you said "let's keep things here" 👍

[myitcv]

@alandonovan @ianthehat - is it worth creating a number of issues, specific to each tool that's being worked on?

I keep directing people to this umbrella issue... but maybe there's merit in getting a bit more granularity on things? Equally, maybe not.

So I'm definitely wouldn't push back if you said "let's keep things here" 👍

[sam3d]

How about the case of goimports importing packages that are both:

• Within the module's local directory
• Outside of the system scoped $GOPATH

---

Assuming the following directory structure:

./
├── go.mod
├── main.go
└── test
    └── test.go

With ./go.mod containing:

module example.com

goimports will happily leave alone and even format ./main.go:

package main

import (
	"log"

	"example.com/test"
)

func main() {
	log.Print("Hello, world!")
	test.Start()
}

With ./test/test.go:

package test

import "fmt"

func Start() {
	fmt.Println("Started properly")
}

---

However, if the "example.com/test" import is removed from main.go, it wont get added or found again. If its import path is changed in any way to render it incorrect (say to "example.com/tes"), goimports simply deletes the line.

It's odd - goimports can clearly tell that package test exists when it's been provided already, but it can't actively seek and add it if it hasn't been.

Is this something that requires looking into the module cache for, or is this a different issue?

[sam3d]

How about the case of goimports importing packages that are both:

• Within the module's local directory
• Outside of the system scoped $GOPATH

---

Assuming the following directory structure:

./
├── go.mod
├── main.go
└── test
    └── test.go

With ./go.mod containing:

module example.com

goimports will happily leave alone and even format ./main.go:

package main

import (
	"log"

	"example.com/test"
)

func main() {
	log.Print("Hello, world!")
	test.Start()
}

With ./test/test.go:

package test

import "fmt"

func Start() {
	fmt.Println("Started properly")
}

---

However, if the "example.com/test" import is removed from main.go, it wont get added or found again. If its import path is changed in any way to render it incorrect (say to "example.com/tes"), goimports simply deletes the line.

It's odd - goimports can clearly tell that package test exists when it's been provided already, but it can't actively seek and add it if it hasn't been.

Is this something that requires looking into the module cache for, or is this a different issue?

[ianthehat]

goimports will assume that the import line "example.com/test" imports a package called test if it cannot find any sources for that package, and it knows the test symbol is used, so it won't delete the line. It does not mean that it knew the package existed...

goimports scans your GOPATH and GOROOT, anything not there it will not find right now.

It will be made to use x/tools/go/packages plus some extra stuff that is beyond the scope of that package. When it does, this case will work (does not involve the module cache, just the list of packages in the current module, which you can already get with go mod -packages

[ianthehat]

goimports will assume that the import line "example.com/test" imports a package called test if it cannot find any sources for that package, and it knows the test symbol is used, so it won't delete the line. It does not mean that it knew the package existed...

goimports scans your GOPATH and GOROOT, anything not there it will not find right now.

It will be made to use x/tools/go/packages plus some extra stuff that is beyond the scope of that package. When it does, this case will work (does not involve the module cache, just the list of packages in the current module, which you can already get with go mod -packages

[sam3d]

Thank you for explaining @ianthehat! I've misunderstood how goimports handles symbols.

[sam3d]

Thank you for explaining @ianthehat! I've misunderstood how goimports handles symbols.

[metakeule]

I have two questions:

1. will x/tools/go/packages move to the standard library with Go 1.12?
2. I can't see any information about the version of a plugin, so how is a tool that assists in importing (like GoSublime) supposed to get a list of possible imports (with versions)?

[metakeule]

I have two questions:

1. will x/tools/go/packages move to the standard library with Go 1.12?
2. I can't see any information about the version of a plugin, so how is a tool that assists in importing (like GoSublime) supposed to get a list of possible imports (with versions)?

[metakeule]

Also: How to prevent that import completion will get really slow with larger projects? (see mdempsky/gocode#32)

[metakeule]

Also: How to prevent that import completion will get really slow with larger projects? (see mdempsky/gocode#32)

[myitcv]

@metakeule

will x/tools/go/packages move to the standard library with Go 1.12?

The plan is for it to move to the standard library at some point, yes, once it has stabilised (ref).

I can't see any information about the version of a plugin

This is similar to the goimports "problem"; see #26717

How to prevent that import completion will get really slow with larger projects?

Many of the problems describe in mdempsky/gocode#32 relate to speed problems brought about by the use of the -source flag, which drives the use of the source importer. go/packages will work using the underlying build cache and so should not suffer these problems.

But in case lag does become an issue for gocode with its current architecture, go/packages will be offering support for long-running programs. So it's conceivable that gocode's cache will switch to be module-based, where it effectively "watches" for relevant changes and refreshes its cache accordingly. Its cache will then be primed for fast responses to ad-hoc queries.

cc @dominikh (in case there's any overlap of interest here with your work)

[myitcv]

@metakeule

will x/tools/go/packages move to the standard library with Go 1.12?

The plan is for it to move to the standard library at some point, yes, once it has stabilised (ref).

I can't see any information about the version of a plugin

This is similar to the goimports "problem"; see #26717

How to prevent that import completion will get really slow with larger projects?

Many of the problems describe in mdempsky/gocode#32 relate to speed problems brought about by the use of the -source flag, which drives the use of the source importer. go/packages will work using the underlying build cache and so should not suffer these problems.

But in case lag does become an issue for gocode with its current architecture, go/packages will be offering support for long-running programs. So it's conceivable that gocode's cache will switch to be module-based, where it effectively "watches" for relevant changes and refreshes its cache accordingly. Its cache will then be primed for fast responses to ad-hoc queries.

cc @dominikh (in case there's any overlap of interest here with your work)

[ianthehat]

I have modified this issue to track the full list of tools, and assigned it to me.
I did this rather than file a new issue because it seems this issue is already referenced in quite a few places. I apologies to @flibustenet for deleting all your words, but this seems like the most practical approach.
Feel free to file individual tracking issues for each tool where needed, and to add any tools not in the list.
We will continue to update this issue moving forwards.

[ianthehat]

I have modified this issue to track the full list of tools, and assigned it to me.
I did this rather than file a new issue because it seems this issue is already referenced in quite a few places. I apologies to @flibustenet for deleting all your words, but this seems like the most practical approach.
Feel free to file individual tracking issues for each tool where needed, and to add any tools not in the list.
We will continue to update this issue moving forwards.

[agnivade]

@ianthehat - Could you take a look at #26827 and close it if you think it falls under this issue ?

[agnivade]

@ianthehat - Could you take a look at #26827 and close it if you think it falls under this issue ?

[aarzilli]

dlv

Delve 1.1.0 works with Go 1.11 and as far as I know doesn't have any problem with modules. I just tried with a toy example and everything seemed to work. If you are aware of something I'd like to know.
It just calls go build and uses the output executable, btw.

[aarzilli]

dlv

Delve 1.1.0 works with Go 1.11 and as far as I know doesn't have any problem with modules. I just tried with a toy example and everything seemed to work. If you are aware of something I'd like to know.
It just calls go build and uses the output executable, btw.

[ianthehat]

@agnivade : godoc is a special beast with all sorts of module issues.
We intend to fix godoc (or an alternative) so that the usage an IDE makes of it works as part of this issue, and possibly fixing the single session local web server variant, but fully fixing godoc in all it's flavours to be module and version aware is a whole separate task, and probably needs its own issue outside of this one.

[ianthehat]

@agnivade : godoc is a special beast with all sorts of module issues.
We intend to fix godoc (or an alternative) so that the usage an IDE makes of it works as part of this issue, and possibly fixing the single session local web server variant, but fully fixing godoc in all it's flavours to be module and version aware is a whole separate task, and probably needs its own issue outside of this one.

[e-dard]

I have been using a next branch on staticcheck that seems to be working very well with modules. Maybe link it on the list on the PR description?

[e-dard]

I have been using a next branch on staticcheck that seems to be working very well with modules. Maybe link it on the list on the PR description?