Support Go modules

[jbrodriguez]

Now that vgo is available (https://research.swtch.com/vgo-tour), vscode-go will remove imports flag errors because it doesn't understand the module (go.mod) concept, assuming you've configured golint to do that.

Other issues will come up as well and the only workaround I can think of is disable lint modules (possibly).

I'm putting this out there since this is something we will definitely require .

[alfred-zhong]

I read the blog and I like vgo, too. But I don't know if vgo should be supported soon.
Although vgo brings very good thoughts, it is still new and experimental. Also it hasn't been bringing into the offical toolchain (Maybe soon in go 1.11 or 1.12, I don't know). Now if you want to use vgo, the get and build commands should also be called from "vgo" command, but not the "go".

[nicpottier]

There's a bit of a chicken and egg problem here. To be honest I'd be using vgo today on my projects if I had support for it in vscode, just so much clearer and faster than dep, glide and co.. Happy to contribute if people want to point me in the right direction.

[jancona]

@nicpottier I've found that I can use vscode with vgo by placing my code at the proper GOPATH location (even though vgo doesn't use GOPATH) and using vgo vendor to put all my dependencies in the vendor/ directory. That way the regular go tools that vscode uses still work, and I can use vgo commands from the command line. By using vgo vendor I don't have to go get my dependencies.

[hackervera]

Looks like vgo is more serious now https://blog.golang.org/versioning-proposal

[illegalnumbers]

What would be needed to move forward on this? How does vscode-go map dependencies and how can we add that path matching on to this extension? Would love to help out.

[tyhi]

The vgo proposal was accepted earlier.

golang/go#24301 (comment)

[ramya-rao-a]

I dont have the expertise to work on this, but I can help anyone who wants to give it a shot.

@nicpottier, @illegalnumbers The Go extension does not maintain any maps of dependencies. It uses go build to get the compile time errors if any and then uses regular expressions to extract the filename, line and column numbers from the error message to create the errors that you see in the editor. See https://github.com/Microsoft/vscode-go/blob/0.6.80/src/goBuild.ts#L51

[nicpottier]

As far as I can tell the error format won't change with vgo, but what will is code navigation.. IE the path to find a specific package is no longer just $GO_PATH/vendor but a version specific path instead. Do you know where that kind of resolving is taking place?

[jrick]

vgo uses its own algorithm to determine the version, and the modules are extracted to $GOPATH/src/v/modulepath@version. You can see all of the currently-resolved modules and versions with vgo list -m.

[nicpottier]

Ah, right you are.. a sample:

% vgo list -m
github.com/jmespath/go-jmespath                v0.0.0-20160202185014-0b12d6b521d8
github.com/jmoiron/sqlx                        v0.0.0-20180406164412-2aeb6a910c2b
github.com/koding/multiconfig                  v0.0.0-20171124222453-69c27309b2d7
github.com/lib/pq                              v0.0.0-20180201184707-88edab080323

So the right version of pq is found in:

$GOPATH/src/v/github.com/lib/pq@v0.0.0-20180201184707-88edab080323

[jrick]

Note that modules can encompass multiple packages, so the github.com/lib/pq/certs package for example is part of the pq module since it shares the same longest module prefix of any referenced module.

Perhaps vgo should gain a way to query for just the resolved module, version, and perhaps even source directory of any arbitrary package. This would making integration with editors and IDEs simpler.

[nicpottier]

Agreed, might be useful to know from the vscode folks where the current code is for resolving these. Would make sure that whatever is added to vgo would satisfy that requirement.

[ramya-rao-a]

@nicpottier

For code navigation we use tools like godef and gogetdoc. The extension itself doesnt do any package resolution.

The only place the extension explicitly deals with vendors is to provide the Go: Add Import feature which lists all the packages available for import for the current file. Say there is a vendor package at github.com/me/myproject/vendor/github.com/you/yourproject folder. In a file github.com/me/myproject/main.go, this package will appear as github.com/you/yourproject when the Go: Add Import feature is invoked. Will this be affected by vgo?

[zmb3]

Hey there @ramya-rao-a - long time no see :-)

Will this be affected by vgo?

I'm no expert, but I believe the vendoring stuff will be affected by vgo, as vgo does away with vendor directories and instead caches versioned dependencies outside the project.

[nicpottier]

Awesome, thanks that's clear with respect to godef and gogetdoc I don't think either of those tools have support for vgo tools yet, but that gives us a clear path forward.

For Add Import seems like using vgo list -m would probably give us the right starting point to figure out which packages are available for import.

@zmb3 I think vgo still supports vendoring.. at least the latest vgo has a % vgo vendor command which builds a local vendor directory for you.

[zmb3]

vgo vendor exists for backwards compatibility, producing a “legacy” project with a vendor directory that must be built within GOPATH.

In day to day operation a project that uses vgo will not have a vendor directory.

At least that’s my understanding based on the backwards compatibility section of the tour here: https://research.swtch.com/vgo-tour

[nicpottier]

@zmb3 Ah, right you are!

[ramya-rao-a]

@zmb3 Nice to see you too :)

cc @uudashr whose version of gopkgs tool is what we use for the Go: Add Imports feature.

[uudashr]

Ya... haven't look into details about vgo.
FYI gopkgs used for:

1. Autocomplete unimported packages
2. Add imports

For brief:
gopkgs created for the sake of speed
Ex:

# time gopkgs
gopkgs  0.32s user 0.59s system 153% cpu 0.598 total

# time go list all
go list all  6.43s user 4.03s system 108% cpu 9.647 total

@zmb3 @nicpottier feel free to hac gopkgs

[RichyHBM]

Hello, @ramya-rao-a @uudashr have there been any updates with regards to this issue?

[ramya-rao-a]

@RichyHBM Not that I know of.

Looks like as long as the tools used by the Go plugin supports vgo, there shouldn't be a problem.

[RichyHBM]

In that case @ramya-rao-a would it be worth having a checklist in the OP or a milestone to track which dependencies have support?

[ramya-rao-a]

@RichyHBM Yes, definitely.

If anyone can make a such a list, that would be greatly appreciated.
For a list of tools that this extension depends on, see https://github.com/Microsoft/vscode-go/wiki/Go-tools-that-the-Go-extension-depends-on

[RichyHBM]

I'll start it up here:

VGO tools support

• go Built in vGo support for the language (exp in Go 1.11)
• gocode for auto-completion
• go-outline for symbol search in the current file
• go-symbols for symbol search in the current workspace
• gopkgs for auto-completion of unimported packages
• guru for the Find all References feature (Official Go tool)
• gorename for renaming symbols (Official Go tool)
• goreturns or goimports for formatting code (goimports works but currently returns src/mod/.. dir, should be fixed with official vgo support )
• godef or gogetdoc for the Go to Definition feature
• godoc or gogetdoc for the documentation that appears on hover (GoDoc Official Go tool)
• golint or gometalinter or megacheck for linting
• dlv for debugging

[RichyHBM]

Maybe a good first step would be to allow specifying a custom Go binary to use (that way you could point it at the vgo binary)

[AlekSi]

Support for modules was merged into Go and will be released as an experimental part of Go 1.11. This message sums it up quite nicely: https://groups.google.com/d/msg/golang-dev/a5PqQuBljF4/61QK4JdtBgAJ.

I don't see the reason to support golang.org/x/vgo tool separately.

[MFoster]

You are correct, I had the compare syntax in reverse... uudashr/gopkgs@master...ikgo:master

[genewoo]

Maybe append this configuration to your setting:

    "files.associations": {
        "*.mod":"go"
    }

[ramya-rao-a]

Folks, I have an update.

golang/go#24661 is the issue used by the Go tools team to track the update of Go modules support in various Go tools.

While that is in progress, the currently available beta version of this extension has fixes for the below features when using Go modules

• Go: Add Import & Go: Browse Packages commands that will show the appropriate packages from the current module instead of GOPATH
• Auto-completion of unimported packages when go.autocompleteUnimportedPackages is set to true

Features that were not affected when using modules in the first place:

• Build : The build on save features as well as the Go: Build ... commands
• Test : Running tests via the Go: Test commands as well as the "run tests" codelens.
• Generating of tests using the Go: Generate ... commands
• File Outline: Either via the Outline view in the explorer or using the command Cmd+Shift+O or Ctrl+Shift+O
• Go: Add tags to struct fields & Go: Remove tags from struct fields commands
• Code snippets
• Debugging

Features that are broken, but we are in the process of fixing:

• Auto-completion
• Code coverage
• Go to definition

Please do try the beta versions and share any feedback/bugs you have.

[ramya-rao-a]

Another update:

• Support for the Go to definition, symbol info on hover and Signature Help features are now available if you are using the gogetdoc tool.
• There is a new beta version of this extension.
• Install the new beta version => add the setting "go.docsTool": "gogetdoc" => run the command Go: Install/Update tools => choose gogetdoc and gopkgs => Press Ok to update. Both these tools support Go modules.
• I've added a new wiki for Go modules support in Visual Studio Code that will have the status of the modules support work as well.

[marwan-at-work]

Hi, Thanks for this Ramya and everyone! If I have GO111MODULE=auto and I'm inside GOPATH (meaning modules are off, though my repo has go.mod), the Auto completion doesn't work. I know you mentioned Auto-completion is not working for go modules but is the regular auto completion for GOPATH also not working if we opt in to beta? Best,

…

On Sun, Sep 23, 2018 at 3:53 AM Ramya Rao ***@***.***> wrote: Another update: - Support for the Go to definition, symbol info on hover and Signature Help features are now available if you are using the gogetdoc tool. - There is a new beta version <https://github.com/Microsoft/vscode-go/wiki/Use-the-beta-version-of-the-latest-Go-extension> of this extension. - Install the new beta version => add the setting "go.docsTool": "gogetdoc" => run the command Go: Install/Update tools => choose gogetdoc and gopkgs => Press Ok to update. Both these tools support Go modules. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#1532 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/APihdSKiMTPF6TZgsWd4CDVWX_ZJShDjks5udz37gaJpZM4SNton> .

-- Marwan Sulaiman Software Developer Mobile: 203-722-5772

[asw101]

I just installed @stamblerre's Beta (https://github.com/stamblerre/gocode), per mdempsky/gocode#46, as follows:

go get -u github.com/stamblerre/gocode
gocode close

This has restored auto-completion for me (cc: @ramya-rao-a). I haven't tested it in great depth yet, but it's great to have it back! @marwan-at-work, this is working inside GOPATH as well.

[ramya-rao-a]

@marwan-at-work The beta version hasn't changed anything in the auto-completion of symbols themselves, only auto-completion of unimported packages. So your issue must be due to something else.

@asw101 Yes, we do intend to use @stamblerre's fork of gocode. But since that is still work in progress, we intend to ship it in such a way that the fork is used only when you use Go modules, otherwise it falls back to the mdempsky's fork. Also, the results from it are not consistent. Once it is consistent, I'll ship the support for the same

[dreamheld]

I think the issue still persists. Maybe I'm doing something wrong but here's what I'm looking at:

1. Making a new directory outside of my current GOPATH.
2. Navigate to the new directory and run $go mod init packtest
3. Create a subfolder "packtest" with a packtest.go in it.
4. packtest.go contains:
   `package packtest
   import "fmt"

func PackPrint(){fmt.Println("PackTest Called")`

5. Navigating up to the root folder and create main.go.

6. Trying to call packtest.PackPrint() from main.go fails. Not only it doesn't auto-complete the new package, it doesn't import it either. It also doesn't have it listed in the "Add Import..." option.

(go.autocompleteUnimportedPackages": true)

[ramya-rao-a]

@dreamheld Can you log a new issue for that please?

[bjm88]

I have all my 3 related projects referencing each other in go.mod and building on command line in go build CLI. However, in Visual Studio I get red errors on the package main main.go first lines..
I've followed the instructions to install the latest master version by cloning this repo....full instructions here.. https://github.com/Microsoft/vscode-go/wiki/Use-the-beta-version-of-the-latest-Go-extension

After restarting vscode I'm still getting this error in my Go 1.11 go.mod based project....

can't load package: package /Users/ben/dev/company1/project1/chat_service: cannot find package "/Users/ben/dev/company1/project1/chat_service" in any of:
/usr/local/Cellar/go/1.11/libexec/src//Users/ben/dev/company1/project1/chat_service (from $GOROOT)
/Users/ben/go/src//Users/ben/dev/company1/project1/chat_service (from $GOPATH)

Visually its showing a red line under my package main declaration at top of main.go. Its weird like looking for itself or my source code in the GOPATH, which I dont want to use GOPATH at all anymore and specifically go get still putting vendor packages there, but not my source code. Any ideas?

[dreamheld]

@dreamheld Can you log a new issue for that please?

Done!

[ikgo]

-i flag should be removed from go build command

…

On Tue, Sep 25, 2018, 19:20 dreamheld ***@***.***> wrote: @dreamheld <https://github.com/dreamheld> Can you log a new issue for that please? Done! — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1532 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AiAFmSXPPShuinLmq2QWQr5qHjw7tvSFks5uelfIgaJpZM4SNton> .

[bjm88]

Can we customize the go build flags using..
"go.buildFlags": []
To exclude -i ? Default seems to be blank?

[l2dy]

@bjm88 See #1532 (comment), you can use "go.installDependenciesWhenBuilding": false.

[bjm88]

I tried that, did not help. Still getting weird error. Also weird is I have 3 projects, 2 services and 1 shared common project for db/logging that the 2 reference. The 2 services seem to have this red line on the package main, but the common one never does. Perhaps some bug when go.mod has a location reference
module company1/product1/identity_service
require (
cloud.google.com/go v0.28.0 // 3rd party, normal ref
company1/product1/go_common v0.0.0 . //local source code not in GOPATH, also we dont have anything special version wise, is using v0.0.0 proper or way to not specify?

)

replace company1/product1/go_common => ../go_common

Is this go.mod setup causing this? It seems so b/c the go_common lib doesn't actually have the issue.

[ramya-rao-a]

@bjm88 Open the output panel and select "Go" from the drop down on the top right. You can see the build, lint, and vet calls being made. Copy the build call and run it in the terminal. If you see the same errors in the terminal, then it is not related to the Go extension to VS Code, but is related to Go and your code setup.

[treeder]

Looks like they're going to make the -i flag a no op in in the go tool: golang/go#27285 (comment)

[ramya-rao-a]

Thanks for letting us know about that @treeder!
I've pushed a change to not use the -i flag when using Go modules.
The next update will have that change

[bjm88]

Thanks @ramya-rao-a for the hint to isolate. I can confirm I see this error output in the output panel for Go. However I can also confirm if I just manually run go build from my command prompt/shell I do NOT see the error. I only see one special flag -o, so I tried setting that also and it didn't make a different. No build issues manually, only within visual studio. The only thing I can see that is different is some issue with assuming source is in GOPATH and my source code is NOT in GOPATH its a workspace outside of GOPATH as its a monolithic repo with other projects non-go. It looks like visual studio code inject the fully qualified path to do a build, which would start at /Users as I'm on a mac, but the actual error is showing it is injecting this "_" underscore in front of the path. Is that some normal convention??

/Users/ben/dev/company/project>Finished running tool: /usr/local/bin/go build -o /var/folders/8_/b0g1vb392ws98pthb2vcf5nr0000gn/T/vscode-godAaLjI/go-code-check-4
_/Users/ben/dev/company/project/go_common/database

Is it related to how vs code is launching on mac and calcuing full path that happens to be outside GOPATH? Just FYI To try and eliminate the error I did put this workspace path into GOPATH but it didn't help, same error.

[ramya-rao-a]

Using _ as the prefix for file paths in the output of a go command is the standard way, go tells you that the file is outside of the GOPATH.

Can you add the below in your settings, reload VS Code and try again?

"go.toolsEnvVars": {
        "GO111MODULE": "on"
    }

[bjm88]

Thank you @ramya-rao-a adding this config
"go.toolsEnvVars": {
"GO111MODULE": "on"
}
fixed the issue. I was surprised this is needed b/c I actually installed the latest main build version of the go plugin that I thought started supporting this. Anyway, thank you!

Also if anyone else running into this and also finding it hard to test. I thought it was another issue with VS code, but actually my build feedback on errors was just really delayed b/c of this other issue where replace commands in go.mod were ignored by go run/go build/go get so it still does an HTTP call to try and get local package and times out and only then continues to finish actual build. This makes it really slow and delayed to find/test these issues on 1.11 right now if u using replace. Its a core go issue not VS code issue.

golang/go#27859

[ramya-rao-a]

Another update:

The Go tools team have forks of the godef and gocode tools that work with Go modules. These 2 tools drive the Go to definition and auto-completion features respectively. The latest beta version of this extension has support for both.

Please install this beta version, follow through the prompts to install/update the tools and let us know how the Go to definition and auto-completion features are working out for you.

The sooner I get some feedback here, the sooner I can release an update with these changes :)

[dreamheld]

Almost there! There are still a couple of bugs. Same configuration as before:

typing "packtest" in main.go automatically populates the import statement. However, it's missing a '/':

Adding the missing '/' and filling out the rest of the package function call results in proper execution:

Hooray!

However, writing out the package name still doesn't auto-populate it's methods after ".":

[ramya-rao-a]

@dreamheld I'll respond to the issue you created for the same yesterday #1942 and we can take it from there.

[ramya-rao-a]

All,

Now that we have some progress on the features when using Go modules, I am closing this issue and locking the conversation. This is because this issue is already way too long for any new-comer to make any sense of.

Here are some tips to continue this conversation:

• Keep an eye on the Go module support in VS Code wiki which will be up to date on the status.
• If you find bugs, first search existing issues with label "go-modules". If you dont find any that relate to your current problem, log a new issue
• If you want to check back in periodically, look at the wiki as well as the issues labelled "go-modules". That should give you the gist of the situation much better than reading through all the comments in this issue.
• Try the beta version that will always have the latest fixes from the master branch

Thank you for understanding.