Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/tools/gopls: duplicate module in workspace with experimentalWorkspaceModule #45184

Open
iamshreeram opened this issue Mar 23, 2021 · 33 comments
Open

Comments

@iamshreeram
Copy link

@iamshreeram iamshreeram commented Mar 23, 2021

Issue

I use multi-module workspace (kinda monorepo). I'm getting below error from gopls for all the modules in pkg -

found module "github.com/Shopify/sarama" twice in the workspace

I found I have different versions of sarama. But I believe that is not causing this. because, The error is pointing all the modules. I have attached the screenshot with this for more details.

What version of Go are you using (go version)?

$ go version
go version go1.15.2 darwin/amd64

Does this issue reproduce with the latest release?

Yes.

What operating system and processor architecture are you using (go env)?

go env
$ go env
GO111MODULE="on"
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/sram/Library/Caches/go-build"
GOENV="/Users/sram/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOINSECURE=""
GOMODCACHE="/Users/sram/ram/project/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/sram/ram/project/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/darwin_amd64"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/dev/null"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/sn/pw7xq23n2vj_wnq16lkgdrxh0000gn/T/go-build150007759=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

gopls settings

    "gopls": {
         "ui.documentation.hoverKind": "SynopsisDocumentation",
         "build.directoryFilters": ["-/Users/sram/ram/project/node/","-node_modules"],
         "build.experimentalWorkspaceModule": true,          
      }

What did you expect to see?

No errors or warnings.

What did you see instead?

Error that says

found module "github.com/Shopify/sarama" twice in the workspace

Below is the screenshot of error:

Screen Shot 2021-03-23 at 11 53 45 AM Screen Shot 2021-03-23 at 9 13 24 AM
@gopherbot gopherbot added this to the Unreleased milestone Mar 23, 2021
@stamblerre
Copy link
Contributor

@stamblerre stamblerre commented Mar 24, 2021

Thanks for filing this issue. It looks like your Go module cache (pkg/mod/...) is in your workspace. Is that intentional? Usually I'd expect the module cache to be outside of the workspace.

Also, can you please share your gopls logs? Information on how to capture them can be found here.

@stamblerre stamblerre removed this from the Unreleased milestone Mar 24, 2021
@stamblerre stamblerre added this to the gopls/v1.0.0 milestone Mar 24, 2021
@stamblerre stamblerre changed the title x/tools/gopls: Error while loading workspace - Duplicate module in workspace x/tools/gopls: duplicate module in workspace with experimentalWorkspaceModule Mar 24, 2021
@iamshreeram
Copy link
Author

@iamshreeram iamshreeram commented Mar 25, 2021

Hi @stamblerre ,
Thanks for looking into this. Please find the attachment for the gopls logs.

Yes, pkg/mod.. is inside workspace. Do you want me to filter it out using gopls filter? I want gopls to look only at my go project directory. Is there any way to restrict it by using filters? I have created #45171 ticket is to address this issue. This issue contains my workspace structure and much more information of my setup if you want to look at it once.

Thanks!

@stamblerre
Copy link
Contributor

@stamblerre stamblerre commented Mar 30, 2021

Hi @stamblerre ,
Thanks for looking into this. Please find the attachment for the gopls logs.

It looks like you need to add "go.languageServerFlags": ["-rpc.trace"] to get full verbosity logs.

Yes, pkg/mod.. is inside workspace. Do you want me to filter it out using gopls filter? I want gopls to look only at my go project directory. Is there any way to restrict it by using filters? I have created #45171 ticket is to address this issue. This issue contains my workspace structure and much more information of my setup if you want to look at it once.

Thanks!

It is pretty atypical for the module cache to be in the workspace, so I would suggest avoiding that if at all possible. It won't work with experimental workspace module mode because you will run into this exact problem that you're seeing with the duplicate modules--experimental workspace mode does not support multiple modules with the same path and different versions in the workspace. If necessary, you can configure a module cache that is outside of your GOPATH by using the GOMODCACHE environment variable.

@iamshreeram
Copy link
Author

@iamshreeram iamshreeram commented Apr 1, 2021

@stamblerre, Please find the attachment for verbose logs.

gopls-vscode-duplicate-workspace-error-logs.log

On move my module cache to different path, Would I be able to copy the content from current cache and put it in different path? Or would it have to be loaded when go get?

@stamblerre
Copy link
Contributor

@stamblerre stamblerre commented Apr 1, 2021

On move my module cache to different path, Would I be able to copy the content from current cache and put it in different path? Or would it have to be loaded when go get?

I would create an empty directory that you want to use for your new module cache, set the GOMODCACHE variable to point to that directory, and then your module cache will be populated automatically whenever you build your code or open the editor. There is no need to manually populate it.

@iamshreeram
Copy link
Author

@iamshreeram iamshreeram commented Apr 9, 2021

I have modified the GOMODCACHE as suggested. The error count has reduced. But, I still see below error. I don't see anything on gopls log.

golang-error

@stamblerre
Copy link
Contributor

@stamblerre stamblerre commented Apr 9, 2021

It looks like the example.com/hello module appears multiple times in this workspace. Do you know where this module is in your workspace / can you remove the duplicate copy? You could also use the directory filters to do this.

@iamshreeram
Copy link
Author

@iamshreeram iamshreeram commented Apr 10, 2021

Yeah, I could see that this has been used in multiple place. I fixed it. Now, I'm seeing different issue.

Below is the screenshot.

Screen Shot 2021-04-09 at 11 49 29 PM

These are from the GoKafka package. Should I replace the module name?

I have added the filter -$GOPATH/src/github.com/segmentio/kafka-go, but gopls isn't picking the filter config.

Also, I have one another issue. When I hover on any function/variable/anything, I don't see any definition. Should I enable any specific setting to enable this?

@heschi heschi removed this from the gopls/v1.0.0 milestone Apr 12, 2021
@heschi heschi added this to the gopls/unplanned milestone Apr 12, 2021
@stamblerre stamblerre self-assigned this Apr 12, 2021
@stamblerre
Copy link
Contributor

@stamblerre stamblerre commented Apr 12, 2021

I believe the directory filters are actually relative to your project root, which was my mistake earlier. For the original directory filter you shared about, it should be "build.directoryFilters": ["-node","-node_modules"] if your workspace root is /Users/sram/ram/project/. You can also take a look at the documentation for the directory filters to see some more examples.

If you want to exclude the kafka package, you will want something like -src/github.com/segmentio/kafka-go. Still, it seems like you have a lot of modules in your workspace if your entire GOPATH is in there, so it might be difficult to use experimentalWorkspaceModule with that set up. It's really meant to work on a select set of modules, not all of the modules on your machine.

@stamblerre stamblerre removed their assignment Apr 16, 2021
@iamshreeram
Copy link
Author

@iamshreeram iamshreeram commented Jun 3, 2021

@stamblerre , Issue after setting up the custom GOMODCACHE -

I use a custom location for GOMODCACHE (~/.gomodcache/) and my GOPATH is ~/project/go. This is to handle the errors from GOPLS while using the mono-repo approach of workspace.

When I do go get, the modules are downloaded to GOMODCACHE but, unable to see them in GOPATH. The VSCode’s Go plugin just looks for the list of modules from GOPATH and GOROOT. So, the plugin is unable to find the modules from GOMODCACHE and unable to provide suggestions on autocomplete.

Is there a way to get the modules into GOPATH instead of GOMODCACHE? Or Is there a way to make vscode to read the modules from GOMODCACHE instead of GOPATH?

@stamblerre
Copy link
Contributor

@stamblerre stamblerre commented Jun 3, 2021

@iamshreeram: It sounds like you are still trying to operate in GOPATH mode in some way, but you are using modules (if you have a GOMODCACHE). The VS Code Go plugin uses gopls, which should pick up your environment, so it shouldn't be looking for modules in your GOPATH if you are in GOPATH mode. Please share your logs, your VS Code settings, and a concrete example if this is the case. In module mode, go get only downloads to the module cache--that is the expected behavior. You can download repositories to add them to your GOPATH.

@iamshreeram
Copy link
Author

@iamshreeram iamshreeram commented Jun 4, 2021

@stamblerre , Thanks for prompt response. The issue is just with plugin or vscode.

Below is the error when I hover on import in vscode -

"github.com/juliangruber/go-intersect" imported but not used
could not import github.com/juliangruber/go-intersect (cannot find package "github.com/juliangruber/go-intersect" in any of 
	/usr/local/go/src/github.com/juliangruber/go-intersect (from $GOROOT)
	/Users/home/project/go/src/github.com/juliangruber/go-intersect (from $GOPATH))

When I build or run the code through the terminal, things go well without any errors. Any thoughts?

@stamblerre
Copy link
Contributor

@stamblerre stamblerre commented Jun 4, 2021

It looks like gopls thinks that you are not using module mode. Are you opening a directory that contains a go.mod file?
Can you please share your gopls logs? Information on how to capture them can be found here.

@iamshreeram
Copy link
Author

@iamshreeram iamshreeram commented Jun 4, 2021

  1. I think, you are right. Gopls is still thinking gomod is off. Is there any way to enforce this?
  2. Yes, the directory contains go.mod.
  3. Gopls server log is really huge. Below is the log while hovering on module name that I import
[Trace - 23:19:13.190 PM] Sending request 'textDocument/codeAction - (45)'.
Params: {"textDocument":{"uri":"file:///Users/home/project/go/src/gecgithub.com/ers/ocf-gql/cmd/aka/gtmOperation.go"},"range":{"start":{"line":20,"character":1},"end":{"line":20,"character":30}},"context":{"diagnostics":[{"range":{"start":{"line":20,"character":1},"end":{"line":20,"character":30}},"message":"could not import github.com/akamai/AkamaiOPEN-edgegrid-golang/edgegrid (cannot find package \"github.com/akamai/AkamaiOPEN-edgegrid-golang/edgegrid\" in any of \n\t/usr/local/go/src/github.com/akamai/AkamaiOPEN-edgegrid-golang/edgegrid (from $GOROOT)\n\t/Users/home/project/go/src/github.com/akamai/AkamaiOPEN-edgegrid-golang/edgegrid (from $GOPATH))","code":"BrokenImport","codeDescription":{"href":"https://pkg.go.dev/golang.org/x/tools/internal/typesinternal?utm_source%3Dgopls#BrokenImport"},"severity":1,"source":"compiler"}],"only":["quickfix"]}}

[Trace - 23:19:13.191 PM] Received response 'textDocument/codeAction - (45)' in 1ms.
Result: null

[Trace - 23:19:43.198 PM] Received notification 'window/logMessage'.
Params: {"type":3,"message":"2021/06/03 23:19:43 background imports cache refresh starting\n"}

[Info  - 11:19:43 PM] 2021/06/03 23:19:43 background imports cache refresh starting

[Trace - 23:19:43.651 PM] Received notification 'window/logMessage'.
Params: {"type":3,"message":"2021/06/03 23:19:43 background refresh finished after 454.324387ms\n"}

[Info  - 11:19:43 PM] 2021/06/03 23:19:43 background refresh finished after 454.324387ms

@stamblerre
Copy link
Contributor

@stamblerre stamblerre commented Jun 4, 2021

I will need at least the start of the log to fully understand the issue. Where is the go.mod file? Is it in the root of your workspace? I will need more information about your project structure.

@iamshreeram
Copy link
Author

@iamshreeram iamshreeram commented Jun 4, 2021

@stamblerre ,

  1. No. go.mod is not in root of workspace. I use multi-module workspace setup. Thats the reason, I had to move the GOMODCACHE to custom location.
  2. My project structure is as below
~
 | /.config
 | /.cabal/
 | /.nvm/
 | .bash_profile
 | .bashrc
 | /project/
 |  | /Clojure/
 |  |  | /allmycljprojects 
 |  | /Shell/
 |  |  | /allmybashprojects 
 |  | /rust/
 |  |  | /allmyrustprojects
 |  | /haskell/
 |  |  | /allmyhaskellprojects
 |  | /python/
 |  |  | /allmypythonprojects 
 |  | /go/
 |  |  | /bin
 |  |  | /pkg
 |  |  | /src
 |  |  |  | /github.com
 |  |  |    | /tarekbadrshalaan/
 |  |  |    | /elastic  
 |  |  |  | /gecgithub.com
 |  |  |    | /ers 
 |  |  |      | /ocf-gql
 |  |  |         | go.mod
 |  |  |         | go.lock
 |  |  |         | cmd
 |  |  |      | /dummy-app
 |  |  |         | go.mod 
 |  |  |  | /gitlab.com
 |  |  |  | /golang.org
 |  |  |  | /gopkg.in
 |  | /scala/
 |  |  | /allmyscalaprojects
 |  | /node/
 |  |  | /allmynodeprojects

Please find the attachment for gopls-server.log.

@iamshreeram
Copy link
Author

@iamshreeram iamshreeram commented Jun 4, 2021

Below is the screenshot of error / issue :

Screen Shot 2021-06-04 at 5 26 08 AM

@stamblerre
Copy link
Contributor

@stamblerre stamblerre commented Jun 4, 2021

It doesn't seem like gopls has detected the go.mod file in the ocf-gql directory. Can you please share your settings? Does VS Code work correctly if you only open the ocf-gql directory?

@iamshreeram
Copy link
Author

@iamshreeram iamshreeram commented Jun 4, 2021

@stamblerre , Yes. Auto completion works well when I open just ocf-gql directory.

Attaching my vscode settings

@stamblerre
Copy link
Contributor

@stamblerre stamblerre commented Jun 4, 2021

I'm not seeing the experimentalWorkspaceModule setting in your gopls configuration. Does adding it fix the problem?

@iamshreeram
Copy link
Author

@iamshreeram iamshreeram commented Jun 5, 2021

@stamblerre , with experimentalWorkspaceModule , the autocomplete that was working with GOPATH has also stopped working.

@stamblerre
Copy link
Contributor

@stamblerre stamblerre commented Jun 7, 2021

@iamshreeram: I'm not entirely sure what you are trying to achieve--you are trying to autocomplete something that is only in your GOPATH? You either need to operate in module mode, in which case your GOPATH does not matter, or you can switch back to GOPATH mode. Experimental workspace module mode requires you to use modules. If you have a concrete example of a problem, I will need to see a full log file and a full explanation of the issue.

@iamshreeram
Copy link
Author

@iamshreeram iamshreeram commented Jun 7, 2021

@stamblerre ,

  1. I'm currently running with module mode. But, for some reason - Go Plugin is still looking for modules in GOPATH & GOROOT when I'm in multi-module workspace setup.
  2. When I'm opening just one project, Go plugin is able to find the modules from the go mod cache.
  3. Below is my current gopls setting -
    "gopls": {
        "build.directoryFilters":["-/Users/home/ram/project","-/Users/home/ram/project/node","+/Users/home/ram/project/go/src/github.com/iamshreeram","+/Users/home/ram/project/go/src/gitlab.com/iamshreeram","+/Users/home/ram/project/go/src/gecgithub.com"],
        "build.env": {
            "GOMODCACHE": "/Users/home/.gomodcache/pkg/mod"
        },
        "build.experimentalWorkspaceModule": true
    },

Please find the attachment for gopls-server.log. This was pulled before adding experimentalWorkspaceModule.

@stamblerre
Copy link
Contributor

@stamblerre stamblerre commented Jun 9, 2021

Please find the attachment for gopls-server.log. This was pulled before adding experimentalWorkspaceModule.

gopls without experimental workspace module mode requires you to only have one module open, and that doesn't seem to be the case with your workspace. I will need the log with that enabled, or you will need to only open a single module.

@iamshreeram
Copy link
Author

@iamshreeram iamshreeram commented Jun 14, 2021

@stamblerre , Sure. Please find the attachment.

@benbuzbee
Copy link

@benbuzbee benbuzbee commented Jun 14, 2021

As a datapoint I disabled experimentalWorkspaceModule because of this

In my case we have a few submodules
| /src/
| | go.mod
| | /deps/
| | | /goproject2/
| | | /goproject3/
| | | | /deps/
| | | | | /goproject2/

In this case goproject2 shows up twice because it is our submodule, and a submodule of a dependency we have.

Admittedly it's a strange project structure but we chose it because we can craft very efficient makefiles that let us modify dependencies quickly.

I wish the following statement were true: "if go build can build knows that to do with your file, then gopls can resolve the same dependencies go build will"

@stamblerre
Copy link
Contributor

@stamblerre stamblerre commented Jun 15, 2021

@iamshreeram: Do you have many workspace folders in your workspace? I am not sure how that will play with experimentalWorkspaceModule mode. But I do see an error:

[Error - 12:40:56 AM] 2021/06/14 00:40:56 initial workspace load failed: found module "Onecf-api" twice in the workspace

Is it possible to remove the duplicate module?
Also, I've recently made a change that will ignore the GOMODCACHE even if it is in the workspace, so that may allow you to remove the GOMODCACHE configuration in your settings, and will possibly resolve this issue. It will be available in the next release.

@benbuzbee: Why do you need experimentalWorkspaceModule if you only have one go.mod file in your project? go build operates in the context of one module, which is why you have to cd into the directory containing the module you want to build in. If you're going to work with multiple modules, gopls needs special handling.

@benbuzbee
Copy link

@benbuzbee benbuzbee commented Jun 15, 2021

Thanks for following up @stamblerre
The reason I have the flag enabled is I want to be able to open our repo root at "." and edit either our go module which is at ./src/go/go.mod or one of the go modules in our submodules, like ./src/third_party/subproject1/go.mod

subproject1 expands to the structure i listed above.

In essence, the use case is a repo with a go module and with submodules that are also go modules. Some modules may be repeated as submodules of submodules etc.

My work around then is using a multi-root workspace with experimentalWorkspaceModule = false; but that has its limitations

@iamshreeram
Copy link
Author

@iamshreeram iamshreeram commented Jun 15, 2021

@stamblerre ,

  1. Yes, I had multi-folder (~/project/go & /go) in single workspace. Now, I have removed and have only one. Attaching the logs - Attachment. Auto-complete is still not working.
  2. When GOMODCACHE is removed, the plugin would always look the modules in GOPATH. Is that right?

@hyangah
Copy link
Contributor

@hyangah hyangah commented Jun 16, 2021

experimentalWorkspaceModule setting is used basically to create a big super module workspace by merging all the modules found in the workspaces, under the assumption that users need all the modules found there for their work (The motivation/overview of the concept is described in https://github.com/golang/proposal/blob/master/design/37720-gopls-workspaces.md, and the new proposal #45713 is not too different from the proposal/37720 wrt how merging of the modules happens).

If identically named modules appear in the workspace multiple times, I am not sure how to resolve the case and choose one of them, but think the error message is correct and working as intended.

@stamblerre
Copy link
Contributor

@stamblerre stamblerre commented Jun 21, 2021

2. When GOMODCACHE is removed, the plugin would always look the modules in GOPATH. Is that right?

No. We configured the GOMODCACHE environment variable earlier to handle the fact that your module cache was in your workspace. GOPATH will only be used when you are in GOPATH mode, i.e. GO111MODULE=off.

The log you shared above shows that you seem to be in GOPATH mode, but it's hard to tell because you are missing go.languageServerFlags": ["-rpc.trace"] in your VS Code settings. Please add that and share a new log.

@p1gd0g
Copy link

@p1gd0g p1gd0g commented Aug 18, 2021

Maybe same case here:

main_project
      |--go.mod (require pkgfoo v1)
      |--dep_project (git submodule)
                |--go.mod (require pkgfoo v1)

found module "pkgfoo" twice in the workspace

@stamblerre
Copy link
Contributor

@stamblerre stamblerre commented Aug 18, 2021

@p1gd0g: That should not happen. Please open a new issue and follow the steps here: https://github.com/golang/tools/blob/master/gopls/doc/troubleshooting.md#file-an-issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
7 participants