Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: 'cannot find module for path' when importing from subdirectories #26645

Closed
markkuit opened this issue Jul 27, 2018 · 10 comments
Closed

cmd/go: 'cannot find module for path' when importing from subdirectories #26645

markkuit opened this issue Jul 27, 2018 · 10 comments

Comments

@markkuit
Copy link

@markkuit markkuit commented Jul 27, 2018

What version of Go are you using (go version)?

go version go1.11beta2 linux/amd64

Does this issue reproduce with the latest release?

Yes.

What operating system and processor architecture are you using (go env)?

GOARCH="amd64"
GOBIN=""
GOCACHE="/home/markkuit/.cache/go-build"
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/markkuit/go/"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/go"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/tmp/gomodsubdirs/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build236792179=/tmp/go-build -gno-record-gcc-switches"

What did you do?

The issue arises when using modules when having imports from subdirectories. I created a easy to reproduce example:

/tmp/gomodsubdirs/go.mod
module testmodule.local/gomodsubdirs

/tmp/gomodsubdirs/main.go

package main

import (
	"./subdir"
	"fmt"
)

func main() {
	fmt.Println("Printing from main!")
	subdir.Print()
}

/tmp/gomodsubdirs/subdir/subdir.go

package subdir

import "fmt"

func Print() {
	fmt.Println("Printing from subdir!")
}

What did you expect to see?

I expected build to be successful both with and without modules, but it looks like putting code in subdirectories is troublesome for go modules (or I'm doing something wrong). I tend to do this quite a lot when I have a package in a subdirectory with a lot of files (mostly boilerplate code), which isn't however worthy of being a separate package itself or is too strictly related to the main package to be put apart from it. Note that I tried having a go.mod inside the subdirectory as well (both empty and filled), but it made no difference.

What did you see instead?

> go build main.go
build .: cannot find module for path _/tmp/gomodsubdirs/subdir
> env GO111MODULE=off go build main.go
> ./main
Printing from main!
Printing from subdir!
@oiooj oiooj added the modules label Jul 27, 2018
@rsc
Copy link
Contributor

@rsc rsc commented Jul 28, 2018

Sorry but this is working as intended. This kind of "relative import" was something we had in the very early days of go, before there was a go command, but when we moved to the go command and GOPATH we finally had a way to give a fully-qualified path for imports, at least within GOPATH, and we standardized on that fully-qualified form for tools and such. The relative imports still worked outside GOPATH, essentially out of a combination of necessity and neglect, but they didn't work as well. Among other things, until Go 1.10 there was no build caching, so that builds were slower. Many tools don't handle them right. But they were left in because otherwise there was no name for that subdirectory.

In modules, there finally is a name for the subdirectory. If the parent directory says "module m" then the subdirectory is imported as "m/subdir", no longer "./subdir".

I do apologize that we don't have something that works both ways, but we definitely want to preserve the property for the vast majority of Go code that imports are fully-qualified, and so we don't want moving to modules to lose that property. Instead moving from "outside GOPATH one-off directories" to modules requires using the newly-available fully-qualified name.

Best,
Russ

@rsc rsc closed this Jul 28, 2018
@markkuit
Copy link
Author

@markkuit markkuit commented Jul 28, 2018

This actually makes perfect sense, and I do agree it was a choice that had be taken. What I was worried about was not having the possibility to organize code in subdirectories anymore; instead, I can confirm specifying the full import path achieves the goal - even if the module isn't under any VCS yet.

I'll take this opportunity to thank you (and the whole team) for your continued efforts, and thank you for clarifying this issue.

@ccll
Copy link

@ccll ccll commented Dec 14, 2018

In modules, there finally is a name for the subdirectory. If the parent directory says "module m" then the subdirectory is imported as "m/subdir", no longer "./subdir".

How about sibling packages?
I have a folder structure:

- Root
  |--- mod1
       |--- go.mod
  |--- mod2
       |--- go.mod

I couldn't find a way to import mod2 from mod1.

@ccll
Copy link

@ccll ccll commented Dec 17, 2018

How about sibling packages?

Never mind, I got the answer in the Go Wiki.

For someone like me, here is the relevant part:
image

@figiel
Copy link

@figiel figiel commented May 25, 2019

Hi @rsc,

In modules, there finally is a name for the subdirectory. If the parent directory says "module m" then the subdirectory is imported as "m/subdir", no longer "./subdir".

Is this something which should work now or is it going to be added?
I can't get this to work with go 1.12:

~/tmp$ tree topdir/
topdir/
├── go.mod
├── subdir
│   └── subdir.go
└── topdir.go

1 directory, 3 files
~/tmp$ cat topdir/go.mod 
module example.com/topdir

go 1.12
~/tmp$ cat topdir/topdir.go 
package topdir

import "topdir/subdir"

type Topdir subdir.Subdir
~/tmp$ cat topdir/subdir/subdir.go 
package subdir

type Subdir interface{}
~/tmp$ cd topdir/
~/tmp/topdir$ go build
topdir.go:3:8: unknown import path "topdir/subdir": cannot find module providing package topdir/subdir

@kochie
Copy link

@kochie kochie commented May 28, 2019

@figiel I think there is a subtle difference in your import compared to what is defined in the import definition. In your go.mod the module is named example.com/topdir so m=example.com/topdir. Therefore in your files you should be importing like example.com/topdir/subdir

@lppgo
Copy link

@lppgo lppgo commented May 28, 2019

I have the same problem. When you close the module, it compiles; when you open it, the same error is reported. I open a new terminal window to compile through.

@xtrimf
Copy link

@xtrimf xtrimf commented Sep 18, 2019

same issue with 1.13 . when to expect patch?

@warent
Copy link

@warent warent commented Nov 30, 2019

@xtrimf You need go modules. It's actually really easy to set it up: https://blog.golang.org/using-go-modules

@xtrimf
Copy link

@xtrimf xtrimf commented Dec 1, 2019

@warent tnx. already did two month ago...

timja added a commit to timja/openvpn that referenced this issue Dec 3, 2019
Fixes build
Most changes are due to this:
golang/go#26645 (comment)
and
mozilla/sops@8302b6a
timja added a commit to timja/openvpn that referenced this issue Dec 3, 2019
Fixes build
Most changes are due to this:
golang/go#26645 (comment)
and
mozilla/sops@8302b6a
onap-github pushed a commit to onap/doc that referenced this issue Mar 25, 2020
* Update docs/submodules/integration.git from branch 'master'
  to d688c4c4e37526e276690b5b51d1044b7e220aff
  - Reduce cyclomatic complexity
    
    Moving CSV data conversion and "expected failure" filtering away from
    main function made testing these features easier. Utility behaviour
    remained unchanged.
    
    Issue-ID: SECCOM-261
    Change-Id: I4cabfc7b352434c84a613c02f44af3c9630be970
    Signed-off-by: Pawel Wieczorek <p.wieczorek2@samsung.com>
    
  - Add "expected failure" support to non-SSL NodePort scanner
    
    This patch makes scanner compatible with its shell predecessor. The same
    "expected failure" list format is used i.e.
    
     # Comment line; will be ignored
     SERVICE1 NODEPORT1
     SERVICE2 NODEPORT2
    
    Single space character is used as a field separator.
    
    Issue-ID: SECCOM-261
    Change-Id: Ieedd4e98a83ffe242c695133fdf7342e17efa9a2
    Signed-off-by: Pawel Wieczorek <p.wieczorek2@samsung.com>
    
  - Run port scan
    
    Issue-ID: SECCOM-261
    Change-Id: I465282a8793191c45d288284a127e80e1fecf513
    Signed-off-by: Pawel Wieczorek <p.wieczorek2@samsung.com>
    
  - Add IP addresses filtering
    
    Each node might be described with 3 types of addresses [1]. Some
    providers also use node annotations [2] for assigned addresses.
    
    This patch filters out all IP addresses from nodes list. External IPs
    take precedence over internal ones. The first address on the extracted
    slice will be later used to run the scan on.
    
    This behaviour could be later modified to e.g. loop over all extracted
    IP addresses (if scan fails).
    
    [1] https://kubernetes.io/docs/concepts/architecture/nodes/#addresses
    [2] https://github.com/rancher/rke/blob/master/k8s/node.go#L18
    
    Issue-ID: SECCOM-261
    Change-Id: Ifd094447f778da378dfe1aee765f552b6ebd669f
    Signed-off-by: Pawel Wieczorek <p.wieczorek2@samsung.com>
    
  - Add temporary "make" target for automated testing compatibility
    
    Utility "sslendpoints" and related packages make use of idiomatic Go
    testing commands, i.e. go test [./...]. Thanks to Go Modules [1] nothing
    else is needed to run internal tests for this tool.
    
    Unfortunately it's not the case for all Go-based Integration tools. In
    order to use a single automated verification script in CI additional
    "make" target is required. It will provide temporary compatibility layer
    with utilities setting up test environment on their own with "make test"
    target.
    
    This patch should be reverted upon removal of such cases (currently:
    after dropping "../k8s/check" tool in favour of Aquasec solution).
    
    [1] https://blog.golang.org/using-go-modules (see "Adding a dependency"
    test execution explanation)
    
    Issue-ID: INT-1498
    Change-Id: I14c83f7f193c7688590366db988ff02c13c036a4
    Signed-off-by: Pawel Wieczorek <p.wieczorek2@samsung.com>
    
  - Add NodePorts filtering with development environment basis
    
    This patch has not made "sslendpoints" fully compatible with
    "check_for_nonssl_endpoints.sh" script yet. It sets up basic development
    environment for Golang-based checkers, though.
    
    Tool output will be added to the README after reaching full
    compatibility with previous (script) version.
    
    Development environment brought by this patch is heavily based on:
    https://github.com/SamsungSLAV/boruta
    
    Issue-ID: SECCOM-261
    Change-Id: I8f035b63bea13785c40971ede5fdbbc9b6810168
    Signed-off-by: Pawel Wieczorek <p.wieczorek2@samsung.com>
    
  - Increase verifiability of security checks
    
    This patch introduces a series of patches that will provide tools which
    will succeed current security check scripts. Its two main reasons are:
    
    * increasing tools verifiability by providing internal tests,
    * improving "expected failure" support by suppressing carefully selected
      set of special cases.
    
    Each tool will use following directory structure (generated with
    "tree -a --charset=ascii" command):
    
    .
    `-- check_module
        |-- Dockerfile
        |-- .dockerignore
        |-- .gitignore
        |-- go.mod
        |-- main.go
        |-- Makefile
        |-- README
        |-- README.rst -> README
        `-- submodule
            |-- submodule.go
            `-- submodule_test.go
    
    This will allow using Go Modules mechanism within its limitations [1]
    for "non-go-get-able modules" [2][3][4] - also in case of separating
    code into several modules used by multiple "check modules", e.g.
    
    .
    |-- common
    |   |-- common.go
    |   |-- common_test.go
    |   `-- go.mod
    `-- check_module
        |-- go.mod
        `-- ...
    
    It would require migration from separate Dockerfiles to a single one
    (multi-stage), though.
    
    Provided Makefiles are intended to simplify local development
    (Docker-less building) and container images preparation. READMEs clarify
    utility requirements and usage - file without extension is for VCS
    reference, symlink for proper syntax rendering.
    
    [1] https://github.com/golang/go/wiki/Modules#is-it-possible-to-add-a-module-to-a-multi-module-repository
    [2] https://github.com/golang/go/wiki/Modules#can-i-work-entirely-outside-of-vcs-on-my-local-filesystem
    [3] golang/go#26645 (comment)
    [4] https://www.dim13.org/go-get-cgit
    
    Issue-ID: SECCOM-261
    Change-Id: I48eeeda66bd5570d249e96e101e431e6bab75cb3
    Signed-off-by: Pawel Wieczorek <p.wieczorek2@samsung.com>
onap-github pushed a commit to onap/integration that referenced this issue Mar 25, 2020
This patch introduces a series of patches that will provide tools which
will succeed current security check scripts. Its two main reasons are:

* increasing tools verifiability by providing internal tests,
* improving "expected failure" support by suppressing carefully selected
  set of special cases.

Each tool will use following directory structure (generated with
"tree -a --charset=ascii" command):

.
`-- check_module
    |-- Dockerfile
    |-- .dockerignore
    |-- .gitignore
    |-- go.mod
    |-- main.go
    |-- Makefile
    |-- README
    |-- README.rst -> README
    `-- submodule
        |-- submodule.go
        `-- submodule_test.go

This will allow using Go Modules mechanism within its limitations [1]
for "non-go-get-able modules" [2][3][4] - also in case of separating
code into several modules used by multiple "check modules", e.g.

.
|-- common
|   |-- common.go
|   |-- common_test.go
|   `-- go.mod
`-- check_module
    |-- go.mod
    `-- ...

It would require migration from separate Dockerfiles to a single one
(multi-stage), though.

Provided Makefiles are intended to simplify local development
(Docker-less building) and container images preparation. READMEs clarify
utility requirements and usage - file without extension is for VCS
reference, symlink for proper syntax rendering.

[1] https://github.com/golang/go/wiki/Modules#is-it-possible-to-add-a-module-to-a-multi-module-repository
[2] https://github.com/golang/go/wiki/Modules#can-i-work-entirely-outside-of-vcs-on-my-local-filesystem
[3] golang/go#26645 (comment)
[4] https://www.dim13.org/go-get-cgit

Issue-ID: SECCOM-261
Change-Id: I48eeeda66bd5570d249e96e101e431e6bab75cb3
Signed-off-by: Pawel Wieczorek <p.wieczorek2@samsung.com>
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
10 participants