cmd/compile: Unclear guidelines on what files/directories are ignored #20337

sectioneight · May 12, 2017

Please answer these questions before submitting your issue. Thanks!

What version of Go are you using (`go version`)?

1.8.1

What operating system and processor architecture are you using (`go env`)?

darwin x86_64

What did you do?

underscore-test ❯ pwd
/Users/ai/src/go/src/scratch/underscore-test
underscore-test ❯ tree -a
.
├── .dot
│   └── dot.go
├── _underscore
│   └── underscore.go
└── main.go

2 directories, 3 files
underscore-test ❯ cat main.go _underscore/underscore.go .dot/dot.go
package main

import (
        "fmt"
        "scratch/underscore-test/_underscore"

        "scratch/underscore-test/.dot"
)

func main() {
        fmt.Println(underscore.Underscore)
        fmt.Println(dot.Dot)
}
package underscore

// Underscore is the best yo
const Underscore = "_"
package dot

// Dot is a dot
const Dot = "."

What did you expect to see?

I expected a compilation error, given that documentation states that "." and "_" prefixed files will be ignored, however this rule does not seem to apply to directories. I ran into this yesterday with @davecheney, who suggested that I modify golang/dep to skip scanning directories which start with _ or .. Currently we're in a weird situation where files are ignored but directories are not, and I can't find any documentation stating what the source of truth should be. Does golang/dep have the correct behavior, or broken behavior? Will the golang toolchain eventually stop compiling directories that start with . or _?

What did you see instead?

It worked:

underscore-test ❯ go build
underscore-test ❯ echo $?
0
underscore-test ❯ ./underscore-test
_
.

Directory and file names that begin with "." or "_" are ignored by the go tool, as are directories named "testdata".

--https://golang.org/cmd/go/#hdr-Description_of_package_lists

dcheney-atlassian · May 12, 2017

Directory and file names that begin with "." or "_" are ignored by the go tool, as are directories named "testdata".

--https://golang.org/cmd/go/#hdr-Description_of_package_lists

Ok, so is it a bug that my example code compiles?

sectioneight · May 12, 2017

Ok, so is it a bug that my example code compiles?

I guess the underlying question is "is there a difference between things that are ignored by the gotool, and what are valid import paths?". Is that correct?

sectioneight · May 12, 2017

I guess the underlying question is "is there a difference between things that are ignored by the gotool, and what are valid import paths?". Is that correct?

I guess the underlying question is "is there a difference between things that are ignored by the go tool, and what are valid import paths?".

yes, there is a difference. Those paths will (probably) not show up in go list ./... but can be imported.

dcheney-atlassian · May 12, 2017

I guess the underlying question is "is there a difference between things that are ignored by the go tool, and what are valid import paths?".

yes, there is a difference. Those paths will (probably) not show up in go list ./... but can be imported.

Ok, so we can close this issue, and I should change golang/dep to recurse into .-prefixed dirs when scanning what source packages are available.

Thanks!

sectioneight · May 12, 2017

Ok, so we can close this issue, and I should change golang/dep to recurse into .-prefixed dirs when scanning what source packages are available.

Thanks!

@sectioneight I think you should do the opposite

deadwood(~/src/scratch/underscore-test) % go list ./...
scratch/underscore-test
deadwood(~/src/scratch/underscore-test) % find . -type d 
.
./.dot
./_underscore

dcheney-atlassian · May 12, 2017

@sectioneight I think you should do the opposite

deadwood(~/src/scratch/underscore-test) % go list ./...
scratch/underscore-test
deadwood(~/src/scratch/underscore-test) % find . -type d 
.
./.dot
./_underscore

Sorry, what exactly do you mean by "the opposite"? Change dep or change go?

sectioneight · May 12, 2017

Sorry, what exactly do you mean by "the opposite"? Change dep or change go?

You should _not_ change dep to recruse into paths starting with a period or an underscore.

…

On Fri, May 12, 2017 at 1:51 PM, Aiden Scandella ***@***.***> wrote: Sorry, what exactly do you mean by "the opposite"? Change dep or change go ? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#20337 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAAcA91TBk-2k4RS5Fr8-odE5vpkN4Ygks5r49dIgaJpZM4NYyyU> .

davecheney · May 12, 2017

You should _not_ change dep to recruse into paths starting with a period or an underscore.

…

On Fri, May 12, 2017 at 1:51 PM, Aiden Scandella ***@***.***> wrote: Sorry, what exactly do you mean by "the opposite"? Change dep or change go ? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#20337 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAAcA91TBk-2k4RS5Fr8-odE5vpkN4Ygks5r49dIgaJpZM4NYyyU> .

Ok. In that case dep will fail if a dependency has a package named ".foo" since it doesn't see the source code.

Suggestions? Or am I missing something really obvious.

sectioneight · May 12, 2017

Ok. In that case dep will fail if a dependency has a package named ".foo" since it doesn't see the source code.

Suggestions? Or am I missing something really obvious.

You can import that package in another package, but it will not match the globbing rules of the go tool. It sounds like dep is conflating two different operations - for a tree of directories; figure out which contain valid Go packages - for a directory of source files; figure out which ones are are in scope according to build rules.

…

On Fri, May 12, 2017 at 1:55 PM, Aiden Scandella ***@***.***> wrote: Ok. In that case dep will fail if a dependency has a package named ".foo" since it doesn't see the source code... We have a real world example of this today with uber/cherami-client-go/.generated. Suggestions? Or am I missing something really obvious. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#20337 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAAcA2-GXNFtBRQs6sZ0MdQg9R7xf1qYks5r49gzgaJpZM4NYyyU> .

davecheney · May 12, 2017

You can import that package in another package, but it will not match the globbing rules of the go tool. It sounds like dep is conflating two different operations - for a tree of directories; figure out which contain valid Go packages - for a directory of source files; figure out which ones are are in scope according to build rules.

…

On Fri, May 12, 2017 at 1:55 PM, Aiden Scandella ***@***.***> wrote: Ok. In that case dep will fail if a dependency has a package named ".foo" since it doesn't see the source code... We have a real world example of this today with uber/cherami-client-go/.generated. Suggestions? Or am I missing something really obvious. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#20337 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAAcA2-GXNFtBRQs6sZ0MdQg9R7xf1qYks5r49gzgaJpZM4NYyyU> .

(we can probably close this and discuss over in the dep issue)

sdboyer · May 12, 2017

(we can probably close this and discuss over in the dep issue)

sectioneight referenced this issue May 12, 2017
Merged
Ignore dot-prefixed files and subdirs when scanning packages #551

sectioneight closed this May 12, 2017

sectioneight referenced this issue May 12, 2017
Closed
Fix ListPackages to not skip "."-prefixed dirs #564

golang locked and limited conversation to collaborators May 12, 2018

gopherbot added the FrozenDueToAge label May 12, 2018

golang/go

cmd/compile: Unclear guidelines on what files/directories are ignored #20337

sectioneight commented May 12, 2017

sectioneight referenced this issue May 12, 2017

Ignore dot-prefixed files and subdirs when scanning packages #551

This comment has been minimized.

dcheney-atlassian commented May 12, 2017 •

edited

This comment has been minimized.

sectioneight commented May 12, 2017

This comment has been minimized.

sectioneight commented May 12, 2017

This comment has been minimized.

dcheney-atlassian commented May 12, 2017

This comment has been minimized.

sectioneight commented May 12, 2017

sectioneight closed this May 12, 2017

sectioneight referenced this issue May 12, 2017

Fix ListPackages to not skip "."-prefixed dirs #564

This comment has been minimized.

dcheney-atlassian commented May 12, 2017

This comment has been minimized.

sectioneight commented May 12, 2017

This comment has been minimized.

davecheney commented May 12, 2017

This comment has been minimized.

sectioneight commented May 12, 2017 •

edited

This comment has been minimized.

davecheney commented May 12, 2017

This comment has been minimized.

sdboyer commented May 12, 2017

golang locked and limited conversation to collaborators May 12, 2018

gopherbot added the FrozenDueToAge label May 12, 2018

golang/go

Join GitHub today

cmd/compile: Unclear guidelines on what files/directories are ignored #20337

Comments

sectioneight commented May 12, 2017

What version of Go are you using (go version)?

What operating system and processor architecture are you using (go env)?

What did you do?

What did you expect to see?

What did you see instead?

sectioneight referenced this issue May 12, 2017

This comment has been minimized.

dcheney-atlassian commented May 12, 2017 • edited

This comment has been minimized.

sectioneight commented May 12, 2017

This comment has been minimized.

sectioneight commented May 12, 2017

This comment has been minimized.

dcheney-atlassian commented May 12, 2017

This comment has been minimized.

sectioneight commented May 12, 2017

sectioneight closed this May 12, 2017

sectioneight referenced this issue May 12, 2017

This comment has been minimized.

dcheney-atlassian commented May 12, 2017

This comment has been minimized.

sectioneight commented May 12, 2017

This comment has been minimized.

davecheney commented May 12, 2017

This comment has been minimized.

sectioneight commented May 12, 2017 • edited

This comment has been minimized.

davecheney commented May 12, 2017

This comment has been minimized.

sdboyer commented May 12, 2017

golang locked and limited conversation to collaborators May 12, 2018

gopherbot added the FrozenDueToAge label May 12, 2018

What version of Go are you using (`go version`)?

What operating system and processor architecture are you using (`go env`)?

dcheney-atlassian commented May 12, 2017 •

edited

sectioneight commented May 12, 2017 •

edited