cmd/gorename: too many open files

[anacrolix]

gorename frequently aborts with the error "too many open files". It doesn't always happen. That then triggers a long cascade of failures of "undeclared name".

open /Users/anacrolix/src/go/src/net/ipsock.go: too many open files
open /Users/anacrolix/src/go/src/net/ipsock_posix.go: too many open files
/Users/anacrolix/src/go/src/net/sock_posix.go:20:2: undeclared name: netaddr

[anacrolix]

This seems relevant, it's the default on my system:

$ ulimit -a
open files                      (-n) 256

[adonovan]

Thanks for the report. I thought I had fixed this already. Will take a look tomorrow.

[anacrolix]

I was on revision 473fd85 of go.tools

[alandonovan]

I was unable to reproduce the problem by setting ulimit -n; I only noticed problems when the limit was so low that it couldn't even get started. But this CL should reduce the concurrency of file I/O:

https://go-review.googlesource.com/#/c/8421/

(The earlier fix I mentioned was for a different parallel I/O operation in the same tool.)

Could you try patching this change to see if it helps? Thanks.

[bakineggs]

I ran into this issue a lot last week when using errcheck, so I submitted CL #8475 as a fix. I didn't realize there was an open issue and a different fix until I saw the conflicting change listed in Gerrit.

On a project I work on, errcheck fails with this error the majority of the time using the master revision of go/loader. It does not fail with this error using the branch from CL 8421 or CL 8475 (although with CL 8421, it should still be possible to have this error occur in conf.FindPackage).

[alandonovan]

I agree that it's still possible in principle for the error to occur. (It's an unfortunate consequence of ignoring the possibility of I/O failures in functions like (*go/build.Context) isDir.) But I think the CL 8421 fix should eliminate it in practice; I suggest you revert CL #8475.

If the problem recurs, we'll revisit that change.

[pmezard]

FWIW, I reproduced it running go/master db2e73f, with ulimit -n == 256. The renaming happened but a lot of "too many open files" errors were displayed.

[anacrolix]

I still get it on occasion with go1.5.

[anacrolix]

anacrolix@Matts-MacBook-Pro:~/tmp$ gorename -from '"github.com/anacrolix/torrent".torrent.DisplayName' -to displayName
While scanning Go workspace:
Package "github.com/mattn/go-gtk/example/expander": open /Users/anacrolix/gopath/src/github.com/mattn/go-gtk/example/expander/expander.go: too many open files.
Renamed 3 occurrences in 2 files in 1 package.

[mbertschler]

I also have this problem often, but not every time I run it.
go version go1.5.1 darwin/amd64