runtime: "go install -a" needs quite a few user processes and doesn't cope if they are restricted

[garyhouston]

Please answer these questions before submitting your issue. Thanks!

1. What version of Go are you using (go version)?
   1.7
2. What operating system and processor architecture are you using (go env)?
   amd64, linux
3. What did you do?
   Rebuilding a program that was fine in 1.6, with:
   go install -a
4. What did you expect to see?
   normal compilation
5. What did you see instead?
   failed with:

# sort
runtime: failed to create new OS thread (have 3 already; errno=11)
runtime: may need to increase max user processes (ulimit -u)
fatal error: newosproc

(followed by stack dump and several similar messages.)

environment:

ulimit -u gives 30 (it's not my server and I can't increase it)

setting GOMAXPROCS=1 was a work-around that gave successful compilation.

[bradfitz]

I think you'll probably have to stick with your workaround. 30 is quite low. It's conceivable you have more processors than that, in which case Go's default GOMAXPROCS would already violate your ulimit.

@ianlancetaylor? @aclements?

[garyhouston]

4 processors. Is there a limit to the number of user processes the build system is likely to need, or does it just increase with the number of files in the project?

[davecheney]

-p=1 will limit the number of concurrent compile jobs
GOMAXPROCS=1 should limit the number of concurrently running goroutines
(including gc workers).

This should reduce the number of threads on, say a 4 processor system, by a
factor of 4^2

On Tue, Aug 16, 2016 at 3:29 PM, Gary Houston notifications@github.com
wrote:

4 processors. Is there a limit to the number of user processes the build
system is likely to need, or does it just increase with the number of files
in the project?

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#16713 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAAcA9Z__odVtgLOA0pJbzkRppEsVN1Xks5qgUqkgaJpZM4Jk8Hh
.

[aclements]

@garyhouston, you say that it hits the limit "earlier" than in 1.6. I'm not at all surprised that it hits a limit of 30, but I'm somewhat surprised that it changed from 1.6 to 1.7. At what point did it fail in 1.6?

It could simply be that 1.7 has a slightly different package graph, which causes it to hit a more concurrent part earlier.

At any rate, I think your workaround (or using -p) is the right answer to an extremely low process limit.

[garyhouston]

Thanks, I made an unwarranted assumption that it "was fine in 1.6", because I never encountered a problem like this previously. But perhaps I didn't use go install -a in 1.6, or maybe my code has changed in some significant way.

I reinstalled 1.6.3 and tried the same command, and got:
runtime/cgo: pthread_create failed: Resource temporarily unavailable

Today I get the same error in 1.7, instead of the original error that helpfully pointed out the ulimit issue. A regular install, without -a, after modifying only a few files, doesn't have any problem.

[aclements]

30 threads is not remotely "quite a few" by today's standards. It's not really reasonable for Go to attempt to automatically deal with such a constrained environment. Given that there appear to be two viable workarounds and this is not a regression from early Go versions, I'm going to go ahead and close this issue.