x/build: solaris builders are flaky

[bradfitz]

The solaris (smartos) builders are flaky.

The failures are often in TestServerAllowsBlockingRemoteAddr, as reproduced here:

$ gomote run user-bradfitz-solaris-amd64-smartosbuildlet-0 go/bin/go test -short net/http
--- FAIL: TestServerAllowsBlockingRemoteAddr (1.59s)
        serve_test.go:959: Request 1: Get http://127.0.0.1:38230: net/http: request canceled (Client.Timeout exceeded while awaiting headers)
        serve_test.go:1008: response 1 addr = ""; want "RA:21.21.21.21:21"
2017/03/02 19:33:21 http2: server: error reading preface from client 127.0.0.1:50770: read tcp 127.0.0.1:57544->127.0.0.1:50770: use of closed file or network connection
FAIL
FAIL    net/http        7.934s

But running just that test (even 2000 times) is fine:

$ gomote run user-bradfitz-solaris-amd64-smartosbuildlet-0 go/bin/go test -short -run=TestServerAllowsBlockingRemoteAddr -count=2000 net/http
ok      net/http        7.161s

Note that the net/http tests only go passively parallel in -short mode, which results in a higher CPU rate. Are we getting throttled?

Notably, the tests pass when run without short:

$ gomote run user-bradfitz-solaris-amd64-smartosbuildlet-0 go/bin/go test net/http
ok      net/http        47.750s
$ gomote run user-bradfitz-solaris-amd64-smartosbuildlet-0 go/bin/go test net/http
ok      net/http        48.087s

Investigate.

/cc @4ad

[bradfitz]

Oh, our 5 Joyent instances are "t4-standard-1G" which is 1G RAM, 1/2 CPU, 25G disk. Doesn't seem like enough CPU.

[bradfitz]

Quadrupled its CPU, wiped the red column, and things look happy now.