x/playground: easily runs into build timeouts by downloading modules

[mvdan]

For example, see https://go.dev/play/p/b4S-VWK-yWR, which needs to download https://pkg.go.dev/cuelang.org/go@v0.5.0-beta.1 and its dependencies. It currently times out at build time, presumably while still downloading the module zips:

timeout running go build
go: downloading cuelang.org/go v0.5.0-beta.1
go: downloading github.com/cockroachdb/apd/v2 v2.0.2
go: downloading github.com/mpvl/unique v0.0.0-20150818121801-cbe035fff7de
go: downloading golang.org/x/text v0.3.8
go: downloading golang.org/x/net v0.0.0-20220722155237-a158d28d115b
go: downloading github.com/google/uuid v1.2.0
go: downloading github.com/pkg/errors v0.8.1
go: downloading gopkg.in/yaml.v3 v3.0.1

Go build failed.

It appears that the timeout happens after about ten seconds.

I wrote a small program a while ago to estimate how big all the zips in go list -m all are. The results, starting with the largest in bytes, are as follows:

$ go-mod-size | sort -k2 -n -r
golang.org/x/text@v0.3.8 8615981
golang.org/x/tools@v0.1.12 3946738
cuelang.org/go@v0.5.0-beta.1 2062942
golang.org/x/sys@v0.0.0-20220722155257-8c9f86f7a55f 1797601
golang.org/x/net@v0.0.0-20220722155237-a158d28d115b 1582191
github.com/cockroachdb/apd/v2@v2.0.2 320918
github.com/protocolbuffers/txtpbfmt@v0.0.0-20220428173112-74888fd59c2b 271386
github.com/pkg/diff@v0.0.0-20210226163009-20ebb0f2a09e 219375
github.com/rogpeppe/go-internal@v1.9.0 210263
golang.org/x/mod@v0.6.0-dev.0.20220818022119-ed83ed61efb9 169515

So a clean go build does download a fair bit - at least 15MiB or so. The total number of modules is 32, per go list -m all | wc -l. I think the playground should be able to download these modules by doing any of:

1. Talking to a fast GOPROXY. GOMODCACHE=$PWD/tmp time go build reports a total runtime of about 3s on my laptop, with a 500Mb ethernet connection and a 11ms ping to proxy.golang.org. I'm sure that a playground server could do better.

2. Reusing a GOMODCACHE on a best-effort basis. The same could be said about GOCACHE for build caching for compile timeouts.

3. Increasing the timeout for multi-file playground inputs with go.mod and go.sum files. For example, given the number of lines in either go.mod or go.sum, we could estimate that this example needs a 20s timeout rather than 10s. Presumably this shouldn't be too harmful, as the playground should cache the result once it runs without a timeout.

4. Implementing a more sophisticated timeout. For example, the playground could have very generous timeouts for downloading modules, given that network traffic is presumably cheap. The timeout for compiling and linking could similarly be more generous than the timeout for running the program, as there's no chance for running arbitrary code to burn CPU or abusing free compute.

cc @toothrot per https://dev.golang.org/owners

[mvdan]

Worth noting that, since go build prints what modules it's downloading, but not what packages it's buildling, it could be that my timeout above is during the compilation or linking of the program. I don't think I can know for sure. If it is indeed not a "module download" timeout, I think that should be made clearer.

[findleyr]

We discussed this in the tools call yesterday, and I feel like I may have prematurely determined that this problem is not easily solvable.

If timeouts are during the build, not the download, we should consider increasing the size of the playground app engine instances:
https://cs.opensource.google/go/x/playground/+/master:app.yaml;l=8;drc=9bfb5bdef13596a261db44e1c28a9fc8ddd7bc77

Right now, they appear to be at the minimum number of CPU:
https://cloud.google.com/appengine/docs/flexible/reference/app-yaml?tab=go#resource-settings

It may be worth just 2x'ing or 4x'ing the instance shape to see if it improves anything.

CC @golang/release

[mvdan]

Two CPUs and two gigabytes of memory don't sound like enough to build some non-trivial examples with dependencies in a reasonable amount of time :) I would agree with that. Though I would also like to see better caching and longer build timeouts.

[gopherbot]

Change https://go.dev/cl/461795 mentions this issue: playground: 4x resources for the tip playground

[mvdan]

For what it's worth, the higher resources don't seem to get us out of the timeouts. Relatively light examples like https://go.dev/play/p/qPDCgnJgO39 still trigger a timeout. Some dependencies like x/text and x/tools still weigh megabytes, but surely that shouldn't be out of the norm :)

[dolmen]

The Go playground is an important tool for the Go community as it allows to easily publish reproducers for issues in the Go project, but also beyond on Go modules that are publicly available. We were relying on this capability for the maintenance of the Testify testing framework, as it is easier to reproduce old issues if a reproducer is directly available on go.dev/play: Testify has a long queue of old issues and easy access to a reproducer in an uptodate environment is key for maintainers' limited time.

However nowadays playground code that uses Testify always fails with a "timeout running go build". Example: https://go.dev/play/p/GWnpcyJSw5T