cmd/compile: only use sequential compile queue algorithm for -v compiles, not all -c=1 compiles

[mdempsky]

The other day I was seeing issues with "toolstash -cmp" failing because compiler output seemed non-deterministic. I submitted golang.org/cl/318229 to change non-concurrent builds to have a deterministic order, on the assumption that concurrent builds are expected to have non-deterministic output. But @josharian points out that's not the case.

However, I can no longer immediately reproduce the non-determinism issue, even with high -c values.

This is a tracking issue to check that with all the compiler refactoring this cycle we haven't broken deterministic output.

/cc @randall77 @cuonglm

[mdempsky]

Okay, it turns out the issue I was running into wasn't that the compiler output was non-deterministic, but just the compiler diagnostics were non-deterministic. This in turn made diffing the log files generated by toolstash -cmp useless.

CL 318229 was unnecessary for deterministic compiler output, but it does correctly make the diagnostics deterministic as needed for toolstash. I think that CL can stay for 1.17.

For 1.18, I think it might make sense to tweak this logic:

go/src/cmd/compile/internal/gc/compile.go

Line 128 in 5203357

if nWorkers := base.Flag.LowerC; nWorkers > 1 {

to keep normal -c=1 builds using the same work queue algorithm as -c>1 builds, and only use the strictly in-order algorithm for -v builds or something.

[josharian]

Phew. Have you confirmed that CL 318229 doesn’t cause a performance regression?

[mdempsky]

@josharian I did not, but I don't anticipate it should:

1. It only changed the behavior of -c=1 builds. If anything, I'd expect compiling all functions on a single goroutine would be marginally faster than spinning up one goroutine per function, and then serializing them on a 1-element semaphore.
2. It's only during the 1.17 dev cycle that we changed -c=1 and -c>1 builds to use the same algorithm. So even if CL 318229 did have any performance impact, it shouldn't be any different than Go 1.16's performance (which used essentially the same algorithm).

[josharian]

Ah, sorry, I missed that. Thanks!

[ianlancetaylor]

@mdempsky This is in the 1.18 milestone; time to move to 1.19? Thanks.

[cherrymui]

One place I need deterministic compiling order is for debugging the compiler. Once I had a compiler bug that caused compiler crashes in multiple functions (in the same package). Once it crashed, it stopped compiling other functions. What function it crashed in was nondeterministic. I wanted it to always crash in the same function, so I can get an SSA dump and some debug outputs. I tried -c=1 but it wasn't deterministic (maybe that changed?). I think there would be good to have a way to enforce deterministic order.

[ianlancetaylor]

@mdempsky This is in the 1.19 milestone; time to move to 1.20? Or Backlog? Thanks.