Consider lower optimization defaults #36025
Description
A motivating quote (see here where I found it, not just my 45% slowdown on Julia's defaults on code I wrote at work): https://discourse.julialang.org/t/rethinking-optimization-lower-ok-for-all-julia-code-for-e-g-faster-startup-as-a-default/39843
"optimising compilers are operating in a region between a problem that’s not worth solving (most parts of most programs) and a problem they can’t solve (the really performance critical parts)"
Especially with @JeffBezanson's comment on LLVM getting slower (is that later slower version only used in 1.6, not 1.5?), consider defaulting to -O1, at least in 1.6, where the default can be 45% slower on a rather long-running script, and -O1 already being a compromise, as -O0 is even faster:
$ hyperfine '~/julia-1.6.0-DEV-8f512f3f6d/bin/julia -O0 --startup-file=no fenics-db.jl'
Benchmark #1: ~/julia-1.6.0-DEV-8f512f3f6d/bin/julia -O0 --startup-file=no fenics-db.jl
Time (mean ± σ): 13.717 s ± 0.102 s [User: 13.721 s, System: 0.761 s]
Range (min … max): 13.555 s … 13.898 s 10 runs
$ hyperfine '~/julia-1.6.0-DEV-8f512f3f6d/bin/julia -O1 --startup-file=no fenics-db.jl'
Time (mean ± σ): 13.959 s ± 0.097 s [User: 13.966 s, System: 0.761 s]
Range (min … max): 13.816 s … 14.103 s 10 runs
$ hyperfine '~/julia-1.6.0-DEV-8f512f3f6d/bin/julia --startup-file=no fenics-db.jl'
Time (mean ± σ): 19.848 s ± 0.135 s [User: 19.822 s, System: 0.777 s]
Range (min … max): 19.630 s … 20.045 s 10 runs
$ hyperfine '~/julia-1.5-ea669c3d3e/bin/julia -O0 --startup-file=no fenics-db.jl'
Time (mean ± σ): 12.940 s ± 0.082 s [User: 12.893 s, System: 0.802 s]
Range (min … max): 12.855 s … 13.105 s 10 runs
This here is a repeated timing (no thermal effects) for the one below it:
$ hyperfine '~/julia-1.5-ea669c3d3e/bin/julia -O1 --startup-file=no fenics-db.jl'
Time (mean ± σ): 12.970 s ± 0.087 s [User: 12.964 s, System: 0.760 s]
Range (min … max): 12.853 s … 13.075 s 10 runs
$ hyperfine '~/julia-1.5-ea669c3d3e/bin/julia -O1 --startup-file=no fenics-db.jl'
Time (mean ± σ): 15.861 s ± 2.509 s [User: 15.691 s, System: 0.838 s]
Range (min … max): 13.002 s … 18.307 s 10 runs
[for that timing above I could hear they fan spinning up, seems got slower over the 10 runs (but min should be reliable far all tests), while I got no outlier warning for any test other than the last test below.]
$ hyperfine '~/julia-1.5-ea669c3d3e/bin/julia --startup-file=no fenics-db.jl'
Time (mean ± σ): 18.363 s ± 0.871 s [User: 18.308 s, System: 0.806 s]
Range (min … max): 17.750 s … 20.764 s 10 runs
Warning: Statistical outliers were detected. Consider re-running this benchmark on a quiet PC without any interferences from other programs. It might help to use the '--warmup' or '--prepare' options.
Yes, -O1 seem like a sensible default (a compromise) for scripts, maybe not all code, where I think we should rely on other tricks, non-default options for with e.g. fast JLL libraries):
https://discourse.julialang.org/t/precompile-a-script/5364/17?u=palli
I see people are giving up on Julia (for scripts, or totally), and some avoid all packages (also a very bad thing), and see my answer above to the former.