Performance regression

[michel2323]

I observed a regression issue going from Enzyme.jl v0.11.0 to v0.11.1.

v0.11.1:

401.197621 seconds (69.04 M allocations: 4.089 GiB, 99.80% compilation time)

v0.11.0:

81.696286 seconds (46.88 M allocations: 2.844 GiB, 99.41% compilation time)

The pprof files are attached.

The code is the one by @swilliamson7 https://github.com/DJ4Earth/eddy-stresses/blob/main/src/main_energy_1.jl .
profile_11.0.pb.gz
profile_11.1.pb.gz

[swilliamson7]

That repo isn't set to public, but I just put the newest version of the code here: https://github.com/DJ4Earth/DJ-barotropic-gyre. The scripts Michel is referring to are in explicit_solver, and the file main_energy_1_chkp.jl runs the whole model

[vchuravy]

• 11.0: https://pprof.me/b214a6e/
• 11.1: https://pprof.me/b2fbacf/

[wsmoses]

Retry on main?

[michel2323]

Some preliminary report after fiddling with it for a while. I somehow get a new error with v0.11.1 and main, which drops after 8 minutes or something. v0.11.0 goes through as usual. Attached is the error log. I thought it's something on in our code that changed, but it may be related to the Julia version or the system (development server is down today). I have to investigate more...
bug_11.1.log

[wsmoses]

@michel2323 can you rerun with this branch and post the log? #822 It just adds some extra information so we can figure out what's going awry.

[michel2323]

@wsmoses Here are the logs. There are two. One with GC enabled and one with GC disabled because they give a different printout.
logs.zip

[wsmoses]

@michel2323 try again on main, I think it should be fixed.