v1.11 generates 50% larger cache files

[joa-quim]

THe compile time is quite variable but the differences are comparable to this.

  | | |_| | | | (_| |  |  Version 1.10.2 (2024-03-01)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

julia> cd("c:/v"); @time using GMT
Precompiling GMT
  1 dependency successfully precompiled in 48 seconds. 87 already precompiled.
 49.657821 seconds (4.51 M allocations: 328.234 MiB, 0.21% gc time, 1.84% compilation time)

  | | |_| | | | (_| |  |  Version 1.11.0-alpha1 (2024-03-01)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

julia> cd("c:/v"); @time using GMT
Precompiling GMT
  1 dependency successfully precompiled in 60 seconds. 112 already precompiled.
 61.731323 seconds (4.22 M allocations: 263.147 MiB, 0.35% gc time, 1.69% compilation time: 15% of which was recompilation)

v1.11 cache file -> ~89.5 MB
v1.10 -> ~59.5 Mb

Load times

  | | |_| | | | (_| |  |  Version 1.10.2 (2024-03-01)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

julia> @time_imports using GMT
               ┌ 2.7 ms SuiteSparse_jll.__init__()
     25.6 ms  SuiteSparse_jll 85.64% compilation time
               ┌ 5.0 ms SparseArrays.CHOLMOD.__init__() 98.93% compilation time
    123.1 ms  SparseArrays 3.99% compilation time
      0.7 ms  Statistics
      0.2 ms  DataValueInterfaces
      0.6 ms  DataAPI
      0.2 ms  IteratorInterfaceExtensions
      0.2 ms  TableTraits
      6.3 ms  Tables
      0.2 ms  Reexport
     12.1 ms  Preferences
      0.3 ms  PrecompileTools
      5.7 ms  StringManipulation
     10.3 ms  Crayons
      0.6 ms  LaTeXStrings
     63.7 ms  PrettyTables
               ┌ 17.4 ms GMT.Gdal.__init__()
               ├ 16.6 ms GMT.__init__()
    319.5 ms  GMT

  | | |_| | | | (_| |  |  Version 1.11.0-alpha1 (2024-03-01)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

julia> @time_imports using GMT
      0.7 ms  Statistics
      0.3 ms  DataValueInterfaces
      0.5 ms  DataAPI
      0.2 ms  IteratorInterfaceExtensions
      0.2 ms  TableTraits
      6.8 ms  Tables
      0.4 ms  Reexport
      8.9 ms  Preferences
      0.4 ms  PrecompileTools
      5.8 ms  StringManipulation
     11.7 ms  Crayons
      0.6 ms  LaTeXStrings
     69.4 ms  PrettyTables
               ┌ 18.5 ms GMT.Gdal.__init__()
               ├ 108.3 ms GMT.__init__() 88.15% compilation time (100% recompilation)
    814.4 ms  GMT 54.63% compilation time (59% recompilation)

[JeffBezanson]

Looks like, partly, new invalidations in GMT.__init__()? (and therefore possibly in other things too)

[JeffBezanson]

Maybe similar to #53511 ?

[KristofferC]

Would be interesting to put back all stdlibs into the sysimage and re-time this to see if the effect is purely for moving out stdlibs or if there are other reasons as well.

[mkitti]

Would be interesting to put back all stdlibs into the sysimage and re-time this to see if the effect is purely for moving out stdlibs or if there are other reasons as well.

We should consider building this and releasing it as an artifact. It would be using at least for testing and some loading sensitive applications as well.

[jaakkor2]

Worst offender in the precompiled size I have seen is https://github.com/Gnimuc/GLTF.jl. v1.11.0-beta1 (211 MB) is about 5x bigger than v1.10.3 (43 MB).

[KristofferC]

As you showed in quinnj/JSON3.jl#279, the issue there seems to be egregious use of @inline on large functions. It isn't obvious why that would change between 1.10 to 1.11 but one reason could be that we are better at precompiling now so more of the (very big due to `@inline) functions get saved to the image file.

[KristofferC]

This seems to be more or less fixed on 1.11 backport branch. This is using the master branch of GMT.jl:

julia> @time using GMT
  0.638436 seconds (668.47 k allocations: 51.051 MiB, 5.54% gc time, 2.08% compilation time)

julia> VERSION
v"1.10.3"

julia> @time using GMT
  0.657463 seconds (730.44 k allocations: 49.958 MiB, 1.27% gc time, 1.48% compilation time)

julia> VERSION
v"1.11.0-beta1.40"

[joa-quim]

But the precompiled image is still ~50% larger.