Precompilation of DataFrames

[bkamins]

I think it would be good to put some most common cases of functions used in DataFrames.jl to the module to force their precompilation as this will improve user experience (and DataFrames.jl is probably one of the packages that new users start with). Any opinions if we should do it and how (I could manually list the cases I think are relevant and code it by hand in the worst case).

[nalimilan]

Good idea. We should also do this for dependencies, in particular CategoricalArrays. A reasonable way of doing that would be to run the full test suite under SnoopCompile.jl.

[quinnj]

Honestly I haven't found using precompile statements to be very helpful (or at least worth the effort to generate them). The problem is that precompile still doesn't generate machine code, so it's really just the lowering cost that you're avoiding. What has been more useful is to include a simple couple calls in the top-level __init__ function that will get called every time the module is loaded. Yes, it adds a little extra time to the initial module loading, but IMO, it can be worth it to have the penalty there instead of the first time a user calls a function.

[nalimilan]

Given how big is the DataFrames API, it doesn't really sound possible to call all functions from __init__. At least setting up precompilation will be useful in preparation of the future Julia version where native code will be stored.

[KristofferC]

+1 for precompile statements not helping much unless it is paired with sysimg building (and then they help tremendously, c.f the Julia REPL).

[nalimilan]

While discussing the h2oai benchmarks, it appeared that it would be useful to provide a function or script to compile functions for all common types. That would allow reporting times net of the initial compilation cost, which is generally only paid once, and in particular would allow separating it from the cost of specializing some functions on the particular operation at hand. See discussion at h2oai/db-benchmark#69.

I guess an easy way to do that would be to copy the output of SnoopCompile to an unexported function, which could be called by the h2oai benchmark, and possibly by users who want to put that in their .juliarc.

[bkamins]

I think it would be worth to do it at least to understand what should be compiled and how long this compilation takes. Then I guess people who want to build a custom sysimg instead of using .juliarc (which is now ~/.julia/config/startup.jl right?) also would have an easier life.

[nalimilan]

Right. Let's try using SnoopCompile while running the full test suite.

[bkamins]

+1 for precompile statements not helping much unless it is paired with sysimg building

@KristofferC Is this still true for Julia 1.4?

Here is a comparison (I am annotating by with @nospecialize - not sure if this matters)

Without precompile:

julia> @time using DataFrames
  0.924471 seconds (1.39 M allocations: 88.058 MiB)

julia> df = DataFrame(rand(10,4));

julia> @time by(df, :x1, [:x2, :x3] => +);
  4.523086 seconds (15.01 M allocations: 759.657 MiB, 8.18% gc time)

With precompile:

julia> @time using DataFrames
  1.311571 seconds (1.60 M allocations: 103.156 MiB)

julia> df = DataFrame(rand(10,4));

julia> @time by(df, :x1, [:x2, :x3] => +);
  3.059164 seconds (8.85 M allocations: 458.088 MiB, 2.41% gc time)

So using precompile increases load time by ~0.4 sec, but decreases time to first result by ~1.4 sec . So in total 1 second is saved.

[KristofferC]

It might have changed or I didn't have a correct assessment earlier. I know Plots.jl has added some and it seems to help there. So if the numbers say it helps, I say go for it.

[bkamins]

@nalimilan - so I will put out all new @nospecialize changes in #2158 and will think what can be proposed for @nospecialize and precompile combinations in this issue (I am not yet sure how they interact in practice and have to investigate).

[bkamins]

@nalimilan - I have checked what we have now. We could handle most of the compilation latency after the package is loaded if we included the following in the init file:

let
    df = DataFrame(a=1, b=2)
    g = groupby(df, :a)
    combine(g, :a => sum)
    innerjoin(df, df, on=:a, makeunique=true)
    unstack(stack(df, :a))
end;

It would make the package startup time grow from 0.9 sec to 5.5 sec. So for now this is for sure prohibitive, but maybe when we separate DataFramesBase.jl actually it could be acceptable to have such code in DataFrames.jl? It is normal for other packages to load in 5-10 seconds (I do not want to say that this is something users like of course).

(we could alternatively do precompile but I think it is simplest just to run the functions)

[KristofferC]

I really dislike packages running unnecessary code at startup. What if I only use the DataFrames functionality spuriously in my session, why are you adding 5 extra seconds of completely avoidable latency to me? It can also introduce invalidations that cause other things to have to be recompiled.

Unconditionally shifting eventual compilation cost to unconditional package load time cost seems like a very bad trade off.

Ref, JuliaData/CSV.jl#325.

[bkamins]

@KristofferC - sure.

BTW: are you aware of the current state of the work towards better caching the compiled code between Julia sessions?

[nalimilan]

I agree it doesn't sound like a good tradeoff to impose 5s on every load to make things faster afterwards. And we never know what kind of functions users will call, so we'd have to do that for all functions...

Would adding these to precompilation give some speedup? With SnoopCompile it should be easy to transform a script into a series of precompile statements.