Make it possible to work without starting kaleido

[waralex]

The launch of kaleido (_start_kaleido_process call) takes quite a long time, and it is not always necessary. For example, Dash.jl uses PlotlyBase only to generate json for PlotlyGraph objects. But at the same time, the launch of kaledio takes 90 percent of the time of using Dash (about 2.5 seconds on my laptop).

It would be really great if it could be avoided.
The options that I see are:

• take out the kaleido to the PlotlyJS.jl
• run it in lazy mode on demand
• Make a separate package for working with plotly json

Perhaps there are other options

[alexcjohnson]

Nice find @waralex - on the Python side we do two things that look different from this:
(1) Kaleido is an optional dependency of plotly.py; if we don't find it when starting up, that functionality is disabled. We've discussed making a core package with minimal dependencies, and a "batteries included" package that installs the core and all the optional dependencies. Most users would then use the full package and everything would work, but those with more specific needs could streamline their installation and imports.
(2) Kaleido is lazy in starting its process.

[sglyon]

Some timing results:

Without auto-start kaleido, PlotlyBase needs to precompile

julia> @time using PlotlyBase  # to precompile, NO auto start kaleido
[ Info: Precompiling PlotlyBase [a03496cd-edff-5a9b-9e67-9cda94a718b5]
  1.897951 seconds (2.12 M allocations: 128.707 MiB, 1.05% gc time, 24.61% compilation time)

julia> p = Plot(rand(3, 2));

julia> @time savefig(p, expanduser("~/deleteme.png"));  # first save, with NO already running kaleido
  1.982171 seconds (3.58 M allocations: 194.289 MiB, 4.99% gc time)

julia> @time savefig(p, expanduser("~/deleteme.png"));  # second save, with already running kaleido
  0.049634 seconds (993 allocations: 276.719 KiB)

Without auto-start kaleido, PlotlyBase already precompiled

julia> @time using PlotlyBase  # already precompiled, NO auto start kaleido
  0.270289 seconds (658.20 k allocations: 43.069 MiB, 2.78% gc time, 21.99% compilation time)

julia> p = Plot(rand(3, 2));

julia> @time savefig(p, expanduser("~/deleteme.png"));  # first save, with NO already running kaleido
  1.947703 seconds (3.60 M allocations: 195.190 MiB, 2.14% gc time)

julia> @time savefig(p, expanduser("~/deleteme.png"));  # second save, with already running kaleido
  0.047320 seconds (985 allocations: 275.703 KiB)

With auto-start kaleido, PlotlyBase needs to precompile

julia> @time using PlotlyBase  # to precompile, auto start kaleido
[ Info: Precompiling PlotlyBase [a03496cd-edff-5a9b-9e67-9cda94a718b5]
  3.056955 seconds (3.45 M allocations: 191.059 MiB, 0.99% gc time, 15.18% compilation time)

julia> p = Plot(rand(3, 2));

julia> @time savefig(p, expanduser("~/deleteme.png"));  # first save, with already running kaleido
  0.874112 seconds (2.07 M allocations: 121.761 MiB, 8.67% gc time)

julia> @time savefig(p, expanduser("~/deleteme.png"));  # second save, with already running kaleido
  0.052915 seconds (894 allocations: 205.797 KiB)

With auto-start kaleido, PlotlyBase already precompiled

julia> @time using PlotlyBase  # already precompiled, auto start kaleido
  1.299427 seconds (2.00 M allocations: 106.514 MiB, 1.36% gc time, 3.79% compilation time)

julia> p = Plot(rand(3, 2));

julia> @time savefig(p, expanduser("~/deleteme.png"));  # first save, with already running kaleido
  0.862388 seconds (2.07 M allocations: 121.839 MiB, 2.95% gc time)

julia> @time savefig(p, expanduser("~/deleteme.png"));  # second save, with already running kaleido
  0.047222 seconds (994 allocations: 276.609 KiB)

With auto-start kaleido (async), PlotlyBase needs to be precompiled

julia> @time using PlotlyBase  # to precompile, async auto start kaleido
[ Info: Precompiling PlotlyBase [a03496cd-edff-5a9b-9e67-9cda94a718b5]
  1.905162 seconds (2.12 M allocations: 128.888 MiB, 1.04% gc time, 24.26% compilation time)

julia> p = Plot(rand(3, 2));

julia> @time savefig(p, expanduser("~/deleteme.png"));  # first save, with already running kaleido -- async
  0.881171 seconds (2.06 M allocations: 121.189 MiB, 5.81% gc time)

julia> @time savefig(p, expanduser("~/deleteme.png"));  # second save, with already running kaleido -- async
  0.047295 seconds (954 allocations: 273.969 KiB)

With autto-start kaleido (async), PlotlyBase already precompiled

julia> @time using PlotlyBase  # already precompiled, async auto start kaleido
  0.266814 seconds (660.70 k allocations: 43.346 MiB, 2.81% gc time, 18.71% compilation time)

julia> p = Plot(rand(3, 2));

julia> @time savefig(p, expanduser("~/deleteme.png"));  # first save, with already running kaleido -- async
  0.860420 seconds (2.06 M allocations: 121.192 MiB, 2.09% gc time)

julia> @time savefig(p, expanduser("~/deleteme.png"));  # second save, with already running kaleido -- async
  0.052744 seconds (961 allocations: 274.156 KiB)

---

From what I see here, my vote would be to leave the auto start Kaleido, but to have it launch async.

Any objections?

[sglyon]

One more set of timing:

❯ time julia --startup-file=no --project=@. -e 'using PlotlyBase; p = Plot(rand(3,2)); savefig(p, expanduser("~/deleteme.png"))'  # to precompile -- async start kaleido        
julia --startup-file=no --project=@. -e   3.65s user 0.30s system 89% cpu 4.405 total

❯ time julia --startup-file=no --project=@. -e 'using PlotlyBase; p = Plot(rand(3,2)); savefig(p, expanduser("~/deleteme.png"))'  # already precompiled -- async start kaleido
julia --startup-file=no --project=@. -e   2.11s user 0.13s system 82% cpu 2.703 total

❯ time julia --startup-file=no --project=@. -e 'using PlotlyBase; p = Plot(rand(3,2)); savefig(p, expanduser("~/deleteme.png"))'  # to precompile -- start kaleido      
julia --startup-file=no --project=@. -e   3.62s user 0.19s system 86% cpu 4.422 total

❯ time julia --startup-file=no --project=@. -e 'using PlotlyBase; p = Plot(rand(3,2)); savefig(p, expanduser("~/deleteme.png"))'  # already precompiled -- start kaleido
julia --startup-file=no --project=@. -e   2.06s user 0.10s system 75% cpu 2.838 total

❯ time julia --startup-file=no --project=@. -e 'using PlotlyBase; p = Plot(rand(3,2)); savefig(p, expanduser("~/deleteme.png"))'  # to precompile -- no start kaleido   
julia --startup-file=no --project=@. -e   3.44s user 0.18s system 85% cpu 4.229 total

❯ time julia --startup-file=no --project=@. -e 'using PlotlyBase; p = Plot(rand(3,2)); savefig(p, expanduser("~/deleteme.png"))'  # already precompiled -- no start kaleido
julia --startup-file=no --project=@. -e   2.06s user 0.09s system 77% cpu 2.759 total

If we are running a script to load, plot, savefig it doesn't seem to matter which option we choose

[waralex]

@sglyon Are you sure that it is necessary in PlotlyBase? This is a fairly high level functionality more related to the display than to the model and it seems to me that it should be taken out to PotlyJS with savefig at all.

[sglyon]

It isn't necessary. I would be willing to move this over

[waralex]

The problem is not only in the launch time. By itself, starting a process that is most likely not needed by the user is not very good, IMHO.

[hhaensel]

I want to come back to this topic.

I suddenly realised that savefig was not working anymore and finally found out why.

While I can understand all the arguments above, I'm missing the possibility to save figures with a small footprint. Loading the full PlotlyJS with heavy dependencies like WebIO is worse than loading Kaleido_jll with PlotlyBase.

I'd propose to define an abstract type AbstractPlot with subtypes Plot and SyncPlot and to generate a package PlotlySave that defines savefig for AbstractPlots.

I'd even propose to use FileIO and to define a new save method

import FileIO.save
function save(io, p:AbstractPlot) [...]

and keeping savefig and savejson for compatibility.

@sglyon What do you think?

[hhaensel]

I have published PlotlySave.jl on my GitHub and created branches hh-abstractplot for my forks of PlotlyBase and PlotlyJS.
The tests run without errors for both PlotlyBase and PlotlyJS.