[WIP] Automatically create example images on build

[SaschaMann]

(fixes #59)

---

Just a WIP commit to signal that I am working on this to avoid duplicate work. GR has been ported already, I'll get to the other backends in the next few days.

For better readability of the diff in the PR, I'll push one commit per backend. Feel free to squash on merge if you prefer.

[daschw]

Awesome! I'm really excited to see someone work on this!

[SaschaMann]

I've had some issues getting the other backends to run properly on my system, so I'll need a few more days.

---

Building the example images takes a lot of time and requires all backends to work properly. If you want to build docs locally after making a change, this can be quite annoying.

I was thinking of setting an env variable via an argument passed to make.jl, like julia make.jl --build-images={none|gr|plotly|...} to restrict it to certain backends and then adding skip || savefig("img/gr/lines.svg"); nothing # hide to each example instead of just savefig("img/gr/lines.svg"); nothing # hide. Is there a better/nicer way to do this?

[SaschaMann]

Sorry, took me a bit longer than I thought to come back to this.

---

Some of the PGFPlot examples throw errors, that I'm not sure how to fix. Couldn't find a solution via Google and I have no experience with PGFPlots, perhaps you can help me out.

Global

y = rand(20,3)
plot(y,xaxis=("XLABEL",(-5,30),0:2:20,:flip),background_color=RGB(0.2,0.2,0.2),leg=false)

hline!(mean(y,1) + rand(1,3),line=(4,:dash,0.6,[:lightgreen :green :darkgreen]))
! Dimension too large.
\pgfplotsaxisdeserializedatapointfrom@private@LUA ...
                                                  \global \pgf@y =#2pt \pgfp...
l.161 \end{axis}

Error saving as SVG
Error showing value of type Plots.Plot{Plots.PGFPlotsBackend}:
ERROR: LaTeX error

The following commands

vline!([5,10])
title!("TITLE")
yaxis!("YLABEL",:log10)
savefig("img/pgfplots/global.svg"); nothing # hide

throw similar errors (Dimension too large).

Subplots

julia> l = @layout([a{0.1h};b [c;d e]])
Plots.GridLayout(2, 1)

julia> plot(randn(100,5),layout=l,
            t=[:line :histogram :scatter :steppre :bar],leg=false,ticks=nothing,border=false)
Error showing value of type Plots.Plot{Plots.PGFPlotsBackend}:
ERROR: MethodError: no method matching pgf_framestyle(::Bool)
Closest candidates are:
  pgf_framestyle(::Symbol) at /home/sascha/.julia/v0.6/Plots/src/backends/pgfplots.jl:121
Stacktrace:
 [1] pgf_axis(::Plots.Subplot{Plots.PGFPlotsBackend}, ::Symbol) at /home/sascha/.julia/v0.6/Plots/src/backends/pgfplots.jl:375
 [2] _update_plot_object(::Plots.Plot{Plots.PGFPlotsBackend}) at /home/sascha/.julia/v0.6/Plots/src/backends/pgfplots.jl:480
 [3] prepare_output(::Plots.Plot{Plots.PGFPlotsBackend}) at /home/sascha/.julia/v0.6/Plots/src/plot.jl:274

Annotations

julia> plot(y,annotations=(3,y[3],text("this is #3",:left)),leg=false)
! Illegal parameter number in definition of \pgfplots@stored@current@data.
<to be read again>
3
l.44 ] {this is #3};

Error saving as SVG
Error showing value of type Plots.Plot{Plots.PGFPlotsBackend}:
ERROR: LaTeX error

julia> annotate!([(5,y[5],text("this is #5",16,:red,:center)),
                 (10,y[10],text("this is #10",:right,20,"courier"))])
! Illegal parameter number in definition of \pgfplots@stored@current@data.
<to be read again>
3
l.44 ] {this is #3};

Error saving as SVG
Error showing value of type Plots.Plot{Plots.PGFPlotsBackend}:
ERROR: LaTeX error

julia> scatter!(linspace(2,8,6),rand(6),marker=(50,0.2,:orange),
                series_annotations=["series","annotations","map","to","series",
                                    text("data",:green)])
! Illegal parameter number in definition of \pgfplots@stored@current@data.
<to be read again>
3
l.87 ] {this is #3};

Error saving as SVG
Error showing value of type Plots.Plot{Plots.PGFPlotsBackend}:
ERROR: LaTeX error

DataFrames

julia> @df iris scatter(:SepalLength,:SepalWidth,group=:Species,title="My awesome plot",xlabel="Length",ylabel="Width",marker=(0.5,[:+ :h :star7],12),bg=RGB(0.2,0.2,0.2))

ERROR: UndefVarError: @df not defined

using DataFrames doesn't fix this, where does @df come from?

[mkborregaard]

Great!. You need using StatPlots to use @df. I know very little about pgfplots myself.

[SaschaMann]

Thanks for the quick response! :)

I'm going to push my commits expecting the build to fail but I want to see if Travis shows any additional errors that I don't get locally.

I'll clean up the git history afterwards, it's a bit of a mess right now

---

Ran into another issue with InspectDR, but I suspect this is a problem with the backend and not Plots or this example:

Bar & Histogram

julia> bar(randn(99))

julia> WARNING: Error in @guarded callback
ERROR: MethodError: no method matching isglyph(::InspectDR.GlyphPolyline)

histogram(randn(1000),nbins=20) & plot(x,y,line=(3,:dash,:lightblue),marker=(Shape(verts),30,RGBA(0,0,0,0.2)),bg=:pink,fg=:darkblue,xlim=(0,1),ylim=(0,1),leg=false) cause the same error.

[mkborregaard]

What's the current status on this?

[SaschaMann]

Few minor things are left that I haven't had time for yet (will try to finish it this week):

• PyPlot
• Adding all dependencies to travis
• Figuring out why some examples are broken (see above)

However, a bigger issue is that it takes forever to build docs locally to test changes because it generates every example image in the process. I haven't found a decent solution to this yet. I think ideally there should be a Documenter feature that lets you skip the execution of certain named examples locally, just like you can skip doctests.

[daschw]

I'm sorry that I only take a closer look at this now, but you should not edit the files in docs/src/examples. They are automatically generated in https://github.com/JuliaPlots/PlotDocs.jl/blob/master/src/PlotDocs.jl#L47-L106. So you only need to do the work once and not for each backend.

[SaschaMann]

Ah, I didn't see that code at all, thanks for the heads up. I'll use that instead, it doesn't have the issue of long build times for every change either.

[mkborregaard]

@SaschaMann @ChrisRackauckas Another advantage of this is that we would know when our front-page examples and the tutorial becomes out of date JuliaPlots/Plots.jl#1648

@anthony-wang has implemented a system to build the images automatically with Documenter for Makie - maybe we could copy that solution?

[SaschaMann]

@michael K. Borregaard I've looked at the way Makie builds the examples and while it seems really nice, it's much more complex than the script that @daschw linked above.

Another advantage of this is that we would know when our front-page examples and the tutorial becomes out of date JuliaPlots/Plots.jl#1648

Perhaps changes to Plots should trigger a CI build of the docs repo to see if anything needs updating. Otherwise you'll only know that it becomes out of date when you change the docs directly which only happens every 1-2 months, while changes to Plots are much more frequent.

[mkborregaard]

Alternatively we could add all of the figures in our docs to our test examples, and link to those in PlotReferenceImages from here. If we just had a "latest" folder there. In fact, that may be the simplest solution overall.

[mkborregaard]

We could just internally allow failures by catching exceptions from those tests and backends we don't want to break on Travis.

[SaschaMann]

Alternatively we could add all of the figures in our docs to our test examples, and link to those in PlotReferenceImages from here. If we just had a "latest" folder there. In fact, that may be the simplest solution overall.

So instead of generating the example plots during the docs building, the docs would simply link to the reference image repos? That does sound like a very simple solution. It also gets around the long build times of running it locally. It would turn the examples into doctest-like things.

One downside could be that the code blocks in the docs can diverge from the example image, because they'd have to be updated in two locations.

[mkborregaard]

Not if we pull them in from Plots when running the docs, like we do with the backend-specific
example code.

[daschw]

Alternatively we could add all of the figures in our docs to our test examples, and link to those in PlotReferenceImages from here. If we just had a "latest" folder there. In fact, that may be the simplest solution overall.

I really like that suggestion.

[SaschaMann]

Sounds like a plan then. I will close this PR for now and move it to an issue. I might have enough time to implement this new plan once my exams are over (last week of sept.). Sorry again for all the delays between answers :/