Vizcon

[mkborregaard]

From Slack:
OK, so I promised to summarise in a few words what we talked about at the informal, beer-based BOF on plotting at JuliaCon (which ended up being essentially people contributing to Plots - @oschulz @pfitzseb @ChristianKurz plus @anthony-wang). Here are some notes:

First, we talked about what julia needs in terms of plotting libraries. Everybody agreed that there are some highquality libraries outside of julia, and it would make sense to use them, primarily the grammar-of-graphics libraries that focus explicitly on plotting data-in-dataframes. Because the table format is so common it’s very easy to use the best tools already out there, and VegaLite was talked about as a promising example.

We also agreed that this could not cover all uses of visualization - there are plots of shapefile maps, rasters, phylogenies, (light)graphs, neural network visualizations, statistical models etc etc. These are characteristic about being plotting methods for user types - the thing that really makes julia julia is the primacy of user types. For that use, we need a julia-package that knows about julia types, and we need a way to specify how types are plotted. Plots has solved this with recipes, and it works.

We then discussed that Plots has some weaknesses, in that it’s somewhat clunky for interactively updating big visualizations, and relies on backends that are very hard to maintain. Everybody was really curious about how Makie developed but felt we didn’t know enough about it yet to pass judgment. We hoped @anthony-wang could channel @SimonDanisch 🙂 It was particularly hard for us to be clear about how the interface to recipes would be. However we all had our hopes up!

There’s also the issue with start-up time, and we discussed whether replacing the plotattributes dict with a NamedTuple would make it easier for the package to precompile methods. We also talked about the old idea of breaking off each backend into backend packages, and @oschulz wanted to try it out in practice. Finally we discussed the possibility of refactoring the recipe framework to not rely on Macros, given that julia is more mature and powerful than in 0.4 when this package was written.

In the end we agreed that what we needed was to have some time together to talk and implement the changes needed to bring recipe-based julia plotting into the new eh century. So we decided to have a small meeting, half jokingly called #vizcon for 2-3 days of hackathon on plotting. We realized that @simon @anthony-wang @pfitzseb @ChristianKurz @jheinen @oschulz Marco and @daschw all live in Germany, and me, @Evizero and @piever all live not too far away, so we decided to try to find a date in September to meet, possibly in Berlin!

Oh, a few things I forgot to mention. @oschulz talked about exciting new developments happening in GR itself, it should be up to @jheinen to announce it, but it was things like adaptive zooming. We also had a discussion of how much of the pipeline of e.g. a histogram should be done by Plots vs passed to the backend, some thought Plots should be a very thin layer, other that the backend should essentially just be used for drawing. This conversation is continuously ongoing! Finally we discussed possibilities for using the backends’ ability to update existing Plots, and save the recalculation of Plots every time they are updated.

So, ephemeral discussion on Slack at #vizcon - I suggest this issue can be used to identify things we'd like to discuss and work on.

[mkborregaard]

OK, so if we're going to get the most out of our work together in October I think we should identify some key areas to discuss and some to work together on. There are really three projects there: Plots, Makie and GR. Of course everybody can just do what they feel like, but it would be even better to try to get some synergy.

I think the first thing to agree on (but which we can probably only agree on there) is the future of JuliaPlots.
Some questions:

1. Will Plots be deprecated in favour of Makie? If so, when will we do it? Is Makie already ready to take over, or should we rather iron out some of the crinkles in Plots and let Makie evolve another year or so? I think Makie is very neatly designed and avoids most of Plots' problems, but it's key to keep in mind that Plots is currently the widest used plotting library in Julia, and all of Julia as a project really needs it (or something equally broadly usable)!

2. So we have two broad possibilities, spend our energy really lifting Plots a step, or alternatively porting functionality to Makie. They are not mutually exclusive, but there is a prioritization.

3. What about GR? Could/should it work as a backend for Makie? Another alternative, if we move away from Plots, is to keep it as an interface to GR. That combination just works so well, and not having to maintain other backends would take away a lot of trouble (but would also disappoint some). GR is so mature and there are so many interesting things happening that I have no doubt it will be a key plotting library in Julia for the forseeable future.

If we lean towards lifting Plots, I would spend some time boiling and categorizing the current 324 issues (which is WAY to big a number, slightly overwhelming, though many are very small details) down into a set of say 5 key priority enhancements + 5 important bugs that we could talk about the prioritization of. And then we need to close or at least mark most of the petty issues so that we get down to a manageable number.

If we lean towards porting functionality to Makie @SimonDanisch is the man of course, but one thing I'd suggest doing was to look at the open and closed feature requests (+PRs?) on Plots over the last 2-3 years and map how many of these 1) are already taken care of in Makie or not relevant, 2) could potentially be ported using the same fix as in Plots or 3) would have to be reimplemented from scratch. Two other obvious suggestions would be to work on 1) a theming library for Makie (as the current plotting defaults are quite different from what many people are used to) and 2) porting existing Plots recipes to AbstractPlotting recipes.

Feedback please! :-)

[SimonDanisch]

Of course I'd love to work mostly on Makie ;) Of course, I would find it understandable if we decide against that.
I will try to solve as many issues as I can until vizcon! Maybe we can depend on how far I get, when choosing whether to work mainly on Plots or Makie!

Regardless of that, I'd love to work with @jheinen on a GR Makie backend... I think that could be totally feasible in the time span we have! :)

[piever]

I have had only a limited experience with Makie (mainly when trying to adapt StatPlots recipes to Makie in StatMakie). Here is what I feel.

Overall, I think I am in favor of moving to Makie, provided that we can do the following:

1. Clarify the recipe system: specifically making a recipe that converts the arguments to the plot and sets some default attributes should be super easy to do (while preserving the reactive nature of Makie: i.e. the plot updates automatically if the underlying data changes)
2. We add proper support to "grouping data" and layout: so far I think these are the main missing bits in terms of functionality.
3. Proper decoupling from GLVisualize and Makie. In particular I think we need at least a "web backend" for interactive plots in the jupyter notebook or Juno (like PlotlyJS in Plots) as well as interactive visualizations in Interact web apps, which I think is already in the works at WebVisualize.jl, and a GR backend to save publication quality vector formats (and potentially to reduce precompilation overhead, as GR is statically compiled)

Concerning 1., I think we are almost there, it's just a matter of adjusting the exact signatures that one needs to overload to define recipes. Concerning 2., I don't understand Makie's internals well enough to know how feasible that is.

Point 3. is crucial in my view, but it seems that there's already a fair bit of separation in that AbstractPlotting does not depend on GLVisualize things and incorporates most of Makie functionality. I think it is important because, IIUC, none of the Plots developers has that much experience with GLVisualize, so we could mainly help in the AbstractPlotting package.

The main reason I'm in favor of transitioning is that it seems to me that Makie requires way less work to be maintained. Given that in Plots all backends interact with Plots at different levels (some things are done natively in the backends, others as Plots recipes) and the pipeline is very complex, this creates a large surface area for bugs that it took a huge effort to slowly iron out. Some backend code are very complicated (for example for the GR backend) and sometimes implementing a new feature requires implementing it for all backends separately.
My understanding is that in Makie the interaction with the backend is extremely low level (basically at the level of the OpenGL shader) so the backend implementations are reasonably small. If truly @SimonDanisch and @jheinen can write a GR backend in two days, then definitely there is a good separation of backend and core library.

So I think we can bet on Makie during VizCon and try to address the points above:

• Finalize recipe system (and how it works with interactivity). I would love to discuss the implementation of interactivity (and potentially see if we can port Makie to use the same framework as Interact, which would allow for really nice interactive GUIs)
• Figure out grouping and layout
• Clarify the backend interface (which could be done in a very hands on way by doing the GR backend)

Finally, I agree with @mkborregaard that going over past issues is probably a good way to figure out how many of Plots problems are solved by Makie and how many are not, especially since there was heroic work (by @daschw and many others) to fix so many bugs that we should really see how many of those apply to get a feel fo the level of polish of Makie at the moment. We should also see in how many cases the fix (or the test) can be ported: it's really important that we try to preserve all the work that went there.

[mkborregaard]

I've been considering what is Plots' winning suite. It has three things it does particularly well, and the question is which ones are the most important - this is key to understanding whether Makie can replace Plots.

1. Interface to several other plotting libraries. This seems to be what most people think as Plots's strong suite, though I don't personally care. I only use GR.
2. RecipesBase. To me the most brilliant and important element.
3. Very terse syntax - columns-are-series can lead to some seriously terse plot commands. I ❤️ that.

You could add a (4.) for the great integration with very fast and pretty GR.

[oschulz]

Clarify the recipe system: specifically making a recipe that converts the arguments to the plot and sets some default attributes should be super easy to do

I'm all for easy-to-write recipes. Ideally, it should be very easy to turn a complex plot into a recipe, since this is the typical user workflow (you make a plot, then you realize you'll need this kind of plot often). So a recipe API that's very close to the plotting API would be great.

[SimonDanisch]

So a recipe API that's very close to the plotting API would be great.

In Makie that's almost 100% the case!

[piever]

1. is a bit double edged: it seems like most devs actually mainly use GR and as a consequence the GR backend is much more maintained than the others. I was cycling among backends when no backend was reliable :) Now I'm also only using GR. In Add GR to REQUIRE #1380 I got the impression that many people actually were happy to have one thing that works reliably.

To me a good set of backends for Makie would be:

• GLVisualize (OpenGL based) for normal interactivity
• WebVisualize (D3js, threejs based) for interactivity in the browser
• Cario /GR for vector graphics

OTOH, I think we'll lose the LaTeX export. I'm not sure how many people are using Plots with PGFPlots, but I really don't see how that'd fit with the Makie setup.

2. In terms of recipes see for example the QQPlot. I think it is a beautiful solution: one overloads the plot! function and that's it:The only shortcoming is that at the moment it comes a bit in the way of interactivity. Meaning, normally you can pass a "Reactive Signal" of data to the plot and the plot updates automatically whereas these recipes break that (so a "Reactive Signal of QQPair" updating would not work). Once this is fixed I think we have everything we need.

A separate discussion is that AbstractPlotting is a bit of a heavy dependency. However, since all plotting commands will be using Makie, MyPackage, I imagine one could use Require to avoid the dependency.

3. This is an interesting point, though I'm personally in favor of tuples of arrays, for two main reasons: it is closer to how we think of tabular data and works better if the arrays have different types / lengths. I'd definitely be curious to see how one can implement that in Makie. It would also be neat to support NamedTuples of arrays and use the keys for the legend now that we have NamedTuple support in base

[mkborregaard]

Good points. Yes the nice thing about tuples for the positional args would be that it would allow input of different lengths. But for me the key point is as much in the attributes: e.g layout = 2, color = :red, ls = [:dash :dot] which mixes shared and individual attributes of things in subplots intuitively and tersely. But a single attribute might also take a tuple, so that breaks down tuple-elements-are-series.

[mkborregaard]

Oh yeah the QQPlot Makie recipe is nice.

[mkborregaard]

My take is that people interested in tex are gradually migrating to PGFPlotsX

[SimonDanisch]

For the magic args: I plan to implement such shenanigans at a single, isolated place, like a preproccess_kwarg function ;)

For PGFPlotsX, as far as I understand, Gadfly has a native julia implementation for it, which we might be able to add as a backend ;)

[mkborregaard]

That makes sense, though I meant the ability to specify args for several series concisely. I've never used the magic args

[oschulz]

Regarding Makie and recipes, I also feel that AbstractPlotting.jl is much to heavy-weight at the moment, compared to RecipesBase. Is there a way to make it more lightweight, so that people be happy to add it to the dependencies of their package to enable plotting?

[oschulz]

Proper decoupling from GLVisualize and Makie

I guess Makie would become a backend for Makie-style recipes and plotting?

[oschulz]

Long-term, ROOTJS could also be a backend candidate (just the pure ROOTJS, not the full C++ CERN ROOT). ROOT plotting is a bit opinionated, so it might not be trivial to implement, but it is very fast and powerful: https://root.cern.ch/js/ (click on the thumbnails, then interact with the plots).

[Evizero]

I just wanted to say that I agree with pretty much all that was said here and that I look forward to lend a helping hand. As most of you probably know I am not actively involved in the plotting ecosystem anymore (aside from minimal UnicodePlots maintainance). So my main plan for vizcon is to donate my programming time and skills, such as they are, to whatever avenue(s) you decide to pursue.

[jheinen]

I'm open for any topic - I just want to explain my expectations and possible contributions for our meeting:

• I'd like to discuss the animation features in Plots.jl and compare it with those in GR. I have the impression, that GR's video stuff could be enriching for Plots (and probably for Makie, too). The current Plots solution requires ffmpeg and other console programs to be installed, which might cause problems due to changes in the command line options, etc. In GR we use an ffmpeg API which is part of the GR run-time. In GR master we can now produce video output in different formats (.mp4, .webm. .mov) on the fly, without storing intermediate PNG images.
• I hope to add the adaptive zooming code and publish a new GR release next week. Right now, there's only a stand-alone Qt application which demonstrates the new capabilities - if time permits I'll integrate those functions into GR.
• I'd question whether it's possible to write a GR backend for Makie in two days as I don't know enough about Makie.jl. But it's probably a good idea to exchange information and see what's possible.

[SimonDanisch]

I'd question whether it's possible to write a GR backend for Makie in two days as I don't know

Yeah it's a bit ambitious considering unknown complications... But in theory you just need to tell Makie how GR draws different primitives via some overloads.
So you basically just need to overload lines, particles, text, surfaces and meshes - which could be done fairly quickly if it's easy to do in GR!

without storing intermediate PNG images.

Makie streams the image on the fly to ffmpeg and is able to export them as mp4, webm, mov and gif! So I guess the biggest added benefit would be, that GR has a solid (it has right?) binary distribution going? FFMPEG is still not as easy to require from a julia package, so would need to be installed more or less manually.

[daschw]

Oh no, I also did not answer yet ...
I don't have to add much either. Basically I agree with everything that has been stated. I can maybe add my personal opinion. I feel that it gets harder and harder to maintain Plots due to the reasons you already mentioned, but also because I don't find that much time lately for OSS contributions. For this reason I also have to admit, that I have not yet managed to test Makie thoroughly or going through the code and really understanding the design. I think that Makie looks awesome judging from the examples Simon posts now and then on slack. Hence, I would really love to dig into it a little more.
However, currently I rather feel like I'm not spending enough time on maintaining Plots and, whenever I start up Julia in my free time, I feel obliged or responsible to work on that, rather than playing around with Makie.
So, I'm in favour of focusing on Makie at vizcon. This means that I will have to get to know it better in the next days. Nevertheless, I'm a little sceptical regarding the deprecation of Plots. How will this look exactly? What's the time frame for this? New issues pop up constantly and I feel that the Plots userbase is still very big or even growing.
So, I'm undecided. I'd like to work on Makie at vizcon and trying to get closer to feature parity with Plots, but I can't yet really imagine a deprecation of Plots - that's of course also due to my limited knowledge of how such things work.

[mkborregaard]

Hi, yeah, a deprecation of Plots now would be premature IMHO. When we do choose to do it, what I imagine we would do would be to put Plots into feature freeze: identify the bug issues that are necessary to fix for stable use of Plots, and fix them, but mark feature requests with wontfix and downprioritize minor bugs (but fix them eventually).
This would enable Plots to be a stable plotting solution for a long period going forward, while freeing up developer resources to make Makie able to do all the awesome things Plots does now and allow it to naturally grow into the viable option for Plots-loving julia users.

[SimonDanisch]

@mkborregaard that's exactly how I would imagine it to be done ;) That will give us a lot of room for figuring out where to go!

[daschw]

Sounds great!

[oschulz]

Also, lot's of people have written Plots recipes - so if Plots were to be deprecated, it either needs a very long transition period, or (my favourite solution) we have to offer backward compatibility to RecipesBase.

[mkborregaard]

I've made a short write-up of Vizcon - the idea is to post it on discourse, but nice to get some feedback and additions first. I also intend to post this as the report back to JuliaComputing about our achievements. Any comments, edits, additions welcome.

Vizcon 29th-30th October, Berlin

We just had "Vizcon", an open meeting for anyone interested in plotting and visualization with Julia. The idea came up at the Juliacon in London, and an open invitation was announced on the Slack #plotting channel, with follow-up planning at the Slack #vizcon channel. Present at the meeting were Simon Danisch (@SimonDanisch), Pietro Vertechi (@piever), Christof Stocker (@Evizero), Daniel Schwabeneder (@daschw), Moritz Schauer (@mschauer), Josef Heinen (@jheinen), Oliver Schulz (@oschulz), Sebastian Pfitzner (@pfitzseb), Diego Javier Zea (@diegozea) and myself (@mkborregaard). Most of us are associated with the JuliaPlots org, so we decided that the purpose of the meeting should be 1) discuss the future of plotting with Julia, 2) stabilize Plots for Julia 1.0, and 3) kick-start contributions to Makie.

Overall the meeting was a great success, with many important discussions and also quite a lot of actual work done. Here's an overview of what happened.

Presentations:

One presentation and discussion involved the future of plotting in Julia. Plots is currently one of the most popular plotting packages in Julia, and as such very important for the viability of Julia as an all-in-one programming language solution for many users. In particular the underlying recipe system in the stdlib-like RecipesBase has been very successful, with just short of 400 reverse dependencies. It's also quite difficult to maintain, because of the relationship to backend packages with very different interfaces. Over the last two years, we've reviewed merged on average 5 pull requests per week, and closed like 150 issues per year. But, issues are opened on Plots at roughly twice that rate, with no sign of diminishing returns, and there is currently >300 open issues on Plots. A presentation of this, and categorized tables of issues and pull requests, can be found here , here and here For long-term stability we thus need either more contributors, or, to find a less maintenance-heavy plotting solution.

At the Vizcon there was great enthusiasm about Makie as a solution for the future, which has a nice clear codebase and is really powerful in terms of interactivity and scope. It also has a system for recipes already and backends in the works. It is very possible that it can take the role of Plots in the future, but development is still open. Many of us present expressed an intent to spend some of our coding time contributing to Makie in the present and future, and that has already begun. Simon gave a guided tour of the code and design of Makie + AbstractPlotting.

Josef gave a presentation of GR, which is a much bigger framework than just a backend to Plots like many Julia users know it, acting as both backend and frontend for a major visualization pipeline. The presentation can be found here.

Discussions

The biggest discussions we had were:

Precompilation and time-to-first-plot

The biggest weakness of Plots, and suffers from the issue that it's hard to tell why this is so slow. Makie is currently even slower. The solution appears to be to get static package compilation to work. GR, which wraps a binary, is very fast from using GR to first plot. We did some work to get this faster, i.e. #1825 and JuliaPlots/PlotUtils.jl#52 .

A new recipe system

The heart and soul of Julia is the primacy of custom types. We found that plotting in julia needs some plotting-package independent way of mapping custom types to a visualization in user packages, while only depending on a small stdlib. RecipesBase goes some of the way for that, but is very tightly linked to Plots and Plots syntax. We decided we need something that is (practically) dependency-free, uses the facilities of julia 1.0 (like named tuples), does not depend on macros, and which has an intuitive syntax that is close to how we tend to use plotting libraries. It should be general enough to be a stdlib usable by all julia packages - using the syntax would have to be done by the plotting package itself or some small glue package, not in a big metapackage like Plots does. We started working on the syntax, and Oliver now has an almost-working prototype: https://github.com/oschulz/plottingrecipes.jl/tree/dev and a prototype glue package for GR https://github.com/oschulz/PlottingGR.jl/tree/dev .

We also agreed that Makie needs a recipe system that is much more lightweight than the one currently offered by AbstractPlotting. This could be such a general recipe system but there are things to iron out, like Makie relies a lot on Observables at the very core but other plotting packages don't. As a response to this, Simon implemented a change to the recipe system so the user will not have to worry about recipes: JuliaPlots/AbstractPlotting.jl#35 .

Declarative grammar / grouping

Many modern plotting packages like ggplot2 and vegalite have a declarative grammar, where you specify which plot elements should be control by which data, and then sensible plots are built automatically. An old idea is that both standard and declarative plotting syntaxes can be united (the library can be multi-paradigm) by expanding the grouping behaviour of Plots. We talked about various ways this could be done, and Pietro now has a working prototype of this for Makie: JuliaPlots/StatsMakie.jl#7

Makie backends

Josef and Simon managed to developing a GR backend for Makie, can now be found here: https://github.com/JuliaPlots/GRMakie.jl .
A UnicodePlots backend is also planned.

Other work

Here is a list of pull requests made during the vizcon that have not been covered in the above:
Deprecate glvisualize backend
#1818

Transfer portfoliocomposition recipe
#1819

Reorganize PlotRecipes
JuliaPlots/GraphRecipes.jl#50

Implement frames axis argument
JuliaPlots/AbstractPlotting.jl#31

Allow recursive merging of plot Attributes
JuliaPlots/AbstractPlotting.jl#32

Refactor AbstractPlotting backend display
MakieOrg/Makie.jl#200 and JuliaPlots/AbstractPlotting.jl#33

Add new example reference image
JuliaPlots/PlotReferenceImages.jl#33

Initial version of GR Makie backend
MakieOrg/Makie.jl#201

New implementation of density, histogram, Table compatibility and lifting
https://github.com/JuliaPlots/StatsMakie.jl/pull/10/files

Remove concept of atomics
MakieOrg/Makie.jl#203

Add linestyle for axes
JuliaPlots/AbstractPlotting.jl#36

Groupedbar recipe
JuliaPlots/StatsMakie.jl#12

Allow heatmap of nx1 or 1xn matrices
#1815

Fix docs build
JuliaPlots/PlotDocs.jl#112

Documenter compatibility update
JuliaPlots/PlotDocs.jl#115

Update reference images for testing
JuliaPlots/PlotReferenceImages.jl#34

Allow customizable RenderMode for txt
JuliaTesting/ReferenceTests.jl#19

Implement cycling color palette for series (/plots)
JuliaPlots/AbstractPlotting.jl#37

Testing UnicodePlots seriously
JuliaPlots/UnicodePlots.jl#85

Allow to use arguments in default theme
JuliaPlots/AbstractPlotting.jl#40

Other developments

• Apart from these things we also worked on various projects that are not covered by the PR's above.
• Better Makie theming
• Work on getting the PlotlyJS Plots backend 1.0 compliant
• Refactor PlotRecipes to focus on graphs only (GraphRecipes) while moving other recipes elsewhere
• Remove the Plots/StatPlots dependency of PlotRecipes and get it 1.0 compliant.

Overall a very productive and fruitful meeting. The NextJournal company in Berlin graciously allowed us to use their meeting and office facilities (and excellent coffee) for the meeting. We're also very grateful to Julia Computing's community funding for covering meals during the two meeting days.

[Evizero]

very nice and thorough. You'll probably want to escape the @githubuser handles, since I suspect Discourse will try to interpret them as discourse handles, which may or may not be the same

[mkborregaard]

Great - the handles are the same for everyone but Simon I think :-)

[mkborregaard]

Also, I need to update the two links where I say "here" - @jheinen would yuo be willing to link to your presentation as well?

[jheinen]

Also, I need to update the two links where I say "here" - @jheinen would yuo be willing to link to your presentation as well?

Done!

[daschw]

Great write-up, @mkborregaard!

[oschulz]

Very nicely written, @mkborregaard !

Oliver now has an almost-working prototype:

Uh, better call it a "rough outline of a prototype, not functional yet" ...

Could you also mention that package load times and time-to-first plot is a concern, and that we're looking for ways to improve them (help welcome)?

[Evizero]

I added this under Makie backends:

A UnicodePlots backend is also planned.

[piever]

Very nice write up! It may be nice to post a link to this PR where Simon is sketching a prototype of a recipe implementation where the user doesn't need to worry about Observables.

[SimonDanisch]

Awesome, thanks a lot @mkborregaard :)
@Evizero, could it be possible to quickly make something like MakieCairo.jl for the UnicodeBackend?

[Evizero]

while we already saw that it will be feasible to do, I still have to think about the right places to hook in. Probably will depend on StatsMakie.jl directly for example (to highjack histograms). Also I am changing some of the internals of UnicodePlots right now.

Basically its a yes to your question except the "quickly" part (which is a sound no), as I am doing things step by step and it will take me a week or two to get to the makie backend.

[pfitzseb]

Looks (and reads) great, @mkborregaard. Thanks for the detailed write-up!