PlotSpec: Add object to hold backend-independent input specification

[ma-laforge]

EDIT: The original purpose of this issue has been achieved in #747 , but it is now an issue for the refactor in the title.

Many plot tools have a way to save/load the plots to/from a file.

...But this solution ties the file type to a plot backend.

Why not have Plots.jl provide a virtual plot backend called :hdf5 (or something similar) that allows you to "render" the plot (and the plot data) to an .hdf5 file?

The plot could then be re-loaded from said .hdf5 file at a later time - then plotted on whichever backend the user desires.

The advantage of the .hdf5 format is that people can still make out the file contents even if you don't have the appropriate reader.

I think users might like this feature.

MA

[lobingera]

I fully support. Some time ago (~ 2years) there was a another discussion about plotting tools on julia-users and on my wish list were:

• externalise internal data -> have the plot as datastructure
• load/save system independent.

[ma-laforge]

Hi @lobingera,

plot as a datastructure:

Personally, I like this idea. This works well for my own needs to generate static plot objects (no animations - or anything fancy). So that's what I did for my own Plots.jl-like module (before I learned of Plots.jl):

https://github.com/ma-laforge/EasyPlot.jl

To avoid unnecessarily loading the HDF5 module, I moved the .hdf5 writer to a module called EasyData.jl

I currently have support for a few "backends" myself under https://github.com/ma-laforge/:

• EasyPlotPlots.jl (Leverages Plots.jl)
• EasyPlotInspect.jl
• EasyPlotMPL.jl
• EasyPlotGrace.jl
• EasyPlotQwt.jl (From Pierre Raybaut).

Having said that
Although I personally prefer representing plots as a datastructure, I am not convinced this will necessarily work well for some of the more "advanced plot features" supported by Plots.jl.

Many of these advanced features seem to work well with the more command-oriented (as opposed to data-oriented) structure used by most plotting packages (including Plots.jl).

Could these features be elegantly tackled using a data-oriented approach? Would people actually prefer the data-oriented approach once they got used to it?

--> I do not really know - because I don't really deal with those particular problems at this time. That's why I am not trying to push my own solution onto people.

I think I would rather see Plots.jl support a generic load/save system in an environment that most users are likely to accept.

Maybe someone else out there has more experience on this subject...

[tbreloff]

To be clear, the return value of a call to 'plot' IS a data structure. It's
a 'Plots.Plot'. It's mainly just a collection of nested attributes. I won't
be adding a dependency on HDF5, but that doesn't stop you from exposing
that functionality somewhere else.

On Saturday, July 16, 2016, ma-laforge notifications@github.com wrote:

Hi @lobingera https://github.com/lobingera,

plot as a datastructure:

Personally, I like this idea. This works well for my own needs to generate
static plot objects (no animations - or anything fancy). So that's what I
did for my own Plots.jl-like module (before I learned of Plots.jl):

https://github.com/ma-laforge/EasyPlot.jl

To avoid unnecessarily loading the HDF5 module, I moved the .hdf5 writer
to a module called EasyData.jl

I currently have support for a few "backends" myself under
https://github.com/ma-laforge/:

• EasyPlotPlots.jl (Leverages Plots.jl)
• EasyPlotInspect.jl
• EasyPlotMPL.jl
• EasyPlotGrace.jl
• EasyPlotQwt.jl (From Pierre Raybaut).

Having said that
Although I personally prefer representing plots as a datastructure, I am
not convinced this will necessarily work well for some of the more
"advanced plot features" supported by Plots.jl.

Many of these advanced features seem to work well with the more
command-oriented (as opposed to data-oriented) structure used by most
plotting packages (including Plots.jl).

Could these features be elegantly tackled using a data-oriented approach?
Would people actually prefer the data-oriented approach once they got used
to it?

--> I do not really know - because I don't really deal with those
particular problems at this time. That's why I am not trying to push my own
solution onto people.

I think I would rather see Plots.jl support a generic load/save system in
an environment that most users are likely to accept.

Maybe someone else out there has more experience on this subject...

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#390 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AA492p66J_OCTcDmQcHs1lz6uDOXkHzsks5qWQvYgaJpZM4JN54l
.

[ma-laforge]

@tbreloff:

To be clear, the return value of a call to 'plot' IS a data structure.

Yes, I realized this might hit a string. Sorry: this statement is more about the way we use the tool than the implementation being constructed from datastructures. I don't quite know how to explain myself, but I wil try:

If I understand nomenclature correctly, Plots.jl implementation seems to conform to the MVC (Model-View-Controller) structure.

In this paradigm, Plots.jl defines a consistent >>Interface<< to the user - as opposed to a generic object. Just to be clear: I'm not saying this is bad in any way. In fact, it has many advantages... I think it is probably a good system for interactive plots like animations.

Other implications
As far as I could tell: you first need to "bind" Plots.jl to a backend before you can start plotting (ex: by calling Plots.backend(:pyplot)).

That means that unlike the solution from EasyPlot.jl, you cannot just build a plot, and "display" it on multiple backends:

using EasyPlot
myplotobj = EasyPlot.new(title="Sample plot") #Type: EasyPlot.Plot
#>>Build plot object here<<
#NOTE: Did not need to load up a backend to fully describe plot

#Now render the plot with "Grace/xmgrace" backend:
using EasyPlotGrace
displayobj = EasyPlotGrace.PlotDisplay() #<: Multimedia.Display
display(displayobj, myplotobj)

#And now render the plot with "Plots.jl/PyPlot+GR" backends:
using EasyPlotPlots
displayobj1 = EasyPlotPlots.PlotDisplay(:pyplot)
displayobj2 = EasyPlotPlots.PlotDisplay(:gr)
display(displayobj1, myplotobj)
display(displayobj2, myplotobj)

#... Could render to other backends as well...

I guess this is a small distinction, but I think it potentially has important impact on the use model.

[tbreloff]

I was about to say that I agree with you, except that the underlying object
will differ by backend. One backend may support a histogram natively, while
another doesn't, so the underlying data and attributes will be different. I
understand the principle, but I don't think it's valuable enough to
redesign anything.

With that said, I have been wanting to store user inputs separate from a
fully processed Plot object, and it's likely that your concept would be
appropriate for that. So you'd serialize a PlotSpec, not a Plot.

(I don't know when/if I'll tackle this)

On Saturday, July 16, 2016, ma-laforge notifications@github.com wrote:

@tbreloff https://github.com/tbreloff:

To be clear, the return value of a call to 'plot' IS a data structure.

Yes, I realized this might hit a string. Sorry: this statement is more
about the way we use the tool than the implementation being constructed
from datastructures. I don't quite know how to explain myself, but I wil
try:

If I understand nomenclature correctly, Plots.jl implementation seems to
conform to the MVC (Model-View-Controller) structure.

In this paradigm, Plots.jl defines a consistent >>Interface<< to the user

• as opposed to a generic object. Just to be clear: I'm not saying this is
  bad in any way. In fact, it has many advantages... I think it is probably a
  good system for interactive plots like animations.

Other implications
As far as I could tell: you first need to "bind" Plots.jl to a backend
before you can start plotting (ex: by calling Plots.backend(:pyplot)).

That means that unlike the solution from EasyPlot.jl, you cannot just
build a plot, and "display" it on multiple backends:

using EasyPlot
myplotobj = EasyPlot.new(title="Sample plot") #Type: EasyPlot.Plot
#>>Build plot object here<<
#NOTE: Did not need to load up a backend to fully describe plot

#Now render the plot with "Grace/xmgrace" backend:
using EasyPlotGrace
displayobj = EasyPlotGrace.PlotDisplay() #<: Multimedia.Display
display(displayobj, myplotobj)

#And now render the plot with "Plots.jl/PyPlot+GR" backends:
using EasyPlotPlots
displayobj1 = EasyPlotPlots.PlotDisplay(:pyplot)
displayobj2 = EasyPlotPlots.PlotDisplay(:gr)
display(displayobj1, myplotobj)
display(displayobj2, myplotobj)

#... Could render to other backends as well...

I guess this is a small distinction, but I think it potentially has
important impact on the use model.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#390 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AA492nqyoEaexFdJHBHLdHj3h5YXQuQ7ks5qWS_4gaJpZM4JN54l
.

[ma-laforge]

Excellent: I think we are starting to close in on the same terminology.

Yes: I agree that different backends might require different underlying objects to account for the different supported features... Though it sounds like "reciepes" might be able to eliminate this need by providing a "failsafe" for non-supported features (here's hoping!).

If I understand your terminology correctly: My EasyPlot.jl does not really compete with Plots.jl's MVC model. It basically implements what you call a PlotSpec - and provides methods to render the PlotSpec to different backends.

Point of the RFC
I don't expect anything to get implemented right away. I have a solution that works for me for the moment. However, I would like to eventually fully migrate to Plots.jl (or something similar) in the future.

Basically: I would like it if my future plot system also included a backend-independent way to archive plots with its data. My experiments show that the HDF5 file format provides a good solution to saving a PlotSpec-type object.

I'm a little affraid of using an actual "serialization" routine though - because that type of solution does not usually work well for archival purposes. I find a more rigidly defined file format that is not tied to the exact Julia object implementation is probably more readable/future proof.

[tbreloff]

I renamed the issue to be in line with something I'd likely take action on... @ma-laforge hopefully this is everything you want it to be :)

[ma-laforge]

No problem. I am sure you will come up with something useful if/whenever you see people needing PlotSpec.

Though I don't yet feel comfortable contributing to Plots.jl yet (I don't yet use enough features to really grasp the architecture & direction)... I am liking the progression so far.

[tbreloff]

I just want to say that I've been thinking about this off-and-on for a few months, and fully intend to refactor Plots at some point to make it happen. I just haven't done it yet. :)

[mkborregaard]

@ma-laforge was this issue closed by #747 ?

[ma-laforge]

@tbreloff : With that said, I have been wanting to store user inputs separate from a fully processed Plot object, and it's likely that your concept would be appropriate for that. So you'd serialize a PlotSpec, not a Plot.

If I am to understand what @tbreloff said, I would say that technically, the issue is not closed by #747. I would have implemented the idea of PlotSpec myself, but I am not really certain how to separate out the concept of a "user input" object (PlotSpec) from the fully processed Plot object.

However, if you are asking me if the original intent of this issue is satisfied, I would say yes. Though #747 is not an archival-quality solution (Basically an object dump that does not guarantee backwards compatibility), I believe it is an adequate solution for now.

I would go ahead and close this issue unless @tbreloff disagreed.

[mkborregaard]

I edited the first post to show that the original intent is satisfied, but that the ideas developed here live on. I'll leave it open, but we'll not take action on it right now as there is lots of talk about different ways of refactoring Plots.

[tbreloff]

My original vision of PlotSpec was that of an object that literally tracked what the user explicitly set. Then as as that was changed, either programmatically (adding data or series or legends, etc) or through user interactivity, Plots would notice the changes and lazily update a built Plot object using a dependency tree of attributes, also giving backends the ability to lazily redraw only those items that changed (if they can support that). This vision would also support analytic pipelines, for example updating a histogram as data is added to the underlying data vector. I don’t think this vision has even beed specced out, let alone finished. If someone wants to put this stuff into a new issue, awesome. Sorry I cannot be the one to work on it! Keep up the great work all! 👌

…

On Tue, Nov 7, 2017 at 2:20 AM Michael Krabbe Borregaard < ***@***.***> wrote: I edited the first post to show that the original intent is satisfied, but that the ideas developed here live on. I'll leave it open, but we'll not take action on it right now as there is lots of talk about different ways of refactoring Plots. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#390 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AA492ljFh1qUW8x-6W0cqVY-akug4h0Wks5s0ASugaJpZM4JN54l> .

[SimonDanisch]

Sounds very similar to what I've been doing in Makie, so this vision is pretty close to being fully realized already!