New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PlotSpec: Add object to hold backend-independent input specification #390

Open
ma-laforge opened this Issue Jul 16, 2016 · 14 comments

Comments

Projects
None yet
5 participants
@ma-laforge
Contributor

ma-laforge commented Jul 16, 2016

EDIT: The original purpose of this issue has been achieved in #747 , but it is now an issue for the refactor in the title.

Many plot tools have a way to save/load the plots to/from a file.

...But this solution ties the file type to a plot backend.

Why not have Plots.jl provide a virtual plot backend called :hdf5 (or something similar) that allows you to "render" the plot (and the plot data) to an .hdf5 file?

The plot could then be re-loaded from said .hdf5 file at a later time - then plotted on whichever backend the user desires.

The advantage of the .hdf5 format is that people can still make out the file contents even if you don't have the appropriate reader.

I think users might like this feature.

MA

@lobingera

This comment has been minimized.

lobingera commented Jul 16, 2016

I fully support. Some time ago (~ 2years) there was a another discussion about plotting tools on julia-users and on my wish list were:

  • externalise internal data -> have the plot as datastructure
  • load/save system independent.
@ma-laforge

This comment has been minimized.

Contributor

ma-laforge commented Jul 16, 2016

Hi @lobingera,

plot as a datastructure:

Personally, I like this idea. This works well for my own needs to generate static plot objects (no animations - or anything fancy). So that's what I did for my own Plots.jl-like module (before I learned of Plots.jl):

https://github.com/ma-laforge/EasyPlot.jl

To avoid unnecessarily loading the HDF5 module, I moved the .hdf5 writer to a module called EasyData.jl

I currently have support for a few "backends" myself under https://github.com/ma-laforge/:

  • EasyPlotPlots.jl (Leverages Plots.jl)
  • EasyPlotInspect.jl
  • EasyPlotMPL.jl
  • EasyPlotGrace.jl
  • EasyPlotQwt.jl (From Pierre Raybaut).

Having said that
Although I personally prefer representing plots as a datastructure, I am not convinced this will necessarily work well for some of the more "advanced plot features" supported by Plots.jl.

Many of these advanced features seem to work well with the more command-oriented (as opposed to data-oriented) structure used by most plotting packages (including Plots.jl).

Could these features be elegantly tackled using a data-oriented approach? Would people actually prefer the data-oriented approach once they got used to it?

--> I do not really know - because I don't really deal with those particular problems at this time. That's why I am not trying to push my own solution onto people.

I think I would rather see Plots.jl support a generic load/save system in an environment that most users are likely to accept.

Maybe someone else out there has more experience on this subject...

@tbreloff

This comment has been minimized.

Member

tbreloff commented Jul 16, 2016

To be clear, the return value of a call to 'plot' IS a data structure. It's
a 'Plots.Plot'. It's mainly just a collection of nested attributes. I won't
be adding a dependency on HDF5, but that doesn't stop you from exposing
that functionality somewhere else.

On Saturday, July 16, 2016, ma-laforge notifications@github.com wrote:

Hi @lobingera https://github.com/lobingera,

plot as a datastructure:

Personally, I like this idea. This works well for my own needs to generate
static plot objects (no animations - or anything fancy). So that's what I
did for my own Plots.jl-like module (before I learned of Plots.jl):

https://github.com/ma-laforge/EasyPlot.jl

To avoid unnecessarily loading the HDF5 module, I moved the .hdf5 writer
to a module called EasyData.jl

I currently have support for a few "backends" myself under
https://github.com/ma-laforge/:

  • EasyPlotPlots.jl (Leverages Plots.jl)
  • EasyPlotInspect.jl
  • EasyPlotMPL.jl
  • EasyPlotGrace.jl
  • EasyPlotQwt.jl (From Pierre Raybaut).

Having said that
Although I personally prefer representing plots as a datastructure, I am
not convinced this will necessarily work well for some of the more
"advanced plot features" supported by Plots.jl.

Many of these advanced features seem to work well with the more
command-oriented (as opposed to data-oriented) structure used by most
plotting packages (including Plots.jl).

Could these features be elegantly tackled using a data-oriented approach?
Would people actually prefer the data-oriented approach once they got used
to it?

--> I do not really know - because I don't really deal with those
particular problems at this time. That's why I am not trying to push my own
solution onto people.

I think I would rather see Plots.jl support a generic load/save system in
an environment that most users are likely to accept.

Maybe someone else out there has more experience on this subject...


You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#390 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AA492p66J_OCTcDmQcHs1lz6uDOXkHzsks5qWQvYgaJpZM4JN54l
.

@ma-laforge

This comment has been minimized.

Contributor

ma-laforge commented Jul 16, 2016

@tbreloff:

To be clear, the return value of a call to 'plot' IS a data structure.

Yes, I realized this might hit a string. Sorry: this statement is more about the way we use the tool than the implementation being constructed from datastructures. I don't quite know how to explain myself, but I wil try:

If I understand nomenclature correctly, Plots.jl implementation seems to conform to the MVC (Model-View-Controller) structure.

In this paradigm, Plots.jl defines a consistent >>Interface<< to the user - as opposed to a generic object. Just to be clear: I'm not saying this is bad in any way. In fact, it has many advantages... I think it is probably a good system for interactive plots like animations.

Other implications
As far as I could tell: you first need to "bind" Plots.jl to a backend before you can start plotting (ex: by calling Plots.backend(:pyplot)).

That means that unlike the solution from EasyPlot.jl, you cannot just build a plot, and "display" it on multiple backends:

using EasyPlot
myplotobj = EasyPlot.new(title="Sample plot") #Type: EasyPlot.Plot
#>>Build plot object here<<
#NOTE: Did not need to load up a backend to fully describe plot

#Now render the plot with "Grace/xmgrace" backend:
using EasyPlotGrace
displayobj = EasyPlotGrace.PlotDisplay() #<: Multimedia.Display
display(displayobj, myplotobj)

#And now render the plot with "Plots.jl/PyPlot+GR" backends:
using EasyPlotPlots
displayobj1 = EasyPlotPlots.PlotDisplay(:pyplot)
displayobj2 = EasyPlotPlots.PlotDisplay(:gr)
display(displayobj1, myplotobj)
display(displayobj2, myplotobj)

#... Could render to other backends as well...

I guess this is a small distinction, but I think it potentially has important impact on the use model.

@tbreloff

This comment has been minimized.

Member

tbreloff commented Jul 16, 2016

I was about to say that I agree with you, except that the underlying object
will differ by backend. One backend may support a histogram natively, while
another doesn't, so the underlying data and attributes will be different. I
understand the principle, but I don't think it's valuable enough to
redesign anything.

With that said, I have been wanting to store user inputs separate from a
fully processed Plot object, and it's likely that your concept would be
appropriate for that. So you'd serialize a PlotSpec, not a Plot.

(I don't know when/if I'll tackle this)

On Saturday, July 16, 2016, ma-laforge notifications@github.com wrote:

@tbreloff https://github.com/tbreloff:

To be clear, the return value of a call to 'plot' IS a data structure.

Yes, I realized this might hit a string. Sorry: this statement is more
about the way we use the tool than the implementation being constructed
from datastructures. I don't quite know how to explain myself, but I wil
try:

If I understand nomenclature correctly, Plots.jl implementation seems to
conform to the MVC (Model-View-Controller) structure.

In this paradigm, Plots.jl defines a consistent >>Interface<< to the user

  • as opposed to a generic object. Just to be clear: I'm not saying this is
    bad in any way. In fact, it has many advantages... I think it is probably a
    good system for interactive plots like animations.

Other implications
As far as I could tell: you first need to "bind" Plots.jl to a backend
before you can start plotting (ex: by calling Plots.backend(:pyplot)).

That means that unlike the solution from EasyPlot.jl, you cannot just
build a plot, and "display" it on multiple backends:

using EasyPlot
myplotobj = EasyPlot.new(title="Sample plot") #Type: EasyPlot.Plot
#>>Build plot object here<<
#NOTE: Did not need to load up a backend to fully describe plot

#Now render the plot with "Grace/xmgrace" backend:
using EasyPlotGrace
displayobj = EasyPlotGrace.PlotDisplay() #<: Multimedia.Display
display(displayobj, myplotobj)

#And now render the plot with "Plots.jl/PyPlot+GR" backends:
using EasyPlotPlots
displayobj1 = EasyPlotPlots.PlotDisplay(:pyplot)
displayobj2 = EasyPlotPlots.PlotDisplay(:gr)
display(displayobj1, myplotobj)
display(displayobj2, myplotobj)

#... Could render to other backends as well...

I guess this is a small distinction, but I think it potentially has
important impact on the use model.


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#390 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AA492nqyoEaexFdJHBHLdHj3h5YXQuQ7ks5qWS_4gaJpZM4JN54l
.

@ma-laforge

This comment has been minimized.

Contributor

ma-laforge commented Jul 16, 2016

Excellent: I think we are starting to close in on the same terminology.

Yes: I agree that different backends might require different underlying objects to account for the different supported features... Though it sounds like "reciepes" might be able to eliminate this need by providing a "failsafe" for non-supported features (here's hoping!).

If I understand your terminology correctly: My EasyPlot.jl does not really compete with Plots.jl's MVC model. It basically implements what you call a PlotSpec - and provides methods to render the PlotSpec to different backends.

Point of the RFC
I don't expect anything to get implemented right away. I have a solution that works for me for the moment. However, I would like to eventually fully migrate to Plots.jl (or something similar) in the future.

Basically: I would like it if my future plot system also included a backend-independent way to archive plots with its data. My experiments show that the HDF5 file format provides a good solution to saving a PlotSpec-type object.

I'm a little affraid of using an actual "serialization" routine though - because that type of solution does not usually work well for archival purposes. I find a more rigidly defined file format that is not tied to the exact Julia object implementation is probably more readable/future proof.

@tbreloff tbreloff changed the title from RFC: Save plots to HDF5 >>Including datasets<< to PlotSpec: Add object to hold backend-independent input specification Jul 17, 2016

@tbreloff

This comment has been minimized.

Member

tbreloff commented Jul 17, 2016

I renamed the issue to be in line with something I'd likely take action on... @ma-laforge hopefully this is everything you want it to be :)

@ma-laforge

This comment has been minimized.

Contributor

ma-laforge commented Jul 18, 2016

No problem. I am sure you will come up with something useful if/whenever you see people needing PlotSpec.

Though I don't yet feel comfortable contributing to Plots.jl yet (I don't yet use enough features to really grasp the architecture & direction)... I am liking the progression so far.

@tbreloff

This comment has been minimized.

Member

tbreloff commented Oct 11, 2016

I just want to say that I've been thinking about this off-and-on for a few months, and fully intend to refactor Plots at some point to make it happen. I just haven't done it yet. :)

@mkborregaard

This comment has been minimized.

Member

mkborregaard commented Nov 6, 2017

@ma-laforge was this issue closed by #747 ?

@ma-laforge

This comment has been minimized.

Contributor

ma-laforge commented Nov 7, 2017

@tbreloff : With that said, I have been wanting to store user inputs separate from a fully processed Plot object, and it's likely that your concept would be appropriate for that. So you'd serialize a PlotSpec, not a Plot.

If I am to understand what @tbreloff said, I would say that technically, the issue is not closed by #747. I would have implemented the idea of PlotSpec myself, but I am not really certain how to separate out the concept of a "user input" object (PlotSpec) from the fully processed Plot object.

However, if you are asking me if the original intent of this issue is satisfied, I would say yes. Though #747 is not an archival-quality solution (Basically an object dump that does not guarantee backwards compatibility), I believe it is an adequate solution for now.

I would go ahead and close this issue unless @tbreloff disagreed.

@mkborregaard

This comment has been minimized.

Member

mkborregaard commented Nov 7, 2017

I edited the first post to show that the original intent is satisfied, but that the ideas developed here live on. I'll leave it open, but we'll not take action on it right now as there is lots of talk about different ways of refactoring Plots.

@tbreloff

This comment has been minimized.

Member

tbreloff commented Nov 7, 2017

@SimonDanisch

This comment has been minimized.

Member

SimonDanisch commented Nov 7, 2017

Sounds very similar to what I've been doing in Makie, so this vision is pretty close to being fully realized already!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment