Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Graph Widget? Plot Widget? #264

Closed
ccoupe opened this issue Jul 26, 2016 · 85 comments
Closed

Graph Widget? Plot Widget? #264

ccoupe opened this issue Jul 26, 2016 · 85 comments
Assignees
Milestone

Comments

@ccoupe
Copy link
Contributor

@ccoupe ccoupe commented Jul 26, 2016

Does anyone need a challenge ( @passenger94 ?). Long ago I did some some time series analysis foolery in Java and wrote a "widget" that would graph multiple time series data, auto scale the content to the window size, zoom in/out and move the zoomed graph left or right.

Hard core data analysis folks would just call out to gnuplot but gnuplot is huge and not really something we want to include in Shoes. Still, some cross platform visualization capability might be useful. I imagine, without thinking much on my part, that It would be Shoes widget (canvas) and we'd use mostly Cairo to draw and C/Ruby to get values out of Ruby arrays. Could it be done with pure Shoes/Ruby? Probably. Performance? I don't know - the test data sets I have are 5K of observations and they drew surprisingly quick on a Windows 98 laptop in Java 1.02,

@ccoupe
Copy link
Contributor Author

@ccoupe ccoupe commented Jul 26, 2016

Wow! That old Java code still works and just as painful to setup. Screen shot of graphing ^SPX (S&P 500) and ^IRX.R (3 month US Treasury Bill rate). The question is how to do it in Shoes and not what the plot tells you.
graph-example

@ccoupe
Copy link
Contributor Author

@ccoupe ccoupe commented Jul 26, 2016

@passenger94
Copy link
Contributor

@passenger94 passenger94 commented Jul 26, 2016

foolery in Java

sounds like a pleonasm :-D

haha, thanks for challenge :-), will look at it, if only for curiosity, i was interested in this when looking at the profiler thing and ways to get visual feedback
lack of time lately, some opportunities maybe next month

@dredknight
Copy link
Contributor

@dredknight dredknight commented Jul 27, 2016

this looks really awesome!
I have to get together and finish that popup thing and I can hop over and check this too :)

@ccoupe
Copy link
Contributor Author

@ccoupe ccoupe commented Aug 5, 2016

Still thinking out loud about this. It should be easy enough to convert that java code into a shoes.widget in pure ruby. Looking at the java, the DataSeries class is actually the bigger problem because I know it's sordid secrets - it has no way to represent 'no data' or 'bad data' for 'this observation' and it assumes all DataSeries have the same date/time starting point and ending point and number of observations. In order to be part of Shoes, the drawing code has to be in lib/shoes/ and while there are reasons to have DataSeries be a Gem, there are good (Loose Shoes symlink) reasons to put it into lib/shoes/DataSeries/

I've uploaded some Stock Market data files (csv format) to here - the contents are obvious. There is a long ago, mostly useless reason they start with '^' but it's what I have.

I've created branch 'graph' which is where this experiment should live.

@ccoupe ccoupe added this to the 3.3.3 milestone Aug 5, 2016
ccoupe added a commit that referenced this issue Aug 5, 2016
* csv.rb is the test program. `ruby csv.rb` works - it's
  not Shoes yet.
* Do you wonder about huge hashes and arrays? Me too.
ccoupe added a commit that referenced this issue Aug 5, 2016
*  still not correct or pretty,
ccoupe added a commit that referenced this issue Aug 8, 2016
@ccoupe
Copy link
Contributor Author

@ccoupe ccoupe commented Aug 8, 2016

Pure Shoes/ruby - test of concept. It works. Of course it's missing all kinds of things.
pure_ruby_graph

Good enough to try sub classing Shoes.widget.

@passenger94
Copy link
Contributor

@passenger94 passenger94 commented Aug 8, 2016

i say YES !!!
👠 👠 👠 🔆

@ccoupe
Copy link
Contributor Author

@ccoupe ccoupe commented Aug 9, 2016

So it's now a Shoes::Widget and it's time to add the axes and labels. OOPS! Shoes doesn't have methods for drawing text at x,y. Time to move c/obj-c? Maybe Shoes should have an Art Shape (like oval, star, etc ) named Art_String ??? so art_string {x:, y:, font:, fontsize:,color:, length: , text:}. I don't know how hard that is to do but it has some appeal and might be useful for other folks for other applications. Probably easier to do than creating a new c/obj-c graph widget so I like that. It might be useful text on top a image (with transforms already available like rotate and skew). Opinions?

ccoupe added a commit that referenced this issue Aug 9, 2016
on #264, widgetize.
on #264, widgetize.
on #264, widgetize.
on #264, widgetize.
on #264, widgetize.
ccoupe added a commit that referenced this issue Aug 9, 2016
@ccoupe ccoupe mentioned this issue Aug 9, 2016
@ccoupe
Copy link
Contributor Author

@ccoupe ccoupe commented Aug 11, 2016

Considering that I'm just converting my java code, it would be conceptually easier to create a widget,
similar to svg which I understand a lot better than Shape manipulation code (#266). It would have better performance for large data sets but not nearly as flexible for future enhancements. It's possible to do both a native widget and a Shoes::Widget version.

I think I'll call the native widget 'plot' unless someone has a better name. API is pretty simplistic so far:
pl = plot width, height. {options} and pl.add {options} | dataseries and pl.redraw in case someone is appending real time data. The option names: and defaults to be determined later but :data => [array of floatables]`, :name => "string", :minv => float, :maxv => float are the minimum.

ccoupe added a commit that referenced this issue Aug 12, 2016
* compiles (loose linux)
* doesn't work - plot.c (shoes_plot_add) doesn't detect TimeSeries class
@ccoupe
Copy link
Contributor Author

@ccoupe ccoupe commented Aug 13, 2016

API alert - some options belong to plot or chart (title and caption, width and height) and some belong to the data (like name, display_name, values and many more); The plot/chart behaves differently depending on how many arrays it's supposed to display. So, it's a two step creation process in Shoes plot width, height, {options) and plot.add {many_hash_args}

ccoupe added a commit that referenced this issue Aug 15, 2016
* too domain specific for Shoes.
  and ca
ccoupe added a commit that referenced this issue Aug 15, 2016
* remove dataseries and Timeseries -- too domain specific to be
  part pf Shoes. Useful to get the API correct - plot (widget) vs
  dots on screen.
* Very sure I don't understand cairo transforsm.
  This just place keeper/backup commit
ccoupe added a commit that referenced this issue Aug 16, 2016
* SO MANY TODO
* doesn't draw the data.
* appears to get the cairo transform correct but who knows?
* could it be better? Hell Yes. I have no idea what I'm doing.
@ccoupe
Copy link
Contributor Author

@ccoupe ccoupe commented Aug 18, 2016

[Updated 2016-08-19]
better_plot

@ccoupe
Copy link
Contributor Author

@ccoupe ccoupe commented Aug 22, 2016

Getting better. So many things to do under the covers.
better-plot

The potential api keeps expanding with all the options the users might like ( turning of the grid lines for example)

ccoupe added a commit that referenced this issue Aug 22, 2016
ccoupe added a commit that referenced this issue Aug 22, 2016
ccoupe added a commit that referenced this issue Aug 22, 2016
* many things to do. Doesn't trigger refresh.
  does not draw left side y axis values
  probably does not handle float values
  may not not handle negative values.
ccoupe added a commit that referenced this issue Aug 23, 2016
@ccoupe
Copy link
Contributor Author

@ccoupe ccoupe commented Aug 25, 2016

The plot widget is almost good_enough for people to try it and find out what doesn't work for them. It does work on all platforms. I might think about including it in 3.3.2. Even if its a little half baked. Documenting it is going to be a challenge.
good-enough

and the script to create that:

# good-graph.rb
Shoes.app width: 620, height: 610 do
  @values1 = [24, 22, 10, 13, 20, 8, 22]
  @x_axis1 = ['a','b','c','d','e','f', 'g']
  @values2 = [200, 150, 75, 125, 75, 50, 125]
  stack do
    para "Plot Widget Demo"
    flow do 
      button "quit" do Shoes.quit end
    end
    widget_width = 600
    widget_height = 400
    @grf = plot widget_width, widget_height, title:"My Graph", caption: 
      "Look at that! Booyah!!" , font: "Helvetica", auto_grid: true
    @grf.add num_obs: @values1.size, values: @values1, xobs: @x_axis1,
       name: "foobar", minv: 6, maxv: 26 , long_name: "foobar Yy"
    button "add #2" do
      @grf.add num_obs: @values2.size, values: @values2,
        name: "bartab", minv: @values2.min, maxv: @values2.max
    end
  end
end
@ccoupe
Copy link
Contributor Author

@ccoupe ccoupe commented Oct 24, 2016

As we often do, we seem to be talking past each other until we figure it out. Remember that chart creation is a two step process - create the widget (chart type) and then add data/labels to it. For line, bar, and timeseries, you can add multiple collections of 'Thing X' to the chart. Thing X also contains important things like the color you want and the min/max used to scale that part of the chart.

I really don't care what the Shoes object is named, I think it needs to exist and it does exist because the C was getting unmanageable and that wouldn't get better for new chart types or future enhancements to Thing X. From Tests/plot/gr3.rb

    @grf2.add values: @values1, labels: @x_axis1,
       name: "Bar", min: 0, max:  30, desc: "foobar Yy", color: "crimson",
       points: true, strokewidth: 12
    cs2 = chart_series values: @values2, labels: @x_axis2,
       name: "Tab", min: 50, max: 230, desc: "BarTab", color: "green",
       points: true, strokewidth: 6
    @grf2.add cs2

Both @grf.add 'hash' and 'chart_series' work. User's preference. The 'hash' version just creates the chart_series for you behind the scenes. Which one is the simplest? Minor differences, IMO and we have both, and always will. . Which one can be sub classed and re-purposed for other duty should someone who is not newbie prefer? - hash args in a Ruby program they cant change by a running script or an object they can build on?

Currently Thing X is called 'chart_series' at the C and Ruby levels. In my wandering around the interwebs I haven't found a common name for what Thing X is. I also haven't seen many FOSS Windows or OSX programs that draws simple charts, simply. Niche? I do care about doing the best thing given the situation we find ourselves in. We all do. Actually I prefer data_series and shoes_plot_data_series but that feels presumptive and invites a name clash with some ruby gem somewhere.

@IanTrudel
Copy link
Collaborator

@IanTrudel IanTrudel commented Oct 25, 2016

@ccoupe thanks for explaining. Hash is simpler. ;)

By the way, does color: supports rgb() values? It should be noted that other Shoes methods support Shoes colornames, such as background, which does not use quotes as in Test/plot/gr3.rb. Consistency is important.

Hopefully you understand that I am not nitpicking. The chart feature is absolutely fantastic. I remember wanting to get some Java charting in the past but the only decent packages were available for sales. So it's refreshing to get it for Shoes.

@passenger94
Copy link
Contributor

@passenger94 passenger94 commented Oct 25, 2016

@BackOrder, yes i do agree, C functions should check for a ruby color (rgb or named) in the hash given to plot related methods (like texts, shapes, etc ... do for stroke, fill and those you mentioned, background, border ...).
The whole plot extravaganza is shiny enough now to spend time on killer "details" :-)

@ccoupe
Copy link
Contributor Author

@ccoupe ccoupe commented Oct 26, 2016

I welcome help and you are helping: Turns out the plot color can easily be handled as you suggest. Bonus points because I get to delete some ugly code. I wasn't fond of the string myself. Fixed.

Another annoyance is having to use strings for "dot" and "rect" - I'd prefer symbols :dot and :rect . Opinions?

The radar/spider chart ineed a lot help if someone has some extra time.

ccoupe added a commit that referenced this issue Oct 26, 2016
*  which includes the built in Colors, and rgb() calls.
*  adds some more flesh to chart_series methods.
@IanTrudel
Copy link
Collaborator

@IanTrudel IanTrudel commented Oct 26, 2016

Great!

Another annoyance is having to use strings for "dot" and "rect" - I'd prefer symbols :dot and :rect . Opinions?

Can't we have similar thing as Shoes colours? That would end up with dot and rect directly (without quotes or colon). Otherwise symbols would be fine as far as Ruby goes but it's always a bit strange to use with named parameters, such as points: :dot (a lot of colons if you notice).

The radar/spider chart ineed a lot help if someone has some extra time.

What's the TODO list on this one?

@ccoupe
Copy link
Contributor Author

@ccoupe ccoupe commented Oct 26, 2016

For better or worse, all of the color names are real Shoes objects in the Shoes name space. While we could do something similar and create Shoes classes for dot, box, rect, and circle, I am not comfortable that - it could to break existing scripts that have those class names. At the C level, I'm not happy to pollute Ruby space with the :symbols either but since "Shoes is not Ruby it only looks like it", symbols seem a better choice at the moment. (assuming I do anything)

What's the TODO list on this one?

Huge. I don't know a sin() from a cos(), radians from degrees, and when to use one vs the other. The code in there is wrong (converted from pycha). One of the magazines I read (Car and Driver) presents their results in a spider graphs (they also get complaints about them - but that's not our problem to solve). Radial axis can run from small number near the center to larger number on the edge. Or larger numbers at the center and smaller at the edge - depending on how they define goodness. - a reversal of current min/max - The code in there might handle that, might not - I suspect not. Then there is issue of labels. One of the charts in your example way above this comment appears to use axis in 0->100,000 but the labels are 50, 70, 90. Another reason for the &blk the chart_series/hash/Thing_X, on add.

@IanTrudel
Copy link
Collaborator

@IanTrudel IanTrudel commented Oct 26, 2016

re: namespace

This is an important question. Creating additional Shoes objects would promote consistency and better syntax but your argument that it might break existing code is absolutely valid. It may not have big impact or may be expected from upgrading process but it's still important enough to consider before moving forward with such approach.

Ruby symbols would be an alternative and we could depreciate Shoes objects (such as colour names) via a warning in Shoes console to once again promote consistency. The depreciation would include support of colour name as symbols and we would phase out Shoes objects some future release.

This is something to think about because Shoes has been having increasingly new features since you took over the development. It comes to reason we need to have some kind of guideline.

re: math

Ruby Math (and most programming API) only support radians. One radian equals 1/2π. Degrees are something human enjoys and goes from 0 to 360 degrees for a full circle; exceeding values results in circling again, negative values for counterclockwise.

What you need to know is that 2π radian equals full circle (360 degrees). You use this reference to know how to split equally. You have X values on a radar chart? 2π / X.

Something along this line... not sure whether it is helpful to you...

@ccoupe
Copy link
Contributor Author

@ccoupe ccoupe commented Oct 27, 2016

For now, I'm going to leave points and a few others as strings args until I think harder about symbols and learn how to use them at the C level. I think I know and what I think is ok-maybe.

What you need to know is that 2π radian equals full circle (360 degrees). You use this reference to know how to split equally. You have X values on a radar chart? 2π / X.

That bit of knowledge helps. I 'll review the the pycha radar to C/cairo conversion with that extra info.

ccoupe added a commit that referenced this issue Oct 27, 2016
* see Test/plot/cstest.rb for odd things that can be done.
* just getting started on the manual.
@ccoupe
Copy link
Contributor Author

@ccoupe ccoupe commented Oct 27, 2016

One use case for chart_series (aka Thing_X): Imagine you want to monitor the temperature of a probe in a BBQ smoker - when its running between set points of 225F and 240F you draw a black line/datapoint on the graph. If it drops below 225F then draw a blue line and if rises above 240F then draw a red line. Imagine it spikes up to 280 and you need rescale the y axis too. You'll need getters and setters for all the hash arguments for chart_series (plot.add) but if you don't have a backing object then those would be getters and setters on the plot object and each of them would require an index or hash arg to pick which series to modify. Unclean in C and unclean in Shoes.

See Test/plots/cstest.rb - it appends (label, value) but it can also replace existing (label, value) - just change that 7 to a 3 for example. Also note in gr4.rb that it uses the return object of a plot.add (currently a chart_series).

@IanTrudel
Copy link
Collaborator

@IanTrudel IanTrudel commented Oct 27, 2016

So Shoes charting allow realtime data and other modifications?

@ccoupe
Copy link
Contributor Author

@ccoupe ccoupe commented Oct 27, 2016

So Shoes charting allow realtime data and other modifications?

One of my design goals. Fundamental. Not RT as process control people think but, every second or two.

@IanTrudel
Copy link
Collaborator

@IanTrudel IanTrudel commented Oct 27, 2016

I'm looking forward to it. :D

@ccoupe
Copy link
Contributor Author

@ccoupe ccoupe commented Oct 27, 2016

Already there in gr1.rb which adds random data points every 5 seconds.

ccoupe added a commit that referenced this issue Oct 27, 2016
* need to document more chart_series methods.
@ccoupe
Copy link
Contributor Author

@ccoupe ccoupe commented Oct 27, 2016

New beta available . Seems to mostly work Windows for graph tests but you need the graph test script which you only get from github sync. Perhaps my test scripts need to be in samples? expert-plot1.rb ...expert-plot6.rb ? Ick.

ccoupe added a commit that referenced this issue Oct 29, 2016
* add samples/good-plots.rb - display all types
  also demontrates that their is gtk refresh/widget redraw issue
  previously noted on Windows.
@ccoupe
Copy link
Contributor Author

@ccoupe ccoupe commented Oct 29, 2016

Who knew? Gruff is a ruby (rails gem) that draws all kinds of charts, including spider, using rmagick as the backend but at the Ruby level the code is pretty similar to what C/Cairo uses. Anyone attempting #266 charting package might want to start with this.

@IanTrudel
Copy link
Collaborator

@IanTrudel IanTrudel commented Oct 29, 2016

It would have been nice but Gruff doesn't seem maintained anymore. The last commit is 5 months old and issues are dating in years.

Furthermore, rmagick as a backend might not be suitable for realtime graphs.

@ccoupe
Copy link
Contributor Author

@ccoupe ccoupe commented Oct 29, 2016

Gruff is source code to copy and modify ourselves. It's not too old and it does work within it's limits and they have test data, scripts and test output. It's hard to install (even on ubuntu and probably very difficult for windows). I would never use it as a gem to include in Shoes.

The algorithms inside are the value to me (and the test's, so I know that it should look like). I doubt anyone will modify the Ruby code to use Shoes shape drawing code in place of rmagick , but they could (#266).

@ccoupe
Copy link
Contributor Author

@ccoupe ccoupe commented Oct 30, 2016

We have radial axis.
radar-1

@IanTrudel
Copy link
Collaborator

@IanTrudel IanTrudel commented Oct 30, 2016

Good. I'd like to recommend a 90 degrees rotation to match other radar graphs samples we shown here (e.g. LibreOffice). You will certainly notice the starting point on your graph is on right-middle quadrant. LibreOffice et co. would normally start on top-middle quadrant.

@ccoupe
Copy link
Contributor Author

@ccoupe ccoupe commented Oct 31, 2016

I was working from the simpler spider in gruff which for whatever reason draws it with that start position. What we really want is Gruff's 'net' chart, which is a bit more complex but easier to comprehend than the pycha code.

@ccoupe
Copy link
Contributor Author

@ccoupe ccoupe commented Nov 2, 2016

This issue is getting a little slow to load and it won't get faster. I've opened #288 for radar/spider issues.

@IanTrudel
Copy link
Collaborator

@IanTrudel IanTrudel commented Nov 15, 2016

I am not a huge fan of pie_percent on L019. How about just percent?

ccoupe added a commit that referenced this issue Nov 18, 2016
@ccoupe
Copy link
Contributor Author

@ccoupe ccoupe commented Jan 21, 2017

Closed in Shoes 3.3.2

@ccoupe ccoupe closed this Jan 21, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
4 participants