Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
python-based graphing tool #10
this tool will allow graphing lots of data while fixing some of the issues with the current gnuplot-based grapher, most notably the scale of the X axis.
eventually, this can be enhanced to perform linear regressions that will allow guessing when the battery will need to be replaced, but so far i have focused only in replacing the existing model.
please test with your dataset to see if it scales as well..
added a commit
this pull request
Sep 26, 2015
i am still analysing the performance, but basically, it looks like the date parsing takes 3 seconds in Python, then there is the overhead of looping which takes a full second just to iterate over all the timestamps. this would be hard to workaround - unless, of course, matplotlib can parse only the dates it shows in the label, something i am not sure of.
then drawing the PNG takes another 3 seconds, so that's another hard limit. parsing the CSV is ~2s, which about covers the 9s the script takes.. so i am not sure there's much more optimising i can realistically do here. 9s seems like a fair delay, considering this represents a few years of data being crunched, a worst-case scenario...
i was running my tests with:
for future reference, here are the two profiling runs i did, before and after the csv importer rewrite:
The details of the above breakdown:
referenced this pull request
Sep 27, 2015
as a complement, here's an example of the graph with 1000k entries with gnuplot vs pygraph:
gnuplot is still significantly faster (1.63s vs 7.62s) here, but, as mentioned elsewhere, about 3 seconds of that is spend drawing the PNG graph, something we can't work around. plus the graph is prettier and has a more readable X axis. i don't quite understand how gnuplot can read that csv file so fast (reading the CSV in Python is slower than the whole gnuplot run)... maybe there's some stuff running in parallel?
but anyways, gnuplot doesn't provide us with expiration time (#12) so i think pygraph still wins. :)