Accuracy

sandal edited this page Dec 6, 2011 · 39 revisions

Features which simply don't work at all are easy to spot, but incorrectly implemented features can be harder to notice before it's too late. Unfortunately, a single malfunctioning feature can be enough to make an otherwise excellent piece of software completely unsuitable for any task which depends on that feature. In the process of building the reporting system I demonstrated in Practicing Ruby 2.13, I ran into exactly this sort of issue with the Gruff graphing library.

To illustrate the problem I ran into, I've slightly modified one of Gruff's sample programs to set the baseline_value property. Based on the code below, I expected that Gruff would draw a baseline at y=3 on the graph.

require 'gruff'

g = Gruff::Line.new
g.title = "My Graph" 
         
g.data("Apples", [1, 2, 3, 4, 4, 3])
g.data("Oranges", [4, 8, 7, 9, 8, 9])
g.data("Watermelon", [2, 3, 1, 5, 6, 8])
g.data("Peaches", [9, 9, 10, 8, 7, 9])
             
g.labels         = {0 => '2003', 2 => '2004', 4 => '2005'}
g.baseline_value = 3
    
g.write('my_fruity_graph.jpg')                              

However, as you can see from the following image, the code which computes where the baseline needs to be drawn is inaccurate. Rather than showing up exactly two-thirds of the way between y=1 and y=4, the baseline is drawn very close to y=4 instead.

Example

It took me a while to spot this issue in my real report, because I was computing a modified average of my data series and didn't closely check its expected value after wiring up the visualization code. With my initial test data, the baseline was appearing reasonably close to where I thought it should have been. However, a subtle change in the input data made it obvious that the baseline was very much in the wrong position, and this forced me to abandon the library after already completing two graph reports.

A benefit of using open source software is that we can find and fix these bugs ourselves, but that is often easier said than done. In this particular situation I decided to just use a different tool, but in other situations digging into Gruff's source may have been the better option. In any case, this example serves as a reminder of how something that does 95% of what you want may still end up failing you in the end.

The key lesson that library maintainers should take away from this story is that if a feature is not working properly, it is better to remove it completely or explicitly disable it until it gets fixed, as this will help prevent users from getting far along in their development process before discovering issues with your code. In my own projects I've done this in some situations but not in others, and in most cases where I didn't, I regretted it.

As for application developers, the important thing to keep in mind is that you inherit both your own mistakes as well as the mistakes of those who have built your dependencies, so be sure to keep up with rigorous manual and automated testing.


Turn the page if you're taking the linear tour, or feel free to jump around via the sidebar.

You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.
Press h to open a hovercard with more details.