Repository navigation
How GraphScan scans graphs
GraphScan can take a screenshotted graph from an article and infer the numerical values shown. It treats the screenshot as a (X x Y x 4) RGBA array.
In general: where the graph is inside a solid box, the solid axes are continuous throughout their corresponding row/ column. As shown in Fig. A, summation of the RGB array over rows/ columns identifies the axes as sharp inflection points. The left/ right or top/ bottom axes can be identified as maxima either side of the image mid-point (shown by dotted and dashed lines in Fig. A).
The left/ right or top/ bottom axes can be differentiated from each other as maxima either side of the image mid-point (shown by dotted and dashed lines in Fig. A).
Figure A: Schematic showing method of automatic axis identification for graph images with solid enclosing borders
In exceptional cases: where the graph does not have upper and right axes (Fig Ai) or a graph feature is parallel to the upper axis (Fig Aii), the method shown in Fig A fails. Here, the ‘manually select area’ button in the GUI may be used, with the user clicking and dragging to select the graph area corresponding to the entered axis limits (based on this code). After clicking confirm, these axis limits and identified edges are applied to the image.
Ai
Aii
Figure 7: Examples of images where automatic axis identification method fails. b) Graph without upper and right bounding box, from H. Köse, et al (10.1016/j.synthmet.2020.116471) , c) graph with black horizontal data (here, Coulombic efficiency) from G. Li, et al 10.1038/ncomms10601).
The interpolation method divides the graph image into strips (approximately corresponding to one galvanostatic cycle, using the array size and axis limits entered in step 7). Each ‘strip’ may include multiple array columns depending on figure size and resolution.
Areas at the top and bottom of each strip can be omitted by adjusting the ‘margin’ to exclude axis ticks that are difficult to remove using the mask function: this is shown by the horizontal green dashed lines in Figure C, left.
The target colour for each data series set in step 8 is converted to a (1x3) RGB array, with each colour subtracted from the (rowsxcolumnsx4) RGBA image array such that the difference for pixels matching the target color will be ~0. The range of values for which this difference is treated as ~0 is set using the ‘tolerance’ slider (shown as vertical black lines in Fig. C, middle).
The range of rows for which all red, green, and blue are within this range are returned as indices. The inferred value is the median of the list of indices is rescaled using the array pixels: capacity (mAh/g) determined using values from step 7.
Figure C: Schematic showing interpolation of numerical data from RGB image. Left: ‘slice’ of single galvanostatic cycle (stretched horizontally for clarity). Red, blue, and black horizontal markers are data. Horizontal green dashed lines are ‘margins’ to exclude axis markers (shown at top). Middle: difference between target RGB value and image RGB value, where 0 indicates a ‘match’. Vertical black lines correspond to match tolerance (set by user). The set of indices where red and green and blue are within tolerance (right, expanded view) are returned, and the median index value used as the marker position.
Where the data is monochrome (multiple data series of black points), the ‘monochrome graph?’ option can be selected (see Fig. 10a). To function correctly, the data series must be entered in order of appearance from top to bottom in step 8. The algorithm identifies sets of neighbouring non-white points in each vertical strip, where sets of a minimum of 4 continuous neighbouring points are used to avoid spurious black pixels.
Figure D: Schematic showing interpolation of numerical data from RGB image where all data are plotted in black. Right: example slice of 3 data series plotted in black. Middle: RGB image values, showing equal for red, green, and blue. Horizontal dashed lines: margins used to exclude axis ticks, vertical dashed line: tolerance set to 0.98 (with values <0.98 passed). Right: Array indices where black points are detected, with continuous ‘True’ values for >4 neighbouring pixels used to infer presence of marker (data from N. Jayaprakash et al (10.1002/anie.201100637 used for illustrative purposes only)