Performance test ?

[riston]

Could you also implement small performance test, how the tool is performing with different variety of image resolutions ? (example 100px x 100px; 300px x 300px; 1024px x 768px)

[marcelerz]

I guess what you are asking is a Benchmark for different resolutions instead of Performance tests that check for performance regressions - please correct me if I am mistaken.

It is generally hard to tell what the average performance would be since it all depends on multiple factors:

• CPU
• Memory - But shouldn't be a problem in most cases
• IO - Bytes have to be loaded
• Algorithm - Including the Comparison and the PNG decoding and encoding

Let's focus on the Algorithm for now:
The image comparison in most cases has a time-complexity of O(w_h). However, if Blink-Diff encounters differences AND the shift options are set, then it will search for any shifts within the image (including sub-pixel issues and anti-aliasing). This comes then to O(w_h_x_y). In addition to that, it is possible to add filters before the comparison (i.e. blur to reduce severe problems like mosaics from JPEGs).

The time it takes to decode a PNG has actually a huge impact on the overall performance, especially as the resolution grows. There is the compression which could be off or set to its highest compression rate. There are compression filters that could be "NONE" (copies the bytes as is) or more complex ones like "PAETH" (combines four pixel values for each pixel). There are also some other things in PNG like interlace modes, transparency handling, and palette management, bit-depth, that all can affect the performance.

All of this makes it really difficult to come up with some kind of data. However, to get you some kind of data, I ran a couple of benchmarks with the same image (but different resolutions), and the same PNG encoding, comparing it to an identical image. Here is a scatter-graph of these comparisons:

The X-axis is the dimension of the image (so x*x). The resolutions are: 100x100, 250x250, 500x500, 750x750, 1000x1000, 2000x2000, 3000x3000. The y-axis is the time it took in milliseconds. I broke out the comparison itself and the comparison with IO (IO includes PNG encoding/decoding). As you can see, the IO is the most part of the Blink-Diff comparison. (The actual access to the HD is the slowest here.)

Here, I created the same scatter-graph without the IO data:

Generally, the comparison has a time-complexity of O(n^2). However, it very much so depends on the IO, the PNG encoding, and the configuration set for the comparisons.

I hope this addressed your question.