Questions about flaky comparison results

[jradom]

Hello,

I use your tool and it had worked great for me until now. Here Is my scenario.
I have an app that is going through an upgrade.

1. I make screenshots using Selenium for several pages and store them as baseline images (before the upgrade).
2. I run my app after the upgrade and make screenshots of the same pages
3. I compare on the fly each new and old/baseline pages and use .setAllowingPercentOfDifferentPixels(10)
4. Sometimes my result is MATCH or MISMATCH or SIZE_MISMATCH and its' random

What can you suggest to make it work stable?
Oh, differences in fact are minimal. I use the Chrome browser to run my app

Your help will be greatly appreciated

Thanks a lot in advance

Jeff

[romankh3]

Hi @jradom,
Firstly, setAllowingPercentOfDifferentPixels(10) is a really powerful configuration, which was added for you (as I remember).

I'd suggest starting comparing without this config, it's for different MISMATCH and MATCH

If we speak about SIZE_MISMATCH - it's harder. Before say something I need to reproduce it. From my perspective, if you have this result, it means, that the screenshots are not the same.

Hope, it was helpful for yiu.

[jradom]

What does SIZE_MISMATCH actually mean? Is it different size in bytes, number of pixels, size in pixels? Please, clarify. Thanks Jeff

…

On Mon, Aug 31, 2020 at 12:39 AM Roman Beskrovnyi ***@***.***> wrote: Hi @jradom <https://github.com/jradom>, Firstly, setAllowingPercentOfDifferentPixels(10) is a really powerful configuration, which was added for you (as I remember). I'd suggest starting comparing without this config, it's for different MISMATCH and MATCH If we speak about SIZE_MISMATCH - it's harder. Before say something I need to reproduce it. From my perspective, if you have this result, it means, that the screenshots are not the same. Hope, it was helpful for yiu. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#191 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AE4ZGPKYCMZW3LW3QXOKONTSDNHR5ANCNFSM4QJDVS2A> .

[romankh3]

SIZE_MISMATCH means, that the image sizes are different. For example:
image1 has 100x1000 pixels
image2 has 101x1000 pixels
In this case, would be SIZE_MISMATCH

Best regards,
Roman

[jradom]

Now, If images on the surface look identical but ill use some tool and will resize a second one like image1 has 100x1000 pixels image2 has 100x1000 pixels Will they match after resizing? Please, let me know what you think. I use Selenium screenshots programmatically to create baselines and actual images currently. Looks like it might has ome issue. Do you a good tool which allow to make reliable screenshots using Java for example to make reliable images and then comare with your tool? Thanks Jeff

…

On Mon, Aug 31, 2020 at 11:51 PM Roman Beskrovnyi ***@***.***> wrote: SIZE_MISMATCH means, that the image sizes are different. For example: image1 has 100x1000 pixels image2 has 101x1000 pixels In this case, would be SIZE_MISMATCH Best regards, Roman — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#191 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AE4ZGPPJ7AW52UDBAMQZSJLSDSKXHANCNFSM4QJDVS2A> .

[romankh3]

I don't know for sure how selenium works, that's why I can't help with it.
Also, I'm not familiar with automation testing tools and approaches. I'm a developer, not qa engineer..
I wish I can help you...

[jradom]

Actually my question was about resizing of a second image and then compare if that will work or not

…

Sent from my iPhone

On Sep 1, 2020, at 1:45 PM, Roman Beskrovnyi ***@***.***> wrote:  I don't know for sure how selenium works, that's why I can't help with it. Also, I'm not familiar with automation testing tools and approaches. I'm a developer, not qa engineer.. I wish I can help you... — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[jradom]

Hi Roman, It looks like we are moving away from Java to Javascript/Typescript. Can you suggest a similar to your tool (it's really great) but which can be used with Javascript/Typescript? Please, let me know Thanks Jeff

…

On Tue, Sep 1, 2020 at 12:33 PM Jeff Radom ***@***.***> wrote: Now, If images on the surface look identical but ill use some tool and will resize a second one like image1 has 100x1000 pixels image2 has 100x1000 pixels Will they match after resizing? Please, let me know what you think. I use Selenium screenshots programmatically to create baselines and actual images currently. Looks like it might has ome issue. Do you a good tool which allow to make reliable screenshots using Java for example to make reliable images and then comare with your tool? Thanks Jeff On Mon, Aug 31, 2020 at 11:51 PM Roman Beskrovnyi < ***@***.***> wrote: > SIZE_MISMATCH means, that the image sizes are different. For example: > image1 has 100x1000 pixels > image2 has 101x1000 pixels > In this case, would be SIZE_MISMATCH > > Best regards, > Roman > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > <#191 (comment)>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/AE4ZGPPJ7AW52UDBAMQZSJLSDSKXHANCNFSM4QJDVS2A> > . >

[jradom]

Roman,

I've just downloaded your Jar file but how can I run it, put parameters and output. Like I said I plan to call CLI from Typrescript

Please, let me know

Thanks a lot in advance

Jeff

[romankh3]

Jeff,
This jar of image-comparison doesn't have main method to run as a jar.
For this purpose must be created the separated project with the main method with image-comparison as a dependency.

[jradom]

If this is doable to make a jar work in a short time frame it’ll be great. Then, I’ll be able to continue using your great library

…

Sent from my iPhone

On Nov 10, 2020, at 7:53 AM, Roman Beskrovnyi ***@***.***> wrote:  Jeff, This jar of image-comparison doesn't have main method to run as a jar. For this purpose must be created the separated project with the main method with image-comparison as a dependency. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[jradom]

Hi Roman, Like I had said before we like your current tool and I plan to continue using it with Typescript because of great features and results. So, is it a feasible time frame you can make CLI with parameters and output exit values available. Please, let me know if I should wait or just try switching to some other tool. (still prefer yours though). Please, let me know one way or another Thanks in advance Jeff

…

On Tue, Nov 10, 2020 at 7:53 AM Roman Beskrovnyi ***@***.***> wrote: Jeff, This jar of image-comparison doesn't have *main* method to run as a jar. For this purpose must be created the separated project with the main method with image-comparison as a dependency. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#191 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AE4ZGPMF3L3ABW4VU2YECSLSPFOYRANCNFSM4QJDVS2A> .