images differences are captured correctly but robot test is passing even when there is a difference with tolerance set to zero

[abdalhamidElnaggar]

Describe the bug
robot test is passing even when there is difference between two images and tolerance is set to 0

To Reproduce
Steps to reproduce the behavior:
1-create new test to compare two images by compare images keyword or compare two images
2-make sure there is a difference between the two images
3-run the test and check robot result

Expected behavior
robot test should fail as there is a difference between the two images
Screenshots
If applicable, add screenshots to help explain your problem.
attached the two images and the diff one
Desktop (please complete the following information):

• OS: MacOS venture 13.3
• Browser : chrome
• Version :110

Additional context

[abdalhamidElnaggar]

it seems like magick is returning the difference as 0 , can this be an issue with magick ?

[jessezach]

@abdalhamidElnaggar I took a look at this. Magick returns a value of 0.002 which is a difference of 0.2%. This gets casted to int which is converting the difference to 0. I'll fix the code to retain the exact value instead of doing the int conversion.

Meanwhile a workaround for this could be to use capture element keyword and capture a portion of the page (headers for example). This should return a higher difference value for the mismatch and would result in a failure

[abdalhamidElnaggar]

@jz-jess thank you .. i will wait for the fix as my plan is to compare full pages not just elements.
thanks again much appreciated.