HTML Report miscalculation between Reference & Test

[Macaret]

I've implemented Visual Regression on a production server, that is triggered when the Gitlab pipeline runs. My issue is that the results are not calculated correctly after I change something to the website and then run a test.

1. Run backstop reference
2. Run backstop test
3. Valid test as it should
4. Change something to the website
5. Run backstop test
6. Valid test - but it should be invalid

What's happening:

• Open the HTML Report;

• The reference & test images are opposite to each other;

• The thumbnail of the reference image shows the original version of the site - ok;

• The thumbnail of the test image shows the new version of the site - ok;

• When I click on the thumbnail of the reference image - the TEST image opens;

I've tried deleting the generated folders that contains the report and the images; I've checked the configuration multiple times;

• On my local computer, the setup works.

[atifmansoor]

The same error is happening to me: clicking on the reference image shows the test image. I'm experiencing this when the tests pass (when the tests fails, then the behavior is as expected)

[mrossi113]

I had the same problem as above issue. I was thinking that BackstopJS was not using the correct reference image; it turned out the mismatch was way too small to pick up in the diff. It was very confusing to have the report show the test image for both reference and test image if no difference is shown. Could the reference image be used when clicking to open the image? (If you right-click the image and open in a new tab, it will show the reference image in the new tab.)