Unexpected differences between reference and test

[armisoft]

Hi, I run backstop reference then backstop test and I always get differences between both without making any changes.
Indeed, I got different differences everytime I run those 2 commands.
Do you see any sense on it?

Here you can find 2 different runs with just some minutes in between:
First: http://i.imgur.com/jan4DUX.png
Second: http://i.imgur.com/E2DJHJ1.png

This is my backstop.json:
{ "id": "Design System", "viewports": [ { "label": "phone", "width": 320, "height": 480 }, { "label": "tablet", "width": 1024, "height": 768 }, { "label": "desktop", "width": 1920, "height": 1080 } ], "onBeforeScript": "puppet/onBefore.js", "onReadyScript": "puppet/onReady.js", "scenarios": [ { "label": "Home", "url": "http://localhost:3000", "referenceUrl": "", "readyEvent": "", "readySelector": "", "delay": 2000, "hideSelectors": [], "removeSelectors": [], "hoverSelector": "", "clickSelector": "", "postInteractionWait": 0, "selectors": [], "selectorExpansion": true, "expect": 0, "misMatchThreshold" : 0.1, "requireSameDimensions": true }, { "label": "Home - SideNav", "url": "http://localhost:3000", "referenceUrl": "", "readyEvent": "", "readySelector": "", "delay": 2000, "hideSelectors": [], "removeSelectors": [], "hoverSelector": "", "clickSelector": ".sidebar__hamburger", "postInteractionWait": 0, "selectors": [], "selectorExpansion": true, "expect": 0, "misMatchThreshold" : 0.1, "requireSameDimensions": true } ], "paths": { "bitmaps_reference": "backstop_data/bitmaps_reference", "bitmaps_test": "backstop_data/bitmaps_test", "engine_scripts": "backstop_data/engine_scripts", "html_report": "backstop_data/html_report", "ci_report": "backstop_data/ci_report" }, "report": ["browser"], "engine": "puppeteer", "engineOptions": { "args": ["--no-sandbox"] }, "asyncCaptureLimit": 5, "asyncCompareLimit": 50, "debug": false, "debugWindow": false }

[garris]

It looks like you have animations. Tell BackstopJS to wait for animations to complete before taking screenshot. Read docs for more info.

[armisoft]

I'm doing it. I tried with 0, 1000, 2000, 8000ms but it is always giving me different results. Even with the same parameters, like 2000ms, I run backstop reference, then backstop test and I got differences, then I run both again without modifying any parameter and I got different differences than the first set of run which is completely ilogic for me. My animation is only a menu sidebar opening which longs 0.3s. This is driving me crazy.

…

On Sat, 25 Jan 2020 at 06:33, Garris ***@***.***> wrote: It looks like you have animations. Tell BackstopJS to wait for animations to complete before taking screenshot. Read docs for more info. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#1140?email_source=notifications&email_token=ACZOZ5MYUXLR2PUFFCOF5XDQ7PFJRA5CNFSM4KJS7DC2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEJ4VZUI#issuecomment-578378961>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACZOZ5MSTAQEY6272NG3OQ3Q7PFJRANCNFSM4KJS7DCQ> .

[garris]

Sorry to hear. Try reducing complexity until you find the issue. Try using debugWindow: true see the docs for tips.

[armisoft]

Complexity is the lowest. You can check it on my first post into the backstop.json.
I'm only asking for this "clickSelector": ".sidebar__hamburger".

[VladimirPittner]

Hi,
I am not 100% sure if I remember correctly (its been some time) but I think delay might not help you here.
What you want to do, is to tell the backstop to wait till the animation ends in the place where you do the click interaction.
Can you share a snippet how you do that?