Different screenshot on different environment

[anton-kulagin]

Hi, @garris . I`m got a problem: I have several CI server and on them i run backstop tests. So the problem is that on both this machine screenshot are different. Either on my machine reference screenshot and my local screenshot are different(both chrome headless)
Step to redroduce:

1. Make reference screenshot on CI server
2. Copy this screenshot to local machine
3. Run the same test
4. Some strange diff for all tests
   

[garris]

What version chrome and backstop are you using. And what version OS are you using?

[garris]

Also please post gist of your config.

[anton-kulagin]

1. Chrome V63
2. CI1 - Windows Server 2008 , CI2 - Windows Server 2012, Local env - Win10. So reference screenshot from CI1 is different for CI2 either local env.

{
      "id": "backstop_default",
      "viewports": [
            {
                  "label": "tablet",
                  "width": 1024,
                  "height": 768
            },
            {
                  "label": "desktop",
                  "width": 1920,
                  "height": 1080
            }
      ],
      "scenarios": [
            {
                  "label": "Tabs_Horizontal_On_Top__AWESOME__Tab_selected",
                  "url": "http://site-adress/tab",
                  "selectors": [
                        ".column-splitter .col-xs-6 .tabs"
                  ],
                  "misMatchThreshold": "0.1",
                  "onBeforeScript": "",
                  "cookiePath": "",
                  "referenceUrl": "",
                  "readyEvent": "",
                  "readySelector": "",
                  "delay": "",
                  "hideSelectors": [],
                  "removeSelectors": [],
                  "onReadyScript": "chromy/clickAndHoverHelper.js",
                  "hoverSelector": "",
                  "clickSelector": ".col-xs-12 .component .col-xs-12.col-xs-6 .col-xs-12.tabs .component-content .tabs-heading li\\:nth-child(3)",
                  "postInteractionWait": "",
                  "selectorExpansion": "",
                  "requireSameDimensions": ""
            }
      ],
      "paths": {
            "bitmaps_reference": "backstop_data/bitmaps_reference",
            "bitmaps_test": "backstop_data/bitmaps_test",
            "engine_scripts": "backstop_data/engine_scripts",
            "html_report": "backstop_data/html_report",
            "ci_report": "backstop_data/ci_report"
      },
      "report": [
            "CI"
      ],
      "engine": "chrome",
      "engineFlags": [],
      "asyncCaptureLimit": 2,
      "asyncCompareLimit": 2,
      "debug": false,
      "debugWindow": false
}

[garris]

Going between environments is one of the hardest things to manage. In many cases, antialiasing happens at a hardware level — and different hardware creates different dithering. On Mac we mostly solved the issue — and going between environments is pretty seamless. I don’t know windows so well. You might need to look into Turin off antialiasing in windows and see if you can make it work. Another approach is to use a docker container. Backstop finally has one — you might want to give it a try. Please post back if you can get it figured out!