Issue running the test in Travis

[lifeofmle]

When I try to run BackstopJS on Travis I get the following error

COMMAND | Error: docker run --rm -it --mount type=bind,source="/home/travis/build/design-system",target=/src backstopjs/backstopjs:4.1.9 test "--moby" "--config=tools/backstop/backstop.json" returned 1
              at ChildProcess.<anonymous> (/home/travis/build/design-system/node_modules/backstopjs/core/util/runDocker.js:50:18)
              at emitTwo (events.js:126:13)
              at ChildProcess.emit (events.js:214:7)
              at Process.ChildProcess._handle.onexit (internal/child_process.js:198:12)

Full output here https://gist.github.com/lifeofmle/f72f2b0d69238ed178acc5d8c5533b6f

Here is my config file
https://gist.github.com/lifeofmle/7e87e0f5b0e2ebf3a275812ce22b8670

It works when I run the command locally.

I don't know how I can debug this issue further

[mzuffolini]

Hi, I'm currently trying to do exactly the same thing. I have a feeling I've solved that step already but then found further issues down the line. Could you post your travis.yml (of course the non-sensible parts)? Maybe I can get you forward to the same problem I'm having, or maybe you fix it and I find hints to fix my problem as well...

EDIT: I have in fact the same identical issue as above.

I have looked for a solution around and I've found out that Puppeteer doesn't come with all the dependencies needed to run headless Chrome.
So, as suggested, I have tried adding this to travis.yml:

language: node_js
dist: trusty
addons:
  apt:
    packages:
      - libnss3
before_install:
- "sysctl kernel.unprivileged_userns_clone=1"
- "export DISPLAY=:99.0"
- "sh -e /etc/init.d/xvfb start"
- sudo apt-get install gconf-service libasound2 libatk1.0–0 libc6 libcairo2 libcups2 libdbus-1–3 libexpat1 libfontconfig1 libgcc1 libgconf-2–4 libgdk-pixbuf2.0–0 libglib2.0–0 libgtk-3–0 libnspr4 libpango-1.0–0 libpangocairo-1.0–0 libstdc++6 libx11–6 libx11-xcb1 libxcb1 libxcomposite1 libxcursor1 libxdamage1 libxext6 libxfixes3 libxi6 libxrandr2 libxrender1 libxss1 libxtst6 ca-certificates fonts-liberation libappindicator1 libnss3 lsb-release xdg-utils wget -y

But this breaks the build, with the following output:

$ sudo apt-get install gconf-service libasound2 libatk1.0–0 libc6 libcairo2 libcups2 libdbus-1–3 libexpat1 libfontconfig1 libgcc1 libgconf-2–4 libgdk-pixbuf2.0–0 libglib2.0–0 libgtk-3–0 libnspr4 libpango-1.0–0 libpangocairo-1.0–0 libstdc++6 libx11–6 libx11-xcb1 libxcb1 libxcomposite1 libxcursor1 libxdamage1 libxext6 libxfixes3 libxi6 libxrandr2 libxrender1 libxss1 libxtst6 ca-certificates fonts-liberation libappindicator1 libnss3 lsb-release xdg-utils wget -y
Reading package lists...
Building dependency tree...
Reading state information...
E: Unable to locate package libatk1.0–0
E: Couldn't find any package by glob 'libatk1.0–0'
E: Couldn't find any package by regex 'libatk1.0–0'
E: Unable to locate package libdbus-1–3
E: Unable to locate package libgconf-2–4
E: Unable to locate package libgdk-pixbuf2.0–0
E: Couldn't find any package by glob 'libgdk-pixbuf2.0–0'
E: Couldn't find any package by regex 'libgdk-pixbuf2.0–0'
E: Unable to locate package libglib2.0–0
E: Couldn't find any package by glob 'libglib2.0–0'
E: Couldn't find any package by regex 'libglib2.0–0'
E: Unable to locate package libgtk-3–0
E: Unable to locate package libpango-1.0–0
E: Couldn't find any package by glob 'libpango-1.0–0'
E: Couldn't find any package by regex 'libpango-1.0–0'
E: Unable to locate package libpangocairo-1.0–0
E: Couldn't find any package by glob 'libpangocairo-1.0–0'
E: Couldn't find any package by regex 'libpangocairo-1.0–0'
E: Unable to locate package libx11–6
The command "sudo apt-get install gconf-service libasound2 libatk1.0–0 libc6 libcairo2 libcups2 libdbus-1–3 libexpat1 libfontconfig1 libgcc1 libgconf-2–4 libgdk-pixbuf2.0–0 libglib2.0–0 libgtk-3–0 libnspr4 libpango-1.0–0 libpangocairo-1.0–0 libstdc++6 libx11–6 libx11-xcb1 libxcb1 libxcomposite1 libxcursor1 libxdamage1 libxext6 libxfixes3 libxi6 libxrandr2 libxrender1 libxss1 libxtst6 ca-certificates fonts-liberation libappindicator1 libnss3 lsb-release xdg-utils wget -y" failed and exited with 100 during .
Your build has been stopped.

Any idea? People seemed to be happy with this up to 20 days ago... Maybe @lifeofmle you want to try as well and see what happens?

[lifeofmle]

@mzuffolini I got the same error as you above when I tried to set those before_install steps

[mzuffolini]

Ok, thanks @lifeofmle. I hope somebody can help...

[garris]

Does this help solve the issue?
https://markus.oberlehner.net/blog/using-docker-to-run-backstopjs-powered-visual-regression-tests/

[lifeofmle]

@garris Thank you for the article. I can see that my config file is being read so I didn't do the copy configuration file step in that article.

I did add the following to my travis.yml file

services:
  - docker

However it still error I initially posted.

[mzuffolini]

Hi @garris, thanks for the article but unfortunately it doesn't solve the issue, I already had that services: - docker line in place. Any other idea by chance?

[garris]

In the docs, there is a sanity test. See here... https://github.com/garris/BackstopJS#sanity-test-does-backstop-work-in-my-environment

Is it possible for you to use this to test the docker implementation in Travis?

I don't think I have tested this test in Travis -- but it would be important that we validate this first. I would very much appreciate it if you can give it a try.

[mzuffolini]

Hi @garris, thanks for this, I've put that script in the "before install" section on travis.yml and this is the outcome:

$ mkdir backstopSanityTest 
$ cd backstopSanityTest 
$ npm install backstopjs
+ backstopjs@4.1.9
updated 3 packages and audited 17822 packages in 16.74s
$ node ./node_modules/backstopjs/cli/ init
internal/modules/cjs/loader.js:638
    throw err;
    ^
Error: Cannot find module '/home/travis/build/[my repo name]/backstopSanityTest/node_modules/backstopjs/cli'
    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:636:15)
    at Function.Module._load (internal/modules/cjs/loader.js:562:25)
    at Function.Module.runMain (internal/modules/cjs/loader.js:829:12)
    at startup (internal/bootstrap/node.js:283:19)
    at bootstrapNodeJSCore (internal/bootstrap/node.js:622:3)
The command "node ./node_modules/backstopjs/cli/ init" failed and exited with 1 during .
Your build has been stopped.

I hope it helps?

[garris]

Cannot find module '/home/travis/build/[my repo name]/backstopSanityTest/node_modules/backstopjs/cli'

This looks like an issue with your Travis config. Maybe try to make sure everything is getting put in the right place?

[mzuffolini]

Thanks, I'll have a look and let you know

[lifeofmle]

@garris Thanks for the reference about running the smoke test.

I got something running and producing the images and fails because I didn't have any reference files and the report folder is not configured properly. I am ok with that.

I guess the error with my Travis issue then is with my config potentially?

Browser Console Log 0: JSHandle:BackstopTools have been installed.
SCENARIO > BackstopJS Homepage
x Close Browser
x Close Browser
      COMMAND | Executing core for "report"
      compare | Reference image not found backstop_default_BackstopJS_Homepage_0_document_0_phone.png
      compare | Reference image not found backstop_default_BackstopJS_Homepage_0_document_1_tablet.png
       report | Test completed...
       report | 0 Passed
       report | 2 Failed
       report | Writing browser report
       report | Resources copied
       report | Copied json report to: /src/backstop_data/bitmaps_test/20190716-053707/report.json
       report | Copied jsonp report to: /src/backstop_data/html_report/config.js
      COMMAND | Executing core for "openReport"
   openReport | Attempting to ping 
   openReport | Remote not found. Opening backstop_data/html_report/index.html
       report | *** Mismatch errors found ***
      COMMAND | Command "report" ended with an error after [0.049s]
      COMMAND | Error: Mismatch errors found.
                    at /usr/local/lib/node_modules/backstopjs/core/command/report.js:189:17
                    at process._tickCallback (internal/process/next_tick.js:68:7)
      COMMAND | Command "test" ended with an error after [1.636s]
      COMMAND | Error: Mismatch errors found.
                    at /usr/local/lib/node_modules/backstopjs/core/command/report.js:189:17
                    at process._tickCallback (internal/process/next_tick.js:68:7)
COMMAND | Executing core for "openReport"
openReport | Attempting to ping 
   COMMAND | Command "test" ended with an error after [59.109s]
   COMMAND | Error: docker run --rm -it --mount type=bind,source="/home/travis/build/designsystem",target=/src backstopjs/backstopjs:4.1.9 test "--moby" returned 1
                 at ChildProcess.<anonymous> (/home/travis/build/designsystem/node_modules/backstopjs/core/util/runDocker.js:50:18)
                 at emitTwo (events.js:126:13)
                 at ChildProcess.emit (events.js:214:7)
                 at Process.ChildProcess._handle.onexit (internal/child_process.js:198:12)
openReport | Remote not found. Opening backstop_data/html_report/index.html
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
The command "script/travis" exited with 1.

[lifeofmle]

I changed my test to point to a deployed version of my website rather than trying to run my website on using Express while on Travis.

Backstop is working for me on Travis and I am getting generated diff files now.

[garris]

Ah, fantastic -- Great to hear!

So, do you think that running your express app locally (inside of travis) was an issue? Are you able to hit your app running in travis with a conventional test? e.g. In travis, can you launch your express app and then do an ordinary integration test which will spin up the server and curl to your app for a response?

[lifeofmle]

@garris I tried to run my express app and wasn't able to curl to check for response so I think the issue is with me rather than Backstop.