Update Dockerfile, install Chrome and Chromedriver #565

m90 · 2018-03-16T18:58:33Z

As wraith now supports headless chrome, it can be installed in the Docker container.

Also update nodejs to version 8 and use the ruby version specified by the repository's .ruby-version file.

After having built this locally info prints:

$ docker run wraith:chrome info
DEBUG: #################################################
DEBUG:   Command run:        info
DEBUG:   Wraith version:     4.2.1
DEBUG:   Ruby version:       ruby 2.4.1p111 (2017-03-22 revision 58053) [x86_64-linux]
DEBUG:   ImageMagick:        Version: ImageMagick 6.8.9-9 Q16 x86_64 2017-07-15 http://www.imagemagick.org
DEBUG:   PhantomJS version:  2.1.1
DEBUG:   chromedriver version:  ChromeDriver 2.37.543619 (a237acd3116cac3b3f0da42a000502ce3fafcb23)
DEBUG:   CasperJS version:   1.1.4
DEBUG: #################################################

m90 · 2018-03-16T19:02:06Z

This would solve #561

as wraith now supports headless chrome, it can be installed in the docker container - also update nodejs to version 8 and use the ruby version specified by the repository's .ruby-version file

rbayliss · 2018-03-22T12:20:36Z

I tried this out locally, and got the following block of errors:

DEBUG: #################################################
DEBUG:   Command run:        capture config.yaml
DEBUG:   Wraith version:     4.2.1
DEBUG:   Ruby version:       ruby 2.4.1p111 (2017-03-22 revision 58053) [x86_64-linux]
DEBUG:   ImageMagick:        Version: ImageMagick 6.8.9-9 Q16 x86_64 2017-07-15 http://www.imagemagick.org
DEBUG:   PhantomJS version:  2.1.1
DEBUG:   chromedriver version:  ChromeDriver 2.37.544315 (730aa6a5fdba159ac9f4c1e8cbc59bf1b5ce12b7)
DEBUG:   CasperJS version:   1.1.4
DEBUG: #################################################
Config validated. No serious issues found.
Creating Folders
SAVING IMAGES
ERROR: javascript error: module is not defined
JavaScript stack:
ReferenceError: module is not defined
    at eval (eval at executeAsyncScript (:457:5), <anonymous>:3:1)
    at eval (eval at executeAsyncScript (:457:5), <anonymous>:5:5)
    at executeAsyncScript (<anonymous>:457:26)
    at <anonymous>:473:29
    at callFunction (<anonymous>:361:33)
    at <anonymous>:371:23
    at <anonymous>:372:3
  (Session info: headless chrome=65.0.3325.181)
  (Driver info: chromedriver=2.37.544315 (730aa6a5fdba159ac9f4c1e8cbc59bf1b5ce12b7),platform=Linux 4.9.87-linuxkit-aufs x86_64)
WARN: Using fallback image instead
... etc, repeated for each path.

m90 · 2018-03-22T12:23:03Z

What does your config file look like?

andyrichardson · 2018-07-11T10:35:07Z

I just tried this out and it works perfectly. Ideal for anyone who's in the same situation as me where they can't use PhantomJS because flex support is sketchy.

@rbayliss the reason you get that error is because your js scripts are in a module format (not supported when using chrome). See this example for how to write scripts for chrome.

viktoradavid · 2018-07-11T15:23:07Z

Thanks a lot! This used to work for me, but now I keep getting ERROR: chrome not reachable :(

However, it doesn't happen every time. Running this for only one url and 1366 width leads to no issues, adding more resolutions/urls or changing width size breaks ~80% of images.

Running the comparison using /configs/capture.yaml
Config validated. No serious issues found.
Creating Folders
SAVING IMAGES
ERROR: chrome not reachable
  (Session info: headless chrome=67.0.3396.62)
  (Driver info: chromedriver=2.39.562737 (dba483cee6a5f15e2e2d73df16968ab10b38a2bf),platf
rm=Linux 4.13.0-46-generic x86_64)
WARN: Using fallback image instead
ERROR: chrome not reachable
  (Session info: headless chrome=67.0.3396.62)
  (Driver info: chromedriver=2.39.562737 (dba483cee6a5f15e2e2d73df16968ab10b38a2bf),platf
rm=Linux 4.13.0-46-generic x86_64)
WARN: Using fallback image instead
ERROR: chrome not reachable
  (Session info: headless chrome=67.0.3396.62)
  (Driver info: chromedriver=2.39.562737 (dba483cee6a5f15e2e2d73df16968ab10b38a2bf),platf
rm=Linux 4.13.0-46-generic x86_64)
WARN: Using fallback image instead
CROPPING IMAGES
COMPARING IMAGES

What's interesting that addition of another url breaks even the url that previously worked. Testing one url at 1366 works, but two urls at 1366 break both of them. Could chrome be somehow "overloaded"?

Any ideas? TIA!

bbrooks · 2018-10-05T13:35:38Z

@viktoradavid I was running into a similar problem. Seems like chrome needs more resources than the default docker box offers. Try running your docker run cmd with --shm-size=512m to increase the /dev/shm size of the container. It defaults to 64m.

When I did this I no longer got chrome not reachable errors.

viktoradavid · 2018-10-06T06:42:58Z

@bbrooks That's what I needed, thank you so much!

BTW I had to increase the shm-size even more (up to 4 gigs) for some big and complex pages to work.

edurenye · 2018-11-02T16:06:31Z

I think we should use docker compose and have chrome or whatever in a separate container so we can use Firefox or whatever we want, or add it to a bigger compose that already have selenium with chrome for lets say use behat. This way we do not have 2 chromes there.

bbrooks · 2018-11-08T13:29:53Z

@edurenye I agree but how would we get wraith to look for and use the dockerized chrome instead of trying to execute it on the wraith container?

edurenye · 2018-11-08T16:09:33Z

With an ENV variable somehow, maybe? I don't know, we have to figure it out.

nicrodgers · 2019-08-20T10:47:04Z

I just tried to use this for the first time, but am running in to some problems.

google-chrome 76 is installed, and ChromeDriver 77, so when you try to run wraith it complains there is a version mismatch and doesn't run.

ERROR: session not created: This version of ChromeDriver only supports Chrome version 77

peruukki · 2020-01-25T11:04:17Z

I just tried to use this for the first time, but am running in to some problems.

google-chrome 76 is installed, and ChromeDriver 77, so when you try to run wraith it complains there is a version mismatch and doesn't run.

ERROR: session not created: This version of ChromeDriver only supports Chrome version 77

I just had the same problem, it installed Chrome 79 but ChromeDriver 80, and this helped for me:

Changed LATEST_RELEASE to LATEST_RELEASE_79 in the URL that determines the ChromeDriver version
Moved the downloaded ChromeDriver binary to /usr/local/bundle/bin/chromedriver instead of /usr/local/bin/chromedriver to replace the existing version there that Wraith picked up

It would be great to detect what Chrome version gets installed and match ChromeDriver with it but I haven't looked into it, it's not that much of a hassle to manually update the version number.

I also moved the Chrome setup steps last so that it doesn't re-execute the font downloads whenever I need to retry the Chrome related stuff, and moved the version output last for easier comparison. Here's my full Dockerfile:

FROM ruby:2.4.1

WORKDIR /wraith

RUN curl -sL https://deb.nodesource.com/setup_8.x | bash -
RUN apt-get install -y libfreetype6 libfontconfig1 nodejs libnss3-dev libgconf-2-4 \
    && rm -rf /var/lib/apt/lists/*

# make sure npm does not need sudo: https://docs.npmjs.com/getting-started/fixing-npm-permissions
RUN mkdir /wraith/.npm-global
ENV NPM_CONFIG_PREFIX=/wraith/.npm-global
ENV PATH=/wraith/.npm-global/bin:$PATH

# install with --unsafe-perm because of https://github.com/Medium/phantomjs/issues/707
RUN npm install -g phantomjs-prebuilt casperjs --unsafe-perm
RUN gem install wraith --no-rdoc --no-ri

# Make sure decent fonts are installed. Thanks to http://www.dailylinuxnews.com/blog/2014/09/things-to-do-after-installing-debian-jessie/
RUN echo "deb http://ftp.us.debian.org/debian jessie main contrib non-free" | tee -a /etc/apt/sources.list
RUN echo "deb http://security.debian.org/ jessie/updates contrib non-free" | tee -a /etc/apt/sources.list
RUN apt-get update
RUN apt-get install -y ttf-freefont ttf-mscorefonts-installer ttf-bitstream-vera ttf-dejavu ttf-liberation

# Make sure a recent (>6.7.7-10) version of ImageMagick is installed.
RUN apt-get install -y imagemagick

# install chrome and chromedriver (unzip is needed for installing chromedriver)
RUN wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add - \
    && echo "deb https://dl.google.com/linux/chrome/deb/ stable main" > /etc/apt/sources.list.d/google.list \
    && apt-get update \
    && apt-get install -y google-chrome-stable unzip \
    && rm -rf /var/lib/apt/lists/* \
    && sed -i 's|HERE/chrome"|HERE/chrome" --disable-setuid-sandbox --no-sandbox|g' \
           "/opt/google/chrome/google-chrome"

RUN export CHROMEDRIVER_RELEASE=$(curl --location --fail --retry 3 https://chromedriver.storage.googleapis.com/LATEST_RELEASE_79) \
    && curl --silent --show-error --location --fail --retry 3 --output /tmp/chromedriver_linux64.zip "https://chromedriver.storage.googleapis.com/$CHROMEDRIVER_RELEASE/chromedriver_linux64.zip" \
    && cd /tmp \
    && unzip chromedriver_linux64.zip \
    && rm -rf chromedriver_linux64.zip \
    && chmod +x chromedriver \
    && mv chromedriver /usr/local/bundle/bin/chromedriver

RUN google-chrome --version \
    && chromedriver --version

ENTRYPOINT [ "wraith" ]

m90 force-pushed the chrome-docker branch 6 times, most recently from 1aa249b to f4eea6f Compare March 16, 2018 19:59

Update Dockerfile, install Chrome and Chromedriver

e69a142

as wraith now supports headless chrome, it can be installed in the docker container - also update nodejs to version 8 and use the ruby version specified by the repository's .ruby-version file

m90 force-pushed the chrome-docker branch from f4eea6f to e69a142 Compare March 20, 2018 08:23

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update Dockerfile, install Chrome and Chromedriver #565

Update Dockerfile, install Chrome and Chromedriver #565

m90 commented Mar 16, 2018 •

edited

Loading

m90 commented Mar 16, 2018

rbayliss commented Mar 22, 2018

m90 commented Mar 22, 2018

andyrichardson commented Jul 11, 2018

viktoradavid commented Jul 11, 2018 •

edited

Loading

bbrooks commented Oct 5, 2018

viktoradavid commented Oct 6, 2018

edurenye commented Nov 2, 2018

bbrooks commented Nov 8, 2018

edurenye commented Nov 8, 2018

nicrodgers commented Aug 20, 2019

peruukki commented Jan 25, 2020

Update Dockerfile, install Chrome and Chromedriver #565

Are you sure you want to change the base?

Update Dockerfile, install Chrome and Chromedriver #565

Conversation

m90 commented Mar 16, 2018 • edited Loading

m90 commented Mar 16, 2018

rbayliss commented Mar 22, 2018

m90 commented Mar 22, 2018

andyrichardson commented Jul 11, 2018

viktoradavid commented Jul 11, 2018 • edited Loading

bbrooks commented Oct 5, 2018

viktoradavid commented Oct 6, 2018

edurenye commented Nov 2, 2018

bbrooks commented Nov 8, 2018

edurenye commented Nov 8, 2018

nicrodgers commented Aug 20, 2019

peruukki commented Jan 25, 2020

m90 commented Mar 16, 2018 •

edited

Loading

viktoradavid commented Jul 11, 2018 •

edited

Loading