fixes #21592 - uses headless chrome instead of phantomjs #4987

ohadlevy · 2017-11-07T12:20:33Z

it seems that we can also cut down the default waiting time
which hopefully will speed up our tests.

theforeman-bot · 2017-11-07T12:20:40Z

waldenraines

Nice change, approved pending green tests.

ohadlevy · 2017-11-08T08:45:01Z

@mmoll @ekohl any idea why the test fail? I'm installing chromedriver via npm (search from chromedriver in http://ci.theforeman.org/job/test_develop_pr_core/14979/database=postgresql,ruby=2.2,slave=fast/consoleFull) and I'm using the relative path at https://github.com/theforeman/foreman/pull/4987/files#diff-7a6aaeae51145fd37e71db58212fd18bR15) which of course works locally. how can I debug / troubleshoot this?

ekohl · 2017-11-08T09:03:05Z

According to https://github.com/SeleniumHQ/selenium/wiki/ChromeDriver it expects chrome to be in /usr/bin/google-chrome. You're just installing the driver and I think we need to look at a package installation of chrome. A quick search for RPMs doesn't show me any for centos7 so I'll need to dive into it. Since it's selenium it should also support firefox which may work better on centos7 slaves. Need to dive into it a bit to figure out our options.

ohadlevy · 2017-11-08T09:04:59Z

I was under the assumption that I was override it - (its possible via https://sites.google.com/a/chromium.org/chromedriver/capabilities#TOC-Using-a-Chrome-executable-in-a-non-standard-location ) also, we could consider setting up a symlink?

…

On Wed, Nov 8, 2017 at 11:03 AM, Ewoud Kohl van Wijngaarden < ***@***.***> wrote: According to https://github.com/SeleniumHQ/selenium/wiki/ChromeDriver it expects chrome to be in /usr/bin/google-chrome. You're just installing the driver and I think we need to look at a package installation of chrome. A quick search for RPMs doesn't show me any for centos7 so I'll need to dive into it. Since it's selenium it should also support firefox which may work better on centos7 slaves. Need to dive into it a bit to figure out our options. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#4987 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AABOx1g61pQOp2iOo0XfK45gy3n8OHBIks5s0W5JgaJpZM4QUtDX> .

ekohl · 2017-11-08T09:08:57Z

But the entire chrome browser isn't on NPM, only the chrome driver for selenium. One reason I can think of why chrome isn't packaged for EL7 is that it uses some kernel features for sandboxing that the EL7 kernel doesn't support, but this is speculation from my side. It may be that the Fedora packages do work. One path we can choose is to get Fedora slaves for the UI tests but I'd prefer to try a more generic solution first with Firefox.

ohadlevy · 2017-11-08T09:10:14Z

On Wed, Nov 8, 2017 at 11:08 AM, Ewoud Kohl van Wijngaarden < ***@***.***> wrote: But the entire chrome browser isn't on NPM, only the chrome driver for selenium. One reason I can think of why chrome isn't packaged for EL7 is that it uses some kernel features for sandboxing that the EL7 kernel doesn't support, but this is speculation from my side. It may be that the Fedora packages do work. One path we can choose is to get Fedora slaves for the UI tests but I'd prefer to try a more generic solution first with Firefox.

Are you suggesting we will drop this pr in favor of firefox based approach?

ekohl · 2017-11-08T09:14:43Z

That would be my suggestion to start with. I have experience with using Firefox + Selenium on CentOS7 and generally that works well. If the setup code is generic enough we can support chrome via an env var for users who prefer that on their desktops.

Note that Firefox and xvfb are already installed on our slaves but the job config may need to be modified. Looks like the 1.11-stable job uses it.

ohadlevy · 2017-11-08T09:18:20Z

On Wed, Nov 8, 2017 at 11:14 AM, Ewoud Kohl van Wijngaarden < ***@***.***> wrote: That would be my suggestion to start with. I have experience with using Firefox + Selenium on CentOS7 and generally that works well. If the setup code is generic enough we can support chrome via an env var for users who prefer that on their desktops. Note that Firefox and xvfb are already installed on our slaves but the job config may need to be modified. Looks like the 1.11-stable job uses it.

the fact is that currently the tests are: a. slow b. inaccurate as they represent an old browser env (which imho is no longer relevant to our users - e.g. IE). I would like to unblock / improve our test infrastructure. whats the timeline you had in mind to implement this? thanks.

ekohl · 2017-11-08T09:25:01Z

That makes total sense, but I haven't thought about it. We have a few priorities to balance no. I think that Rails 5.1 SCL + Updated Ruby is also a high priority blocker for developers but the fixing the Dynflow service for the 1.16 release is my top priority. Perhaps this is small enough to do it before the Rails 5.1 effort but I'll get back to you on that.

ohadlevy · 2017-11-08T09:29:57Z

another option is to run it on travis - as its already configured to support chrome.
the main problem is that currently travis works on one language at a time (e.g. ruby or node) and not both.

ekohl · 2017-11-08T09:33:05Z

Travis has matrix builds. See https://github.com/theforeman/puppet-foreman/blob/master/.travis.yml for how we use separate RVMs. Pretty sure you can choose a different language in the include.

ohadlevy · 2017-11-08T09:34:48Z

I failed to make this work (tried at ManageIQ/manageiq-ui-classic#2540) without success.

…

On Wed, Nov 8, 2017 at 11:33 AM, Ewoud Kohl van Wijngaarden < ***@***.***> wrote: Travis has matrix builds. See https://github.com/theforeman/ puppet-foreman/blob/master/.travis.yml for how we use separate RVMs. Pretty sure you can choose a different language in the include. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#4987 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AABOx7RcA5atzFRv5HiQOkPqEqilqbZMks5s0XVRgaJpZM4QUtDX> .

ekohl · 2017-11-08T09:36:22Z

It appears you're right: travis-ci/travis-ci#4090

ehelms · 2017-11-08T12:43:00Z

The Travis configuration is already running node based tests. Where does the Ruby part come into play?

ohadlevy · 2017-11-08T12:49:18Z

@ehelms chrome integration tests run against the rails server.... you need to be able to execute rake jenkins:integration

ehelms · 2017-11-08T12:53:18Z

One broader issue I will point out is that our current standard devel environment via Forklift is based on using CentOS 7 given that is the production RPM distro we support. This change would imply that developers could no longer run these tests within the local environment.

Has anyone tried installing the 'Fedora' only Chrome on EL7? From what I read, when they dropped it they dropped it intending to drop for EL6 but dropped it for all. Should be a fairly easy test on a Forklift environment to configure:

[google-chrome]
name=google-chrome - \$basearch
baseurl=http://dl.google.com/linux/chrome/rpm/stable/\$basearch
enabled=1
gpgcheck=1
gpgkey=https://dl-ssl.google.com/linux/linux_signing_key.pub

Install chrome and try this PR out. If that works, then we can add this to our slaves no?

ehelms · 2017-11-08T12:54:14Z

@ohadlevy ahh so these are for the older JS tests not the new ones coming with the React changes? Or even those requires a running Rails server?

ohadlevy · 2017-11-08T12:59:48Z

ahh so these are for the older JS tests not the new ones coming with the React changes? Or even those requires a running Rails server?

this PR aim is to replace phantomjs with headless chrome- the motivation is to:

reduce integeration test time
use a more modern infrastructure - phantomjs seems like a less of an ideal tool today, we need polyfills etc as its emulated fairly older browsers.
(I must admit I don't know if we have an official minimum browser defined anywhere).

regarding the tests itself:

phatomjs and chrome are both doing full end to end tests using an automated browser - you require a
full rails server.
webpack based code tests (using jest) do not require a browser - they run purely in travis today using jsdom.

ehelms · 2017-11-09T14:12:10Z

So testing this out, I was able to install Chrome on EL7 without issues (running the tests now) using that repository. From what I read, they "pulled" Chrome from EL6 due to incompatibility and in the process pulled it from all all EL despite it still working just fine.

Assuming tests pass, I'd opt for adding this repository to our slaves and allowing the use of Chrome here.

ekohl · 2017-11-09T14:28:20Z

@ehelms I think that's fine.

ohadlevy · 2017-11-11T18:04:30Z

@ehelms whats the next step ?

In order to use chrome headless with Selenium, and get rid of phantomjs. For the moment, both can be installed to test theforeman/foreman#4987 When that PR is merged and we no longer need phantomjs and firefox, we can remove them on another PR.

dLobatog

@ohadlevy @ehelms @ekohl I think a good strategy to move forward is:

Installing chrome alongside firefox/phantomjs in our slaves
Making sure tests pass on Jenkins in this PR
Remove phantomjs/firefox

I've submitted theforeman/foreman-infra#381, as soon as that is merged please trigger the tests here again to see if they pass. I'd be thrilled to see a default_max_wait_time=5 working 😄

In order to use chrome headless with Selenium, and get rid of phantomjs. For the moment, both can be installed to test theforeman/foreman#4987 When that PR is merged and we no longer need phantomjs and firefox, we can remove them on another PR.

mmoll · 2018-01-05T20:34:47Z

@ohadlevy a lot of tests fail and I have the feeling a lot of tests are run with this new configuration the first time after a long while... how can that be?

dLobatog · 2018-02-14T08:49:19Z

[test foreman]

dLobatog

@ohadlevy I think this needs rebasing so tests pass, could you try that? Thanks!

mmoll · 2018-02-17T17:08:16Z

rebased, no change.

ohadlevy · 2018-03-12T10:42:27Z

@dLobatog @mmoll - can we upgrade the chrome version?

mmoll · 2018-03-12T11:58:11Z

Currently not, as it was agreed to use the chromium version coming in EPEL.

mmoll · 2018-05-04T01:40:58Z

test/integration_test_helper.rb

@@ -213,6 +212,7 @@ def database_cleaner_strategy

  def login_admin
    SSO.register_method(TestSSO)
+    visit '/'


while this fixes some tests, some are also failing because of this (e.g. TEST=test/integration/fact_value_test.rb)

mmoll · 2018-05-04T01:56:52Z

I think the changes itself are OK, somebody:tm: would need to have a look into the problem with the cookies vs. visting a page first and then fix a few remaining problems.

it seems that we can also cut down the default waiting time which hopefully will speed up our tests.

ohadlevy · 2018-07-04T13:20:13Z

closing in favor of #5770

theforeman-bot added Needs testing Not yet reviewed labels Nov 7, 2017

waldenraines previously approved these changes Nov 7, 2017

View reviewed changes

dLobatog mentioned this pull request Nov 13, 2017

Install google-chrome in slaves theforeman/foreman-infra#381

Merged

dLobatog reviewed Nov 13, 2017

View reviewed changes

dLobatog suggested changes Feb 14, 2018

View reviewed changes

theforeman-bot added Waiting on contributor and removed Needs re-review labels Feb 14, 2018

mmoll force-pushed the headless-chrome branch from bb61011 to 9ee880e Compare February 17, 2018 14:35

theforeman-bot added Needs re-review Needs testing and removed Waiting on contributor labels Feb 17, 2018

mmoll added Waiting on contributor and removed Needs re-review labels Feb 17, 2018

ohadlevy force-pushed the headless-chrome branch from 9ee880e to c145a62 Compare March 11, 2018 14:43

theforeman-bot added Needs re-review Needs testing and removed Waiting on contributor labels Mar 11, 2018

ohadlevy force-pushed the headless-chrome branch from c145a62 to cef1974 Compare March 11, 2018 18:47

mmoll force-pushed the headless-chrome branch from cef1974 to fcef5e4 Compare May 4, 2018 00:07

mmoll reviewed May 4, 2018

View reviewed changes

mmoll force-pushed the headless-chrome branch from fcef5e4 to 72d7123 Compare May 11, 2018 18:40

fixes #21592 - uses headless chrome instead of phantomjs

cc882a4

it seems that we can also cut down the default waiting time which hopefully will speed up our tests.

mmoll dismissed waldenraines’s stale review via cc882a4 June 7, 2018 17:35

mmoll force-pushed the headless-chrome branch from 72d7123 to cc882a4 Compare June 7, 2018 17:35

ohadlevy mentioned this pull request Jul 4, 2018

[WIP] Stabilize integration tests #5770

Closed

ohadlevy closed this Jul 4, 2018

ohadlevy deleted the headless-chrome branch October 21, 2018 13:48

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fixes #21592 - uses headless chrome instead of phantomjs #4987

fixes #21592 - uses headless chrome instead of phantomjs #4987

ohadlevy commented Nov 7, 2017

theforeman-bot commented Nov 7, 2017

waldenraines left a comment

ohadlevy commented Nov 8, 2017

ekohl commented Nov 8, 2017

ohadlevy commented Nov 8, 2017 via email

ekohl commented Nov 8, 2017

ohadlevy commented Nov 8, 2017 via email

ekohl commented Nov 8, 2017

ohadlevy commented Nov 8, 2017 via email

ekohl commented Nov 8, 2017

ohadlevy commented Nov 8, 2017

ekohl commented Nov 8, 2017

ohadlevy commented Nov 8, 2017 via email

ekohl commented Nov 8, 2017

ehelms commented Nov 8, 2017

ohadlevy commented Nov 8, 2017

ehelms commented Nov 8, 2017

ehelms commented Nov 8, 2017

ohadlevy commented Nov 8, 2017

ehelms commented Nov 9, 2017

ekohl commented Nov 9, 2017

ohadlevy commented Nov 11, 2017

dLobatog left a comment

mmoll commented Jan 5, 2018 •

edited

Loading

dLobatog commented Feb 14, 2018

dLobatog left a comment

mmoll commented Feb 17, 2018

ohadlevy commented Mar 12, 2018

mmoll commented Mar 12, 2018

mmoll May 4, 2018

mmoll commented May 4, 2018

ohadlevy commented Jul 4, 2018

fixes #21592 - uses headless chrome instead of phantomjs #4987

fixes #21592 - uses headless chrome instead of phantomjs #4987

Conversation

ohadlevy commented Nov 7, 2017

theforeman-bot commented Nov 7, 2017

waldenraines left a comment

Choose a reason for hiding this comment

ohadlevy commented Nov 8, 2017

ekohl commented Nov 8, 2017

ohadlevy commented Nov 8, 2017 via email

ekohl commented Nov 8, 2017

ohadlevy commented Nov 8, 2017 via email

ekohl commented Nov 8, 2017

ohadlevy commented Nov 8, 2017 via email

ekohl commented Nov 8, 2017

ohadlevy commented Nov 8, 2017

ekohl commented Nov 8, 2017

ohadlevy commented Nov 8, 2017 via email

ekohl commented Nov 8, 2017

ehelms commented Nov 8, 2017

ohadlevy commented Nov 8, 2017

ehelms commented Nov 8, 2017

ehelms commented Nov 8, 2017

ohadlevy commented Nov 8, 2017

ehelms commented Nov 9, 2017

ekohl commented Nov 9, 2017

ohadlevy commented Nov 11, 2017

dLobatog left a comment

Choose a reason for hiding this comment

mmoll commented Jan 5, 2018 • edited Loading

dLobatog commented Feb 14, 2018

dLobatog left a comment

Choose a reason for hiding this comment

mmoll commented Feb 17, 2018

ohadlevy commented Mar 12, 2018

mmoll commented Mar 12, 2018

mmoll May 4, 2018

Choose a reason for hiding this comment

mmoll commented May 4, 2018

ohadlevy commented Jul 4, 2018

mmoll commented Jan 5, 2018 •

edited

Loading