Headless version for testing #228

Open
mcolyer opened this Issue Apr 9, 2014 · 33 comments

Projects

None yet
@mcolyer
Contributor
mcolyer commented Apr 9, 2014

@zcbenz how much work do you think it would be to create a headless version of atom-shell that could be used as a replacement for phantomjs?

phantomjs is lagging behind more and more from what actual web browsers do today and it would be great to have something more up to date to use for headless testing.

@mcolyer mcolyer added the question label Apr 9, 2014
@zcbenz
Contributor
zcbenz commented Apr 10, 2014

By using hidden browser window atom-shell can actually do what phantomjs does, the example in phantomjs's homepage could be translated to atom-shell:

BrowserWindow = require('browser-window');

console.log('Loading a web page');
var page = new BrowserWindow({show: false});
var url = 'http://www.phantomjs.org/';
page.on('loading-state-changed', (event, isLoading) {
  if (!isLoading)
    //Page is loaded!
    require('app').exit();
});
page.loadUrl(url);

Of course we may need to add some more APIs for automation testing purpose.

The only problem is instead of drawing into a virtual buffer, atom-shell actually draws the page into a real window, this would require a graphic environment, on Window and OS X it doesn't matter, however on Linux we have to use xvfb to provide a X server for atom-shell. This is by design in Chromium's content API, so we can not do anything to remove the dependency of graphic environment.

@zcbenz zcbenz added discussion and removed question labels May 6, 2014
@cpsubrian

I've had success with this lately (using Xvfb on an Ubuntu server). My use-case is capturing screenshots of templated pages. In fact, I've found that atom-shell through Xvfb on the server (an m3-large) has better performance than on my local Macbook pro. This led me to want to get atom-shell working through Xvfb in osx as well.

Since osx ships with Xvfb, that part is easy. Can I get atom-shell to use an Xvfb display in osx? Using the standard DISPLAY env variable like I do in linux is not working. Perhaps libchromiumcontent doesn't know how to use X11 when running in Darwin?

@zcbenz
Contributor
zcbenz commented Jun 26, 2014

Xvfb only works for applications that written for X11 environment, on OS X atom-shell uses Cocoa for display, which I believe can not work with Xvfb.

@cpsubrian

Yeah kinda put that together. Probably not really a way to compile from source for X11 instead?

@FWeinb
Contributor
FWeinb commented Jun 26, 2014

@zcbenz it is currently not possible to create a BrowserWindow that is bigger than the current resolution on OS X, even if you use new BrowserWindow({show: false});

@FWeinb FWeinb referenced this issue in sindresorhus/pageres Jul 3, 2014
Closed

Use node-webkit to create screenshots #91

@sindresorhus
Contributor

@FWeinb can you open a separate specific ticket for the above?

@FWeinb
Contributor
FWeinb commented Jul 10, 2014

Created #475

@zcbenz
Contributor
zcbenz commented Mar 16, 2015

I'm closing this because there is no way to draw a page without actually creating a native widget in Chromium, and I don't think they will ever allow it.

For automatic tests, we do support Selenium.

@zcbenz zcbenz closed this Mar 16, 2015
@etiktin
Contributor
etiktin commented Aug 29, 2015

The CEF project has support for off-screen rendering, so you can draw the screen into a buffer instead of a window. Regarding X server for Linux, it seems there's a way to work without it by adding a target called Ozone (see discussion here).

@zcbenz
Contributor
zcbenz commented Aug 31, 2015

@etiktin Thanks for the information! I'm reopening this since there is an existing implementation on how to do it.

@zcbenz zcbenz reopened this Aug 31, 2015
@zcbenz zcbenz added enhancement and removed discussion labels Aug 31, 2015
@matthewmueller

We could really use support for this. We recently replaced Phantom with Electron in Nightmare and love it so far, but it doesn't work out of the box on Linux.

Here's what we need to do right now to get it working: segmentio/nightmare#224 (comment)

@RickIniguez

I've been given the task to automate user behavior for one of our web applications that will be converted to a stand alone desktop Electron application. Before our company decided to make this move, we created page objects using the chrome web driver and interacted with the web application by invoking buttons/dropdowns/textboxes using the css selectors. Is there a way to do this with a web application that is wrapped with the Electron shell? The company plans on using the menu bar options to invoke certain functionality and I tried to get to the default menu bar options like File/Edit/Help using the JavaScript driver with no success. Are there any examples out there on how to do this?

@fritx
Contributor
fritx commented Sep 23, 2015

segmentio/nightmare#224 (comment) it seems @matthewmueller's snippet works on Linux 👍

@bjrmatos bjrmatos referenced this issue in jsreport/jsreport Oct 25, 2015
Closed

Electron pdf recipe #125

@avindra
avindra commented Nov 17, 2015

Has anyone gotten headless testing working on SuSE? Specifically SLES?

@puzrin
puzrin commented Nov 17, 2015

@fritx The same used for SlimerJS, but that's NOT headless mode.

@Vanuan
Vanuan commented Dec 8, 2015

@fritx that's what @zcbenz was saying, you have to have Xvfb running. Both CEF3 and Chromium Content Shell currently depend on Xlib. But with completion of Ozone: https://www.chromium.org/developers/design-documents/ozone
you'd be able to provide any low level I/O.

@Vanuan
Vanuan commented Dec 8, 2015

Apparently, there is a master bug in Chromium itself: https://code.google.com/p/chromium/issues/detail?id=546953

@Vanuan
Vanuan commented Dec 9, 2015

This is interesting:

Date: Wed Dec 02 15:35:21 2015

[headless] Initial skeleton of headless/public/

Create outline of future Headless API.

@Vanuan
Vanuan commented Dec 9, 2015

Does ChromeDriver work with electron?

@dhruvbhatia

A headless binary that doesn't require xvfb would open up new environments such as AWS Lambda - sign me up!

@isiahmeadows

@Vanuan Have you heard of Nightmare? That might help you if there's nothing you specifically need from ChromeDriver.

@Vanuan
Vanuan commented Dec 21, 2015

Does it have Capybara/Selenium driver?

@kirkouimet
Contributor

+1

@lastmjs
lastmjs commented Apr 30, 2016

I'm a little confused. Is there a headless mode? Can we effectively do this with BrowserWindow({show: false})? This would be very useful to me, I'm trying to get this to work so that we can create server-side web components: https://github.com/scramjs/scram-markup-engine

@lastmjs lastmjs referenced this issue in maxogden/electron-spawn May 1, 2016
Closed

Best way to load and render html #19

@lastmjs
lastmjs commented May 1, 2016

I think I've answered my own question as I've been looking around. Electron does not natively support a sophisticated headless mode. Nightmare seems to allow something like it, but you still have to do some configuration to get it to work on certain systems with no graphical environment. Electron can also do it if you use BrowserWindow({show: false}), but you must use xvfb to provide a graphical environment on headless Linux systems (which doesn't seem too bad actually). Correct me if I'm wrong, and +1 to this feature.

@fritx fritx referenced this issue in vuejs-templates/webpack May 17, 2016
Closed

Replace phantomjs/selenium with electron for testing? #128

@samwillis

With the new chromium headless project [1] would it be possible to make electron headless without using xvfb?

I believe the current limitation was with libchromium? Have the chrome guys fixed that?

1: https://chromium.googlesource.com/chromium/src/+/master/headless/README.md

@mko-io
mko-io commented Jun 21, 2016

Any progress on this? This would be really useful for testing

@thoop thoop referenced this issue in prerender/prerender Jun 30, 2016
Open

Look into replacing PhantomJS with Electron #318

@amilajack
amilajack commented Jul 6, 2016 edited

segmentio/nightmare is a perfect for this. Simply:

const nightmare = Nightmare({
  show: true
});
@isiahmeadows

@amilajack For simple cases like running simple Mocha unit tests headlessly, Nightmare would be like using a 20-pound sledgehammer to drive in a small nail (read: massive overkill). It's a fully fledged, batteries included browser automation library that can not only perform basic navigation and input, but even save HTML and PDF files to disk or take screenshots. Exactly 0% of this library should be necessary to run simple unit tests.

@amilajack
amilajack commented Jul 7, 2016 edited

@isiahmeadows @mcolyer said that he wanted a headless version of atom-shell that could be used as a replacement'. Electron is pretty much exactly that with extra features.

@isiahmeadows

Yeah, but why would you need sugar for what you don't use? (I was referring to all the sugar - you could theoretically re-implement Electron in its entirety with just vanilla Node + OpenGL bindings).

The most common use case for headless browsers are things like what mocha-phantomjs and Karma already exist for - running browser unit tests from the CLI. Most people use xvfb, a headless X server, on Travis if they need to test Firefox/Chrome, because that doesn't have a running X server, and you could even run Electron with that, but headless browsers like PhantomJS and SlimerJS don't need an X server. Electron + Nightmare still needs an X server of some kind (even if it's xvfb) to run, and this issue is asking for that dependency to be removed, but it most likely won't happen until Chromium itself can go headless and those changes propagated to libchromiumcontent.

@sandstrom
sandstrom commented Jan 8, 2017 edited

Since the ~2 weeks ago Chromium support headless mode.

With the latest Chrome Canary on Linux you can enable headless mode with the --headless command line flag. (https://bugs.chromium.org/p/chromium/issues/detail?id=546953#c148)

@sindresorhus @zcbenz Will this change in Chromium make any difference here?

Electron is already wonderful, and a headless mode would make it even better!

(It would also be useful for Nightmare, which is based on Electron)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment