Running Nightmare headlessly on Linux #224
Comments
|
I tried this and got it returned properly. But if I tried with a website that have a SLL problem, like below, it hang there for, forever. It just hang here. No error, and electron no exited. |
|
What's your environment for running the cnn.js script? Entirely headless server, or in a terminal in an existing X session, or OS X etc? |
|
I do have the same issue on a headless server. Everything runs correctly on OSX but not on my build server. |
|
Yikes, major issue. Seems like the electron linux binding is broken or something. Anyone have any ideas? |
matthewmueller
changed the title
|
Odd that at least you get an error message when I don't. Mine just fails silently. My guess is that it can't get the framebuffer, which is where xvfb would normally come in. But I've tried running electron with that too and just get gtk errors. |
|
Okay. I'm not exactly sure how to codify this right now (or what layer to support this in), but to get it working on the official node (docker) image, you need to do the following: # Install dependencies
apt-get update &&\
apt-get install -y libgtk2.0-0 libgconf-2-4 \
libasound2 libxtst6 libxss1 libnss3 xvfb
npm install segmentio/nightmare
# Start Xvfb
Xvfb -ac -screen scrn 1280x2000x24 :9.0 &
export DISPLAY=:9.0
# Test it
apt-get install vim
vim index.js
# <paste in example>
node index.jsMore info: electron/electron#228 Could use some help on getting this bug resolved. |
matthewmueller
changed the title
|
That doesn't solve it for me on my environment. Still no errors, but still no title from the CNN example either. I had previously got Gtk3::Webkit Perl module running, which also uses xvfb, and had everything installed except libxss1 already. So I know my xvfb works. Going round in circles trying to find suggestions. I see the same as this person when running electron through xvfb-run, https://gist.github.com/AspireToCodeBetter/130877925f52c8fb2557. And the Stack Overflow question points people here, http://stackoverflow.com/questions/32612868/running-electron-atom-shell-headlessly-on-linux-server-through-nightmare-js. |
|
Oh I don't think this is it, but can you try running master? I upgraded it to fix: electron-userland/electron-prebuilt#54 (comment) |
|
Hey! That did it. I can now start Xvfb on display 9 as you posted before, then I can now run the cnn example. Looks like it was an incomplete Electron install before. So can you modify your launch of electron process to use xvfb-run? |
|
@richard5mith i'm not very familiar with I think it's starting a server ( |
|
I was thinking something like this as a variation on your steps, which is how I had run Gtk Webkit . That does seem to launch correctly and the renderer process starts. So I'm not sure if you can do something similar from your JS and then do the IPC with that process. |
|
Actually, this works, without having Xvfb running in the background first as per your previous steps... |
|
@richard5mith thanks for that, runs for me now on my build server! |
|
Nice! Now just need to figure out the best way to get these dependencies on linux boxes (without using custom buildpacks or anything) |
|
@matthewmueller that might help: Using the shippable container to run nightmare and mocha. |
|
@matthewmueller I've narrowed it down to Hope that helps! |
|
Thanks man, unless I'm missing something I think this should do it though: #224 (comment). Or are you talking about without node dependencies? I think the remaining items to sort out are:
I plan on looking at this sometime this week or weekend, but any time offered sorting this stuff out would help accelerate this fix. |
|
I'm unable to fix my #223 Heroku deploy. Updated to Nightmare v2.0.6 and tried custom buildpacks but couldn't sort it out. Has anyone used xvfb on Heroku? |
|
Would be good if we could get some sort of post install script going so it just worked on heroku. Sorry guys, I haven't had much time to look into this lately, but hopefully we can get this sorted out soon |
|
@matthewmueller https://github.com/Rob--W/node-xvfb might be of use for controlling the xvfb server. |
|
@NotSentient Jabin, Simply Wall St looks outstanding. Good find on the xvfb module. |
|
same issue there. |
|
@stevenmiller888 Take a look at https://github.com/captain401/heroku-buildpack-xvfb. You'll also want heroku-buildpack-apt as in the instructions. (You might also need to include libnotify4 and libxss1 in your Aptfile - let me know if I need to update the instructions.) |
|
My website was deployed on EC2, and I tried |
|
@otaviomedeiros : Hi ! Did u manage to launch multiple long script at the same time with this config ? |
|
@dachow : it's seems that you killed the electron child process before the other request had time to finish. show us your code ! |
|
@GautierT Hi! I have ran it with express. The basic structure of code like this: It processes the request only once |
|
@dachow, that's because you are ending the instance on first run. |
|
@dachow I think you can make a new instance everytime the route is called, if your resources can handle it so |
|
Folks following my original post on August 11, 2016 I have some more information to share. I noticed some others had similar problem that I just identified. Going back to my post you'll see I mentioned how to get nightmare.js working on linux via the use of pm2 and Xvfb. This still applies to a linux bare metal of vm installation. However to run nightmare in a docker a few things to keep in mind:
Hopefully one or more of these points helps you folks. |
|
@GautierT, no need to have multiple Xvfb displays in my experience. One Xvfb display and as many simultaneous nightmare processes as your system can handle reasonably. |
|
One more thing that I encountered when trying to run using xvfb-run. Apparently, if a process executing Xvfb has pid 1, then Xvfb does not send SIGUSR1 which is what xvfb-run relies on. |
|
Since the ~2 weeks ago Chromium support headless mode.
@matthewmueller @rosshinkley Will this change in Chromium make any difference here? Nightmare is awesome, and a headless mode (without xvfb) would be very useful! 🥇 (here is a related issue in Electron electron/electron#228) |
|
@sandstrom That's really cool! Looking forward to see it working in nightmare! |
|
Heroku just bumped the size of |
|
I used |
|
@abcfy2 Man read the log there is a clue. I'll give you a hint |
|
It's a little strange. I find the |
|
Hey folks, I'm going to lock this conversation since there's a ton of good information available and each comment emails approximately 300 people. If you have an issue with getting Nightmare working on Electron please first read through the comments here. If you cannot find an answer, open a new issue. Thanks! |
|
I want to leave an example here in case anyone comes to this thread later and finds it hard to follow. Here's a minimal way to get it running on ubuntu (via Docker): Dockerfile FROM node:8.3.0
RUN apt-get update
# Installing the packages needed to run Nightmare
RUN apt-get install -y \
xvfb \
x11-xkb-utils \
xfonts-100dpi \
xfonts-75dpi \
xfonts-scalable \
xfonts-cyrillic \
x11-apps \
clang \
libdbus-1-dev \
libgtk2.0-dev \
libnotify-dev \
libgnome-keyring-dev \
libgconf2-dev \
libasound2-dev \
libcap-dev \
libcups2-dev \
libxtst-dev \
libxss1 \
libnss3-dev \
gcc-multilib \
g++-multilib
RUN npm install nightmare xvfbindex.js const Nightmare = require('nightmare')
const Xvfb = require('xvfb')
main().catch(console.error)
// main function
async function main() {
const close = await xvfb()
const nightmare = Nightmare()
const [err, title] = await poss(run(nightmare))
if (err) {
// cleanup properly
await nightmare.end()
await close()
throw err
}
console.log(title)
// shut'er down
await nightmare.end()
await close()
}
// run nightmare
//
// put all your nightmare commands in here
async function run(nightmare) {
await nightmare.goto('https://google.com')
const title = await nightmare.title()
return title
}
// xvfb wrapper
function xvfb(options) {
var xvfb = new Xvfb(options)
function close() {
return new Promise((resolve, reject) => {
xvfb.stop(err => (err ? reject(err) : resolve()))
})
}
return new Promise((resolve, reject) => {
xvfb.start(err => (err ? reject(err) : resolve(close)))
})
}
// try/catch helper
async function poss(promise) {
try {
const result = await promise
return [null, result]
} catch (err) {
return [err, null]
}
}Makefile build:
@docker build -t nightmare .
run:
@docker run -it -v $(PWD)/index.js:/index.js nightmare node index.js
run.debug:
@docker run -it -v $(PWD)/index.js:/index.js -e DEBUG=nightmare* nightmare node index.jsUsage make build # build the docker container
make run # run the container (`make run.debug` to see debug logs)Hope this helps! 🎉 |
and others
I've taken the cnn.js example, and my only change is changing the require line from ../nightmare to nightmare, but when I run it I only get...
I don't ever seem to get the title of the page.
I've tried various other examples from Nightmare < 2 from around the web, and I can't get any of them to work either. Nothing ever errors, I just never seem to get any screenshot, pdf or elements pulled off the page in the output.
After 4 hours of bashing my head, I'm not sure what else to try.
The text was updated successfully, but these errors were encountered: