Cluster doesn't work with Foreman #107

Closed
lackac opened this Issue Jun 10, 2011 · 6 comments

Projects

None yet

3 participants

@lackac
lackac commented Jun 10, 2011

I wanted to use cluster on Heroku which would require it to be run by Foreman.

Minimal setup which reproduces the issue

app.js

var http = require('http');

module.exports = http.createServer(function(request, response) {
  response.writeHead(200, {'Content-Type': 'text/plain'});
  response.end('Hello World\n');
});

web.js

var cluster = require('cluster');

cluster(__dirname + '/app')
  .set("workers", 1)
  .listen(process.env.PORT);

Procfile

web: node web.js

Now, after running foreman start in one terminal and issuing curl http://localhost:5000 in another I get the following error:

$ curl http://localhost:5000
curl: (7) couldn't connect to host

Running web.js directly with node works as does running app.js with foreman (after adding a .listen(process.env.PORT) call).

More details

I added some plugins to see what is going on.

cli reported the process numbers correctly and I could see the processes running with pstree:

\-+= 04860 LacKac -zsh
  \-+= 09634 LacKac ruby: foreman master  
    \-+- 09635 LacKac ruby: foreman web  
      \-+= 09636 LacKac node server.js
        \--- 09637 LacKac /usr/local/Cellar/node/0.4.7/bin/node /Users/LacKac/Code/sandbox/foreman_cluster/server.js

logger created the log files but only master.log had any contents:

[Fri, 10 Jun 2011 13:14:49 GMT] INFO master started
[Fri, 10 Jun 2011 13:14:51 GMT] INFO master started
[Fri, 10 Jun 2011 13:14:51 GMT] INFO spawned worker 0

Why would I want to run cluster on Heroku?

Why, for running more than one process on one dyno of course. See a similar solution with unicorn.

@tj
Member
tj commented Jun 10, 2011

hmm, well it would obviously be nice for things to work on heroku, but I have no interest in adding baked-in support for arbitrary tools like that, should be doable with a plugin though

@JosephHalter

+1 - I did hit exactly the same issue today.

@lackac
lackac commented Jun 13, 2011

@visionmedia, I only thought to bring this up because it might point to a design flaw in cluster. I don't know if I'm right and I haven't looked into the reason of the issue at all, but it might be that something simple prevents cluster from working with foreman. It might not be even foreman specific and fixing it could benefit other setups as well.

I'll try to look into it later this week if I find the time, but any pointers on what could be wrong would be most welcome.

@JosephHalter

I did try to hack something similar to cluster yersterday. But using 4 workers doesn't seem to make a sensible difference in term of req/s I can handle. I benchmark with:

ab -k -t 10 -c 1000 http://evening-robot-961.herokuapp.com/restaurant/4df550c5c3aaaa0100000001

I am always around 2000req/s on heroku with 1 dyno. Hopefully, it's enough for me (this url get a document from mongo and output some of its attributes in json using expressjs and mongoose). Please note that I get exactly the same level of performances in ruby (using sinatra, mongo_mapper and unicorn with 4 workers):

ab -k -t 10 -c 1000 http://cold-mist-128.herokuapp.com/restaurant/4df55a0fab9e270007000001

Both node.js and ruby can be used to build a restful api handling approx 2000req/s per dyno. If you need to handle more traffic you better crank up the number of dynos, I saw linear increase with approx 10000req/s on both ruby and node.js projects when increasing the number of dynos to 5.

@tj
Member
tj commented Jun 13, 2011

@JosephHalter I dont know much about how they set things up, but you should see roughly linear gain if you have a worker per cpu

@lackac it should be fine as long as they are setting the port, maybe it's because PORT is a string so node is trying to use it has a local socket path or something

@lackac
lackac commented Jun 13, 2011

@visionmedia, that was it. .listen(parseInt(process.env.PORT)) solved the issue. Thanks!

@lackac lackac closed this Jun 13, 2011
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment