New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cluster doesn't work with Foreman #107

Closed
lackac opened this Issue Jun 10, 2011 · 6 comments

Comments

Projects
None yet
3 participants
@lackac

lackac commented Jun 10, 2011

I wanted to use cluster on Heroku which would require it to be run by Foreman.

Minimal setup which reproduces the issue

app.js

var http = require('http');

module.exports = http.createServer(function(request, response) {
  response.writeHead(200, {'Content-Type': 'text/plain'});
  response.end('Hello World\n');
});

web.js

var cluster = require('cluster');

cluster(__dirname + '/app')
  .set("workers", 1)
  .listen(process.env.PORT);

Procfile

web: node web.js

Now, after running foreman start in one terminal and issuing curl http://localhost:5000 in another I get the following error:

$ curl http://localhost:5000
curl: (7) couldn't connect to host

Running web.js directly with node works as does running app.js with foreman (after adding a .listen(process.env.PORT) call).

More details

I added some plugins to see what is going on.

cli reported the process numbers correctly and I could see the processes running with pstree:

\-+= 04860 LacKac -zsh
  \-+= 09634 LacKac ruby: foreman master  
    \-+- 09635 LacKac ruby: foreman web  
      \-+= 09636 LacKac node server.js
        \--- 09637 LacKac /usr/local/Cellar/node/0.4.7/bin/node /Users/LacKac/Code/sandbox/foreman_cluster/server.js

logger created the log files but only master.log had any contents:

[Fri, 10 Jun 2011 13:14:49 GMT] INFO master started
[Fri, 10 Jun 2011 13:14:51 GMT] INFO master started
[Fri, 10 Jun 2011 13:14:51 GMT] INFO spawned worker 0

Why would I want to run cluster on Heroku?

Why, for running more than one process on one dyno of course. See a similar solution with unicorn.

@tj

This comment has been minimized.

Member

tj commented Jun 10, 2011

hmm, well it would obviously be nice for things to work on heroku, but I have no interest in adding baked-in support for arbitrary tools like that, should be doable with a plugin though

@JosephHalter

This comment has been minimized.

JosephHalter commented Jun 11, 2011

+1 - I did hit exactly the same issue today.

@lackac

This comment has been minimized.

lackac commented Jun 13, 2011

@visionmedia, I only thought to bring this up because it might point to a design flaw in cluster. I don't know if I'm right and I haven't looked into the reason of the issue at all, but it might be that something simple prevents cluster from working with foreman. It might not be even foreman specific and fixing it could benefit other setups as well.

I'll try to look into it later this week if I find the time, but any pointers on what could be wrong would be most welcome.

@JosephHalter

This comment has been minimized.

JosephHalter commented Jun 13, 2011

I did try to hack something similar to cluster yersterday. But using 4 workers doesn't seem to make a sensible difference in term of req/s I can handle. I benchmark with:

ab -k -t 10 -c 1000 http://evening-robot-961.herokuapp.com/restaurant/4df550c5c3aaaa0100000001

I am always around 2000req/s on heroku with 1 dyno. Hopefully, it's enough for me (this url get a document from mongo and output some of its attributes in json using expressjs and mongoose). Please note that I get exactly the same level of performances in ruby (using sinatra, mongo_mapper and unicorn with 4 workers):

ab -k -t 10 -c 1000 http://cold-mist-128.herokuapp.com/restaurant/4df55a0fab9e270007000001

Both node.js and ruby can be used to build a restful api handling approx 2000req/s per dyno. If you need to handle more traffic you better crank up the number of dynos, I saw linear increase with approx 10000req/s on both ruby and node.js projects when increasing the number of dynos to 5.

@tj

This comment has been minimized.

Member

tj commented Jun 13, 2011

@JosephHalter I dont know much about how they set things up, but you should see roughly linear gain if you have a worker per cpu

@lackac it should be fine as long as they are setting the port, maybe it's because PORT is a string so node is trying to use it has a local socket path or something

@lackac

This comment has been minimized.

lackac commented Jun 13, 2011

@visionmedia, that was it. .listen(parseInt(process.env.PORT)) solved the issue. Thanks!

@lackac lackac closed this Jun 13, 2011

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment