maxSockets does not limit the number of socket connections

[anklos]

Hi folks,

I am trying to use pool({maxSockets: 100}) to increase the default maxSockets number. I find that once I set pool in the option, no matter what value you give to maxSocket, it seems always enable maxSockets to be Infinity.

My local test:

## index.coffee
restify = require 'restify'
client  = require './client'

server = restify.createServer()

server.get '/test', client.send

server.listen 8080, () -> console.log 'server started'

## client.coffee

request = require 'request'

exports.send = (req, res, next) ->
  options =
    url: "https://google.com.au"
    strictSSL: true
    secureProtocol: true
    pool:
       maxSockets: 5

  request options, (err, response, body) ->
    res.send 200, body

so if you npm start and go to localhost:8080/test, you would send a get request to https://google.com.au

Now I use the performance tool wrk to run this test: wrk -t12 -c3000 -d30s http://127.0.0.1:8080/test, which initiates 12 threads to create 3000 connections and last 30 seconds.

Next get the ip of google.com.au by host google.com.au and I get:

google.com.au has address 74.125.237.111
google.com.au has address 74.125.237.127
google.com.au has address 74.125.237.120
google.com.au has address 74.125.237.119

Then in the console I do watch "netstat -an | grep 74.125 | wc -l

From the output I can see there are always hundreds of socket connections opening no matter what value I give maxSockets. But If I don't set pool : { maxSockets: xxx} in the options, I will only see around 10 opening sockets from console.

Is my way of using maxSockets correct? Do I miss something here?

Thanks

[FredKSchott]

Does a new options object get created for every call? If so, your pool might not be shared between requests, causing each 1 request to have a maxSockets of 5 (which in effect, means there is no limit).

You can pass in the pool to request.defaults() to make sure that each request shares the same pool object, instead of creating a new one for each call.

The documentation around how pool works could probably be improved as well.

[anklos]

@FredKSchott Good point, it uses a new option for each call. I thought request object is cached when the first time it is loaded, so it should not matter any pool sharing by passing a new options object?

I tried to use request.defaults() by doing this:

request = require 'request'

request.defaults
   pool:
     maxSockets: 100

exports.send = (req, res, next) ->
  options =
    url: "https://google.com.au"
    strictSSL: true
    secureProtocol: true

  request options, (err, response, body) ->
    res.send 200, body

However I can only see around 10 rather than 100 socket connections during the performance test.

[FredKSchott]

see this comment: #1150 (comment)

request.defaults() doesn't change the actual request module, it returns a new, wrapped module for you to use.

[anklos]

Great, it works, thanks a lot!