setting poolSize to 0 does not disable it #319

Closed
davecramer opened this Issue Apr 1, 2013 · 15 comments

Projects

None yet

5 participants

@davecramer

Sets it to the default of 10

@brianc
Owner
brianc commented Apr 2, 2013

That might be a bug with node generic-pool. Why do you want to set the pool size to 0? What do you expect it to do?

You can set the pool size to 1, effectively making it a single file queue of all queries. Or just instantiate clients directly.

@davecramer

Brian,

As per your docs, I expect it to not use pooling. We have an app which we
want to control the pooling at a much more granular level.
For instance user A gets 3 connections, user B gets 10.

pgbouncer works fine for this and we have other apps which use the pool as
well.

Yes I am aware that instantiating clients directly would work, but it is
much cleaner if it is done inside pg.

Thanks,

Dave Cramer

On Mon, Apr 1, 2013 at 8:14 PM, Brian C notifications@github.com wrote:

That might be a bug with node generic-pool. Why do you want to set the
pool size to 0? What do you expect it to do?

You can set the pool size to 1, effectively making it a single file queue
of all queries. Or just instantiate clients directly.


Reply to this email directly or view it on GitHubhttps://github.com/brianc/node-postgres/issues/319#issuecomment-15745094
.

@strk
Contributor
strk commented Jul 3, 2013

subscribe, I'm looking for ways to disable pooling too

@strk
Contributor
strk commented Jul 3, 2013

Note that the reason why I want to disable pooling is because I'm willing to use an external pooler, for its better handling of "session reset". ref #391

@davecramer

Same here. Plus much better granularity, and control. pg_bouncer has the
ability to set limits per db, per user, etc.

Dave Cramer

On Wed, Jul 3, 2013 at 11:11 AM, strk notifications@github.com wrote:

Note that the reason why I want to disable pooling is because I'm willing
to use an external pooler, for its better handling of "session reset". ref
#391 #391


Reply to this email directly or view it on GitHubhttps://github.com/brianc/node-postgres/issues/319#issuecomment-20421925
.

@brianc
Owner
brianc commented Jul 3, 2013

You can completely bypass the pool & build clients with their constructor too. This "disables" pooling by not using it. I still think the 0 issue is a bug, but just curious, is there a reason you don't just create a client instance?

var pg = require('pg');
var client = new pg.Client(/* same connection params as are passed to pg.connect */);
client.connect(function(err) {
  //etc....
});
@strk
Contributor
strk commented Jul 4, 2013

No reason, I just hoped to leave the decision between enabling
or not the pool to a simple configuration variable without
touching existing code, but it didn't work as documented.

@davecramer

Ya, it's kind of foreign to have to change the way you call things to get
pooling or not.

Ideally you would be able to just configure it or at worst
require("pgpool") or ("pg")

Dave Cramer

On Thu, Jul 4, 2013 at 2:41 AM, strk notifications@github.com wrote:

No reason, I just hoped to leave the decision between enabling
or not the pool to a simple configuration variable without
touching existing code, but it didn't work as documented.


Reply to this email directly or view it on GitHubhttps://github.com/brianc/node-postgres/issues/319#issuecomment-20461383
.

@brianc
Owner
brianc commented Jul 5, 2013

Yeah makes total sense. Just as a heads up...there are a few things to keep in mind when you don't use pooling.

  1. Postgres supports far fewer connections than your web server, so you can get connection refused errors if you are under a lot of load
  2. There is substantial overhead in establishing a new connection so this is going to be way slower than using a pool of pre-connected clients.

I figure you know those things already, just wanted to make sure.

@davecramer

Yes, thanks. The goal would be to replace pg's pooling with pgbouncer or
the like.

As I said it has considerable flexibility. FWIW, some people actually use
two pgbouncers in serial. One per webserver connecting to another on the db
server for considerable number of pooled connections.

Dave Cramer

On Fri, Jul 5, 2013 at 1:09 PM, Brian C notifications@github.com wrote:

Yeah makes total sense. Just as a heads up...there are a few things to
keep in mind when you don't use pooling.

  1. Postgres supports far fewer connections than your web server, so
    you can get connection refused errors if you are under a lot of load
  2. There is substantial overhead in establishing a new connection so
    this is going to be way slower than using a pool of pre-connected clients.

I figure you know those things already, just wanted to make sure.


Reply to this email directly or view it on GitHubhttps://github.com/brianc/node-postgres/issues/319#issuecomment-20529344
.

@tanqhnguyen

Hi, I also use pgbouncer for pooling connections to postgresql. I am using pg module version 2.6.2. I am still unable to completely disable the pool setting. I have to work around by using this piece of code

module.exports = {
  createClient: function(callback) {
    var client = new Client(connectionString);
    client.connect(function(error){
      callback(error, client, function(){
        client.end();
      });
    });
  }
}

Since my code has already called done(), I simply put a manually call to client.end(). Are there any recommended ways to skip the pool?

@brianc
Owner
brianc commented Sep 26, 2013

Sorry I haven't gotten around to fixing this one yet. The recommended way to skip the pool is to instantiate a new client as you are doing. I don't think it's wise to connect & disconnect a client on every query, just connect one when your app starts and leave it open. It should be quite easy to add reconnection logic for if/when you lose a connection to your backend db too. The time it takes to establish a new connection is orders of magnitude longer than the time it takes to issue a query.

@tanqhnguyen

I have just put this into production, after 5 minutes, the pgbouncer pool is full with idle connections from the nodejs application.

Setting poolSize to 0 (or 1) makes it even worse because the whole application just hangs

To partially solve this I have to fallback to 0.10.2 which is mega old. I will try to figure out the more efficient way to do this

@yaronn
yaronn commented May 9, 2014

@laoshanlung did you ever figure that out? isn't creating clients with "new client()" and calling done() immediately on result enough?

@brianc
Owner
brianc commented Jun 21, 2016

The best way to disable pooling is not to use it. Ive decided againt supporting poolSize:0 and will remove docs on it soon

@brianc brianc closed this Jun 21, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment