Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
Keep connection pool errors occurring #1628
Steps to reproduce
Currently I have been using npgsql to populate pgsql records about 120,000 records per hour.
It happens approximately once two weeks so it's hard to reproduce and the npgsql detail log itself is too big to store.
Once npgsql throws connection pool exhausted exceptions, it keeps throwing the exceptions continuously by the time I reset the IIS manually. In the meantime, the API for resetting connection pools of npgsql doesn't solve the symptoms, either.
Further technical details
Npgsql version: 3.2.2
It would be appreciated if you advise me anything related to the issue.
@DJingSeo just so I'm sure I understand:
I have some pgsql databases in Amazon. One of them is used as a global database and the others are for each region like Oregon, N. Virginia and so on. For this reason, a application has four npgsql connection strings and the details are as followings:
The majority of the issue has occurred during 'Populating records to a regional database' but selecting has also rarely suffered from the same issue.
One more thing is that the issue wih selection looks being restored a few mins after it gets the first exhaustion.(I mean it didn't require IIS reset) The issue with insertion, however, doesn't look being restored as I posted first. The issue can be related to the execution time but I'm not sure about the cause.
Feel free to let me know if you need anything from me.
That was a question, not a piece of information :) I just wanted to make sure that after calling
Unfortunately there's not much I can do without some sort of hint or code sample which would allow me to reproduce the issue... A look at the pool code doesn't reveal anything obvious bug that would explain this, and it's the first time someone has complained about it.
Are you sure there's no chance all these exceptions are simply legitimate, i.e. that you're trying to open more connections than allowed by your Max Pool Size parameter? The fact that the issue disappears after a few minutes seems to point that way - if an actual bug in the pool led it to enter a bad state, it's unlikely it would exit that bad state on its own after some time...
I tried to make the pools reviving by calling "Npgsql.NpgsqlConnection.ClearAllPools();" few weeks ago but it kept throwing the exceptions.
I had tried to reproduce this issue for more than 4 months by using Jmeter and other load test tools but I failed. That's why I didn't raise this issue before. :-(
I'm not sure that the exceptions are legitimate. The issue doesn't disappear in most times without resetting IIS.
I can get the detail log of npgsql when it happens not from the beginning of a web application but during the issue is happening in a server. Do you think this log can help you to investigate somehow? If so, I believe I might be able to provide it in few weeks.
I'm not sure what the log would teach us, but it's worth a try. First, if there's some edge case where you're forgetting to properly close connections (i.e. connection leak), the logs could help diagnose that - although that doesn't sit well with the exhausted state disappearing on its own. If we find some exception in the log obviously that would be great, but other than that....
But it's worth a try, especially if you can get logs with loglevel trace. Other than that there's really nothing much I can do.