Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

database/sql: driver unexpectedly closing connections that should be returned to the pool #28495

Closed
rchakra3 opened this issue Oct 30, 2018 · 3 comments

Comments

@rchakra3
Copy link

@rchakra3 rchakra3 commented Oct 30, 2018

Please answer these questions before submitting your issue. Thanks!

What version of Go are you using (go version)?

go 1.11

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

Ubuntu 16.04.5 LTS
GOARCH="amd64"

What did you do?

I have 2 go servers, one of which talks to a SQL Server RDS instance, and another that talks to a managed SQL Server instance in Azure.

I believe there is a slight difference in the way the 2 backends work - RDS has a single port (1433) on which the client authenticates and subsequently establishes the connection. Azure SQL seems to authenticate on port 1433 and then redirect the client to another service that actually handles the connections.

In both cases I've got substantial load running against the servers. At least 500 requests/s, with peaks of about 2k req/s. Each of these requests results in a Select query which returns a single row (that I call .Scan() on) with a primary key lookup - so really short lived connections to SQL. The average time per query is 50-80ms on both, with p95 in the 100-150ms range

I'm using the go database/sql driver with an MS-SQL implementation (Specifically go-mssqldb).

I've set Max Idle connections and Max Open connections to 64. I have NOT set MaxConnLifetime.

The Connection Pool was initialized using:

        db, err := sql.Open("sqlserver", connString)
	if err != nil {
		return nil, err
	}
	db.SetMaxIdleConns(64)
	db.SetMaxOpenConns(64)

What did you expect to see?

64 long running Established connections to the database that are occasionally idle but quickly reused.

What did you see instead?

Generally 64 Established connections to the database, with the number often dropping down to somewhere between 50 and 64. This also results in 200-400 connections to the database in the TIME_WAIT state at any given time.

@agnivade
Copy link
Contributor

@agnivade agnivade commented Oct 31, 2018

The functions set an upper bound to the max no. of connections. It does not say anything about the minimum no. of connections. Maybe this is expected ? @kardianos

@agnivade agnivade changed the title database/sql driver unexpectedly closing connections that should be returned to the pool database/sql: driver unexpectedly closing connections that should be returned to the pool Oct 31, 2018
@kardianos
Copy link
Contributor

@kardianos kardianos commented Oct 31, 2018

@rchakra3 As to why there are sometimes 50 open connections, I would imagine sometimes an error is returned and or something times out and the connection is dropped.

database/sql, unlike other pools, connects to the database lazily and eagerly reuses connection. So that part makes sense.

I'm not sure why you have 200-400 connections in the TIME_WAIT state.

@rchakra3
Copy link
Author

@rchakra3 rchakra3 commented Nov 6, 2018

@kardianos @agnivade thanks for the details!
After digging in further, I think this might be specific to the sql client implementation I'm using. I've created an issue in that repo to see if anyone has any ideas. If not I'll dig in further.

I'm closing this issue until I'm sure the client implementation isn't causing this behavior.

@rchakra3 rchakra3 closed this Nov 6, 2018
@golang golang locked and limited conversation to collaborators Nov 6, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
5 participants
You can’t perform that action at this time.