Join GitHub today
GitHub is home to over 50 million developers working together to host and review code, manage projects, and build software together.Sign up
GitHub is where the world builds software
Millions of developers and companies build, ship, and maintain their software on GitHub — the largest and most advanced development platform in the world.
database/sql: prepared statements leak connections with high concurrency #5323
When using database/sql with prepared statements and concurrency higher that MaxIdleConns connections are leaked and recreated until database cannot accept anymore. See https://groups.google.com/d/msg/golang-nuts/-YbB2Qjg41g/Afuaf8nSmGEJ for a related discussion. The problem is that all connections that have ever been used by Stmt are marked as dependent on that Stmt, and that dependency is only removed when Stmt is closed. When connection, that was over MaxIdleConns limit, is returned, it is not actually closed, because it is still used by Stmt. Later, when Stmt tries to find a free connection, is sees that connection is not free and doesn't use it. This causes database/sql to create driver connections over and over again, until underlying database refuses and replies with errors. I think dependency should only be added in Stmt.connStmt and removed in releaseConn callbacks, because prepared statements don't really appear to "use" the connection, merely "cache" it. On the other hand such solution would mean that drivers's Stmt.Close is never called.
Ouch, this is terrible, sorry. I agree it makes any halfway-normal workload fall apart quickly and hold open connections forever. I can reproduce and have a test for it now. I started working on a fix. If I can't this into Go 1.1, I'll try to get it into the most immediate point release.
Labels changed: added packagebug, removed priority-triage.
Owner changed to @bradfitz.
Status changed to Accepted.
Hey, I think I may be seeing this problem. note: I'm a newbie, so maybe I'm doing something wrong! note also: I'm using this through https://github.com/go-sql-driver/mysql but if you look at this code: https://github.com/vimeo/carbon-tagger/blob/master/carbon-tagger.go#L115 which is basically a function that creates some prepared statements and reads structs from a channel, for every struct it reads, it executes a few of the prepared statements. I was trying to find out if I should explicitly run close anywhere but I don't think so, since I want to reuse the prepared statements. also note that they are all inserts, except for one select which only returns one row so I run QueryRow on it. what I notice is that when the code runs, it generates a large amount of new connections to mysql, and when all items are processed, the amount of connections gradually drops back to 0 (!) I have db.SetMaxIdleConns(80) but that doesn't seem to do much, it goes way higher than that. before I hit it, I reach "max fd's reached" and/or "mysql error too many connections"