Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Checking that connection was not closed by the server before reusing it from the pool. #1002

Closed

Conversation

damour
Copy link
Contributor

@damour damour commented May 7, 2021

Checking that conn.write() doesn't return an error during query execution is not enough to prevent errors like EOF for closed connections by the server because write() syscall returns success for the socket that received FIN packet (half-closed TCP connection).
EOF error will be returned only after the read() call but in this case, we cannot distinguish half-closed connection(after the previous query) and read timeout for the new query so query retry is not safe and driver propagates the error to the application.
We use a lightweight non-blocking read from the socket that immediately returns the result in driver.ResetSession() method before a connection is reused for another query so a new connection will be taken from the pool if server initiated connection closing and application can work smoothly.

Implementation was taken from go mysql driver.

https://github.blog/2020-05-20-three-bugs-in-the-go-mysql-driver/
golang/go#15735 (comment)

…it from the pool.

Checking that conn.write() doesn't return an error during query execution is not enough to prevent errors like EOF for closed connections by the server because write() syscall returns success for the socket that received FIN packet (half-closed TCP connection).
EOF error will  be returned only after the read() call but in this case, we cannot distinguish half-closed connection(after the previous query) and read timeout for the new query so query retry is not safe and driver propagates the error to the application.
We use a lightweight non-blocking read from the socket that immediately returns the result in driver.ResetSession() method before a connection is reused for another query so a new connection will be taken from the pool if server initiated connection closing and application can work smoothly.
@jackc
Copy link
Owner

jackc commented May 8, 2021

Are MPL and MIT compatible? I'm concerned about what that might do to the license.

Beyond that, I'm somewhat dubious about this solution. Though I guess if it has been in use for a few years that's a positive sign.

Here's the things that concern me:

  1. It is incompatible with TLS connections. That's a pretty big deal to me. I suppose people still run with unencrypted sometimes, but I thought that best practice nowadays is to encrypt even on private networks.
  2. No Windows support -- not a deal breaker -- but I'd rather support it than not.
  3. This would add overhead to Unix domain socket connections for no benefit.
  4. The test seems to flicker on CI - though I'm guessing that is fixable.

What if a hook was added to allow arbitrary logic to be inserted in ResetSession? That would allow this behavior to be inserted per application.

@damour
Copy link
Contributor Author

damour commented May 13, 2021

@jackc Thank you for the extended answer!

I prepared another PR which provides the ability to define any custom function in ResetSession PTAL.

I'm closing this one.

Just to clarify your point about TLS connection:

It is incompatible with TLS connections.

Could you please briefly explain why this solution breaks TLS connections?
We don't use TLS unfortunately and I don't familiar with implementation details. At the first glance connCheck() should work with any socket, doesn't matter the protocol.

@damour damour closed this May 13, 2021
@jackc
Copy link
Owner

jackc commented May 14, 2021

Could you please briefly explain why this solution breaks TLS connections?

tls.Conn does not implement syscall.Conn.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants