Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handle max_allowed_packet client-side #40

Closed
bgrainger opened this issue Sep 5, 2016 · 4 comments
Closed

Handle max_allowed_packet client-side #40

bgrainger opened this issue Sep 5, 2016 · 4 comments
Assignees
Milestone

Comments

@bgrainger
Copy link
Member

bgrainger commented Sep 5, 2016

If a packet that is larger than max_allowed_packet (a server-defined variable) is sent, the server will reset the connection and send an error packet with the response Got a packet bigger than 'max_allowed_packet' bytes. However, once this happens the connection is destroyed and the client can't recover.

Additionally, it's a waste of time to send the query to the server when we know it'll be rejected. By running SHOW VARIABLES when the connection is established, we could get the value of max_allowed_packet, and check packets to ensure they don't exceed that length before sending them.

@bgrainger
Copy link
Member Author

bgrainger commented Mar 31, 2017

For efficiency in the typical use case, perhaps we could only SHOW VARIABLES once a SQL statement larger than 4MB (the MySQL Server default) has been prepared in the client?

@caleblloyd
Copy link
Contributor

caleblloyd commented Mar 31, 2017

What about cases where the client has upped it's max_allowed_packet size before sending a large packet, but after we theoretically record max_allowed_packet? We would reject a query client-side that would have succeeded.

mysql> SELECT @@global.max_allowed_packet;
+-----------------------------+
| @@global.max_allowed_packet |
+-----------------------------+
|                     4194304 |
+-----------------------------+
1 row in set (0.00 sec)

mysql> SET @@global.max_allowed_packet = 8388608;
Query OK, 0 rows affected (0.00 sec)

mysql> SELECT @@global.max_allowed_packet;
+-----------------------------+
| @@global.max_allowed_packet |
+-----------------------------+
|                     8388608 |
+-----------------------------+
1 row in set (0.00 sec)

Maybe a better approach is to record number of bytes sent in the packet, and throw an Outer Exception if packet size was > 4MB with "Exception after submitting XXMB Packet, ensure max_allowed_packet is greater than XXMB" -> Inner Exception (original exception)

@bgrainger
Copy link
Member Author

bgrainger commented Mar 31, 2017

MySQL Server does send a final packet with the contents #08S01, Got a packet bigger than 'max_allowed_packet' bytes but then it immediately resets the socket. As a result, our library throws a SocketException ("An existing connection was forcibly closed by the remote host" or "An established connection was aborted by the software in your host machine") when trying to read that packet in:

System.dll!System.Net.Sockets.Socket.Receive(byte[] buffer, int offset, int size, System.Net.Sockets.SocketFlags socketFlags)	Unknown
MySqlConnector.dll!MySql.Data.Protocol.Serialization.SocketByteHandler.ReadBytesAsync(System.ArraySegment<byte> buffer, MySql.Data.Protocol.Serialization.IOBehavior ioBehavior) Line 24	C#
MySqlConnector.dll!MySql.Data.Protocol.Serialization.BufferedByteReader.ReadBytesAsync(MySql.Data.Protocol.Serialization.IByteHandler byteHandler, System.ArraySegment<byte> buffer, int count, MySql.Data.Protocol.Serialization.IOBehavior ioBehavior) Line 34	C#
MySqlConnector.dll!MySql.Data.Protocol.Serialization.BufferedByteReader.ReadBytesAsync(MySql.Data.Protocol.Serialization.IByteHandler byteHandler, int count, MySql.Data.Protocol.Serialization.IOBehavior ioBehavior) Line 29	C#
MySqlConnector.dll!MySql.Data.Protocol.Serialization.ProtocolUtility.ReadPacketAsync(MySql.Data.Protocol.Serialization.BufferedByteReader bufferedByteReader, MySql.Data.Protocol.Serialization.IByteHandler byteHandler, System.Func<int?> getNextSequenceNumber, MySql.Data.Protocol.Serialization.ProtocolErrorBehavior protocolErrorBehavior, MySql.Data.Protocol.Serialization.IOBehavior ioBehavior) Line 13	C#
MySqlConnector.dll!MySql.Data.Protocol.Serialization.ProtocolUtility.ReadPayloadAsync(MySql.Data.Protocol.Serialization.BufferedByteReader bufferedByteReader, MySql.Data.Protocol.Serialization.IByteHandler byteHandler, System.Func<int?> getNextSequenceNumber, System.ArraySegment<byte> previousPayloads, MySql.Data.Protocol.Serialization.ProtocolErrorBehavior protocolErrorBehavior, MySql.Data.Protocol.Serialization.IOBehavior ioBehavior) Line 53	C#
MySqlConnector.dll!MySql.Data.Protocol.Serialization.StandardPayloadHandler.ReadPayloadAsync(MySql.Data.Protocol.Serialization.ProtocolErrorBehavior protocolErrorBehavior, MySql.Data.Protocol.Serialization.IOBehavior ioBehavior) Line 31	C#
MySqlConnector.dll!MySql.Data.Serialization.MySqlSession.TryAsync(System.Func<MySql.Data.Protocol.Serialization.ProtocolErrorBehavior, MySql.Data.Protocol.Serialization.IOBehavior, System.Threading.Tasks.ValueTask<System.ArraySegment<byte>>> func, MySql.Data.Protocol.Serialization.IOBehavior ioBehavior, System.Threading.CancellationToken cancellationToken) Line 471	C#
MySqlConnector.dll!MySql.Data.Serialization.MySqlSession.ReceiveReplyAsync(MySql.Data.Protocol.Serialization.IOBehavior ioBehavior, System.Threading.CancellationToken cancellationToken) Line 187	C#
MySqlConnector.dll!MySql.Data.MySqlClient.Results.ResultSet.ReadResultSetHeaderAsync(MySql.Data.Protocol.Serialization.IOBehavior ioBehavior, System.Threading.CancellationToken cancellationToken) Line 37	C#
MySqlConnector.dll!MySql.Data.MySqlClient.MySqlDataReader.ReadFirstResultSetAsync(MySql.Data.Protocol.Serialization.IOBehavior ioBehavior, System.Threading.CancellationToken cancellationToken) Line 239	C#
MySqlConnector.dll!MySql.Data.MySqlClient.MySqlDataReader.CreateAsync(MySql.Data.MySqlClient.MySqlCommand command, System.Data.CommandBehavior behavior, MySql.Data.Protocol.Serialization.IOBehavior ioBehavior, System.Threading.CancellationToken cancellationToken) Line 227	C#
MySqlConnector.dll!MySql.Data.MySqlClient.CommandExecutors.TextCommandExecutor.ExecuteReaderAsync(string commandText, MySql.Data.MySqlClient.MySqlParameterCollection parameterCollection, System.Data.CommandBehavior behavior, MySql.Data.Protocol.Serialization.IOBehavior ioBehavior, System.Threading.CancellationToken cancellationToken) Line 59	C#
MySqlConnector.dll!MySql.Data.MySqlClient.MySqlCommand.ExecuteReaderAsync(System.Data.CommandBehavior behavior, MySql.Data.Protocol.Serialization.IOBehavior ioBehavior, System.Threading.CancellationToken cancellationToken) Line 143	C#
MySqlConnector.dll!MySql.Data.MySqlClient.MySqlCommand.ExecuteDbDataReader(System.Data.CommandBehavior behavior) Line 116	C#
System.Data.dll!System.Data.Common.DbCommand.ExecuteReader()	Unknown

@bgrainger bgrainger self-assigned this Mar 31, 2017
@bgrainger bgrainger added this to the 1.0 milestone Mar 31, 2017
@bgrainger
Copy link
Member Author

bgrainger commented Apr 1, 2017

Fixed in 0.16.1.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

No branches or pull requests

2 participants