EndOfStreamException in MySqlSession.ConnectAsync

[bgrainger]

@GomesNayagam writes:
can you reopen this issue @bgrainger
i am getting the same exception "Attempted to read past the end of the stream" now with 0.28.0 below are the call stack from dot net core library

"StackTrace":"   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at MySql.Data.Protocol.Serialization.ProtocolUtility.DoReadPayloadAsync(BufferedByteReader bufferedByteReader, IByteHandler byteHandler, Func`1 getNextSequenceNumber, ArraySegmentHolder`1 previousPayloads, ProtocolErrorBehavior protocolErrorBehavior, IOBehavior ioBehavior)
   at MySql.Data.Protocol.Serialization.StandardPayloadHandler.ReadPayloadAsync(ArraySegmentHolder`1 cache, ProtocolErrorBehavior protocolErrorBehavior, IOBehavior ioBehavior)
   at MySql.Data.Serialization.MySqlSession.ReceiveReplyAsync(IOBehavior ioBehavior, CancellationToken cancellationToken)
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at MySql.Data.Serialization.MySqlSession.TryAsyncContinuation(Task`1 task)
   at System.Threading.Tasks.ContinuationResultTaskFromResultTask`2.InnerInvoke()
   at System.Threading.Tasks.Task.Execute()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.ConfiguredValueTaskAwaitable`1.ConfiguredValueTaskAwaiter.GetResult()
   at MySql.Data.Serialization.MySqlSession.<ConnectAsync>d__49.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at MySql.Data.MySqlClient.ConnectionPool.<GetSessionAsync>d__0.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at MySql.Data.MySqlClient.MySqlConnection.<CreateSessionAsync>d__61.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at MySql.Data.MySqlClient.MySqlConnection.<OpenAsync>d__14.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at MySql.Data.MySqlClient.MySqlConnection.Open()
   at Dapper.SqlMapper.<QueryImpl>d__124`1.MoveNext()
   at System.Collections.Generic.List`1..ctor(IEnumerable`1 collection)
   at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
   at Dapper.SqlMapper.Query[T](IDbConnection cnn, String sql, Object param, IDbTransaction transaction, Boolean buffered, Nullable`1 commandTimeout, Nullable`1 commandType)
   at Dapper.FastCrud.SqlStatements.GenericSqlStatements`1.BatchSelect(IDbConnection connection, AggregatedSqlStatementOptions`1 statementOptions)
   at Warp.Foundation.Application.Explorer.Dapper.Explorer`1.Query(Action`1 Query)
   at 

(split from #124)

[bgrainger]

What MySQL Server software and version are you connecting to? On what OS?
What is your connection string (with password redacted)?

[GomesNayagam]

Environment: Ubuntu 16.04
DotNET core: 1.1.1
Mysql connector : MySqlConnector.0.28.0 (https://github.com/mysql-net/MySqlConnector)
Mysql version: 5.7

[bgrainger]

Judging from the exception (specifically, ReceiveReplyAsync), an incomplete packet is being received after sending the handshake response. If this is reproducible, are you able to get a packet capture with tcpdump, Wireshark, or similar? You can email that to the address on my profile.

[GomesNayagam]

I saw that too, header packet(4 bytes) is corrupted.

This issue happening on production since the load is high, so taking packet data is non trivial task for me. i will try to do some load test in local environment and see if i can reproduce and setup wireshark, mean time if you get any hint let me know.

[bgrainger]

Do you have any kind of proxy (ProxySQL, HAProxy, etc.) between your .NET Core application and MySQL Server? Are you running MySQL Server 5.7.20 or an earlier release?

Are you using SslMode=Preferred (the default) or higher? If so, does changing to SslMode=None to force a plaintext connection avoid the problem?

[GomesNayagam]

we use haproxy to switch between prmiary and seconday master(2 master 1 slave setup) "connectionString": "Server=xxxxx;Uid=xxxx;Pwd=xxxxx;Database=xxxx;Maximum Pool Size=40;Connect Timeout=120;Connection Lifetime=240;" Would you confirm you are throwing the exception of type System.IO.EndOfStreamException, i guess right it is of generic exception with error message.

[bgrainger]

Suggested troubleshooting steps:

• disable TLS (SslMode=None) if possible
• connect directly to MySQL Server instead of HAProxy (may require Implement server load balancing #391)

[GomesNayagam]

sslmode=none completely gave weird response for me, so i am not using it. production is not accepting without haproxy.

[bgrainger]

From all the information you've provided, it seems like this is a server issue (responding with invalid data).

Would the most helpful thing be for MySqlConnector to wrap the EndOfStreamException in an MySqlException so you can catch it and handle it like any other connection failure (timeout, etc.)?

[GomesNayagam]

thanks, that will be help full. Appreciate your time spending on this issue.

On Fri, Nov 10, 2017 at 2:41 PM Bradley Grainger ***@***.***> wrote: From all the information you've provided, it seems like this is a server issue (responding with invalid data). Would the most helpful thing be for MySqlConnector to wrap the EndOfStreamException in an MySqlException so you can catch it and handle it like any other connection failure (timeout, etc.)? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#388 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABDBllunhrWZjtFUf7IjGXGKVBgetsUNks5s1NEvgaJpZM4QRcsI> .

-- Thanks, Gomathinayagam (Gomes) Ph : (847) 371 1464 Linkedin: in.linkedin.com/in/gomesnayagam/ Blog : www.zerothoughts.in

[bgrainger]

Improved in 0.31.0.