OpenAsync throws "System.Net.Sockets.SocketException: Broken pipe"

[alanprot]

Sometimes i'm receiving SocketException when calling _con.OpenAsync(cancellationToken).

Follow the stack:

MySql.Data.MySqlClient.MySqlException: Unable to connect to any of the specified MySQL hosts. ---> System.Net.Sockets.SocketException: Broken pipe
   at MySql.Data.SocketAwaitable.GetResult()
   at MySql.Data.Protocol.Serialization.SocketByteHandler.<DoWriteBytesAsync>d__4.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Threading.Tasks.ValueTask`1.get_Result()
   at MySql.Data.ValueTaskExtensions.ContinueWith[T,TResult](ValueTask`1 valueTask, Func`2 continuation)
   at MySql.Data.Protocol.Serialization.ProtocolUtility.WritePacketAsync(IByteHandler byteHandler, Int32 sequenceNumber, ArraySegment`1 contents, IOBehavior ioBehavior)
   at MySql.Data.Protocol.Serialization.ProtocolUtility.WritePayloadAsync(IByteHandler byteHandler, Func`1 getNextSequenceNumber, ArraySegment`1 payload, IOBehavior ioBehavior)
   at MySql.Data.Serialization.MySqlSession.TryAsync[TArg](Func`3 func, TArg arg, IOBehavior ioBehavior, CancellationToken cancellationToken)
   at MySql.Data.Serialization.MySqlSession.SendAsync(PayloadData payload, IOBehavior ioBehavior, CancellationToken cancellationToken)
   at MySql.Data.Serialization.MySqlSession.<TryPingAsync>d__24.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at MySql.Data.MySqlClient.ConnectionPool.<GetSessionAsync>d__0.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at MySql.Data.MySqlClient.MySqlConnection.<CreateSessionAsync>d__57.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at MySql.Data.MySqlClient.MySqlConnection.<OpenAsync>d__11.MoveNext()
   --- End of inner exception stack trace ---
   at MySql.Data.MySqlClient.MySqlConnection.<OpenAsync>d__11.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)

Maybe await SendAsync(PingPayload.Create(), ioBehavior, cancellationToken).ConfigureAwait(false); should be inside try catch to?

https://github.com/mysql-net/MySqlConnector/blob/master/src/MySqlConnector/Serialization/MySqlSession.cs#L141

[caleblloyd]

@alanprot I think you're right about await SendAsync(PingPayload.Create(), ioBehavior, cancellationToken).ConfigureAwait(false); needing to be in the try block.

Are your connections to your MySQL Server NAT'd? I'm trying to write a test case for this, but can't figure out a good way to emulate a NAT timeout.

[alanprot]

Why are you thinking it is caused by a NAT timeout? This is happening in my dev environment and I think it occurs when the server spends some time without serving any requests (idle)...

My MySql server is not behind a NAT.

[caleblloyd]

Broken pipe typically occurs either when you either unplug a network cable or encounter a NAT timeout. This happens because the server has closed the connection and sent a TCP FIN packet, but the client never got the FIN packet.

Since the client still thinks the connection is open, it continues the TCP conversation with the server from where it left off. The server has closed the connection already though, so it sends a RST packet. This causes a broken pipe exception.

It's most likely NAT timeout or your network is going away and then coming back.

[alanprot]

Understood.

But the applications and mysql servers are inside the same network (AWS VPC) using only internal ips.
Maybe is something with the network then.

[bgrainger]

Please try 0.9.1 to confirm whether this is fixed for you.

[alanprot]

Ok ... I'll do it now and let you know if it happens again.

Thanks!
Obrigado

[naughtyGitCat]

it happens again, during the last several versions

System.Net.Sockets.SocketException (32): Broken pipe
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.CreateException(SocketError error, Boolean forAsyncThrow)
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.SendAsync(Socket socket, CancellationToken cancellationToken)
   at System.Net.Sockets.Socket.SendAsync(ReadOnlyMemory`1 buffer, SocketFlags socketFlags, CancellationToken cancellationToken)
   at MySqlConnector.Protocol.Serialization.SocketByteHandler.DoWriteBytesAsync(ReadOnlyMemory`1 data)
   at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine)
   at MySqlConnector.Protocol.Serialization.SocketByteHandler.DoWriteBytesAsync(ReadOnlyMemory`1 data)
   at MySqlConnector.Protocol.Serialization.SocketByteHandler.WriteBytesAsync(ReadOnlyMemory`1 data, IOBehavior ioBehavior)
   at MySqlConnector.Protocol.Serialization.ProtocolUtility.WritePacketAsync(IByteHandler byteHandler, Int32 sequenceNumber, ReadOnlyMemory`1 contents, IOBehavior ioBehavior)
   at MySqlConnector.Protocol.Serialization.ProtocolUtility.WritePayloadAsync(IByteHandler byteHandler, Func`1 getNextSequenceNumber, ReadOnlyMemory`1 payload, IOBehavior ioBehavior)
   at MySqlConnector.Core.ServerSession.SendReplyAsync(PayloadData payload, IOBehavior ioBehavior, CancellationToken cancellationToken)
   at MySqlConnector.Core.CommandExecutor.ExecuteReaderAsync(IReadOnlyList`1 commands, ICommandPayloadCreator payloadCreator, CommandBehavior behavior, Activity activity, IOBehavior ioBehavior, CancellationToken cancellationToken)
   at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine)
   at MySqlConnector.Core.CommandExecutor.ExecuteReaderAsync(IReadOnlyList`1 commands, ICommandPayloadCreator payloadCreator, CommandBehavior behavior, Activity activity, IOBehavior ioBehavior, CancellationToken cancellationToken)
   at MySqlConnector.MySqlCommand.ExecuteReaderNoResetTimeoutAsync(CommandBehavior behavior, IOBehavior ioBehavior, CancellationToken cancellationToken)
   at MySqlConnector.MySqlCommand.ExecuteReaderAsync(CommandBehavior behavior, IOBehavior ioBehavior, CancellationToken cancellationToken)
   at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine)
   at MySqlConnector.MySqlCommand.ExecuteReaderAsync(CommandBehavior behavior, IOBehavior ioBehavior, CancellationToken cancellationToken)
   at MySqlConnector.MySqlCommand.ExecuteDbDataReaderAsync(CommandBehavior behavior, CancellationToken cancellationToken)
   at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine)
   at MySqlConnector.MySqlCommand.ExecuteDbDataReaderAsync(CommandBehavior behavior, CancellationToken cancellationToken)
   at NPoco.DatabaseType.ExecuteReaderAsync(Database database, DbCommand cmd)
   at NPoco.Database.ExecutionHookAsync[T](Func`1 action)
   at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine)
   at NPoco.Database.ExecutionHookAsync[T](Func`1 action)
   at NPoco.Database.ExecuteReaderHelperAsync(DbCommand cmd)
   at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine)
   at NPoco.Database.ExecuteReaderHelperAsync(DbCommand cmd)
   at NPoco.Database.ExecuteDataReader(DbCommand cmd, Boolean sync)
   at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine)
   at NPoco.Database.ExecuteDataReader(DbCommand cmd, Boolean sync)
   at NPoco.Database.QueryAsync[T](T instance, Expression`1 listExpression, Func`2 idFunc, Sql Sql, PocoData pocoData)+MoveNext()
   at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine)
   at NPoco.Database.QueryAsync[T](T instance, Expression`1 listExpression, Func`2 idFunc, Sql Sql, PocoData pocoData)+System.Collections.Generic.IAsyncEnumerator<T>.MoveNextAsync()
   at System.Runtime.CompilerServices.ConfiguredCancelableAsyncEnumerable`1.Enumerator.MoveNextAsync()
   at System.Linq.AsyncEnumerable.<ToListAsync>g__Core|620_0[TSource](IAsyncEnumerable`1 source, CancellationToken cancellationToken)
   at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine)
   at System.Linq.AsyncEnumerable.<ToListAsync>g__Core|620_0[TSource](IAsyncEnumerable`1 source, CancellationToken cancellationToken)
   at System.Linq.AsyncEnumerable.ToListAsync[TSource](IAsyncEnumerable`1 source, CancellationToken cancellationToken)
   at NPoco.Database.FetchAsync[T](String sql, Object[] args)
   at DBAAlertBroker.Services.BusinessService.GetRelatedServicesAsync(String ip, Int32 port, String dbName)
   at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine)
   at DBAAlertBroker.Services.BusinessService.GetRelatedServicesAsync(String ip, Int32 port, String dbName)
   at DBAAlertBroker.Services.BusinessService.OperateAsync(AlertObj alertObj)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.AsyncStateMachineBox`1.MoveNext(Thread threadPoolThread)
   at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(IAsyncStateMachineBox box, Boolean allowInlining)
   at System.Threading.Tasks.Task.RunContinuations(Object continuationObject)
   at System.Threading.Tasks.Task`1.TrySetResult(TResult result)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.SetExistingTaskResult(Task`1 task, TResult result)
   at DBAInternalCommon.Services.DBAMetaService.GetProcessInfoAsync(String url)

[naughtyGitCat]

we do not use nat, but physical host with static ip , the latest version does not fix too

mysql conn string is

    "MySQL": {
      "ReadWriteURI": "server=XXXXXX;port=3306;user=XXXXX;password=XXXXX;database=XXXXX",
      "ReadOnlyURI": "server=YYYYY;port=3306;user=YYYY;password=YYYY;database=YYYY;DefaultCommandTimeout=188;SslMode=None;MaximumPoolsize=1588;MinimumPoolSize=8;ConnectionReset=false;ConnectionIdlePingTime=2;ConnectionIdleTimeout=88"
    },

@caleblloyd