Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OpenAsync throws "System.Net.Sockets.SocketException: Broken pipe" #150

Closed
alanprot opened this issue Dec 15, 2016 · 8 comments · Fixed by #151
Closed

OpenAsync throws "System.Net.Sockets.SocketException: Broken pipe" #150

alanprot opened this issue Dec 15, 2016 · 8 comments · Fixed by #151

Comments

@alanprot
Copy link

@alanprot alanprot commented Dec 15, 2016

Sometimes i'm receiving SocketException when calling _con.OpenAsync(cancellationToken).

Follow the stack:

MySql.Data.MySqlClient.MySqlException: Unable to connect to any of the specified MySQL hosts. ---> System.Net.Sockets.SocketException: Broken pipe
   at MySql.Data.SocketAwaitable.GetResult()
   at MySql.Data.Protocol.Serialization.SocketByteHandler.<DoWriteBytesAsync>d__4.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Threading.Tasks.ValueTask`1.get_Result()
   at MySql.Data.ValueTaskExtensions.ContinueWith[T,TResult](ValueTask`1 valueTask, Func`2 continuation)
   at MySql.Data.Protocol.Serialization.ProtocolUtility.WritePacketAsync(IByteHandler byteHandler, Int32 sequenceNumber, ArraySegment`1 contents, IOBehavior ioBehavior)
   at MySql.Data.Protocol.Serialization.ProtocolUtility.WritePayloadAsync(IByteHandler byteHandler, Func`1 getNextSequenceNumber, ArraySegment`1 payload, IOBehavior ioBehavior)
   at MySql.Data.Serialization.MySqlSession.TryAsync[TArg](Func`3 func, TArg arg, IOBehavior ioBehavior, CancellationToken cancellationToken)
   at MySql.Data.Serialization.MySqlSession.SendAsync(PayloadData payload, IOBehavior ioBehavior, CancellationToken cancellationToken)
   at MySql.Data.Serialization.MySqlSession.<TryPingAsync>d__24.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at MySql.Data.MySqlClient.ConnectionPool.<GetSessionAsync>d__0.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at MySql.Data.MySqlClient.MySqlConnection.<CreateSessionAsync>d__57.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at MySql.Data.MySqlClient.MySqlConnection.<OpenAsync>d__11.MoveNext()
   --- End of inner exception stack trace ---
   at MySql.Data.MySqlClient.MySqlConnection.<OpenAsync>d__11.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)

Maybe await SendAsync(PingPayload.Create(), ioBehavior, cancellationToken).ConfigureAwait(false); should be inside try catch to?

https://github.com/mysql-net/MySqlConnector/blob/master/src/MySqlConnector/Serialization/MySqlSession.cs#L141

@caleblloyd
Copy link
Contributor

@caleblloyd caleblloyd commented Dec 15, 2016

@alanprot I think you're right about await SendAsync(PingPayload.Create(), ioBehavior, cancellationToken).ConfigureAwait(false); needing to be in the try block.

Are your connections to your MySQL Server NAT'd? I'm trying to write a test case for this, but can't figure out a good way to emulate a NAT timeout.

@alanprot
Copy link
Author

@alanprot alanprot commented Dec 15, 2016

Why are you thinking it is caused by a NAT timeout? This is happening in my dev environment and I think it occurs when the server spends some time without serving any requests (idle)...

My MySql server is not behind a NAT.

@caleblloyd
Copy link
Contributor

@caleblloyd caleblloyd commented Dec 15, 2016

Broken pipe typically occurs either when you either unplug a network cable or encounter a NAT timeout. This happens because the server has closed the connection and sent a TCP FIN packet, but the client never got the FIN packet.

Since the client still thinks the connection is open, it continues the TCP conversation with the server from where it left off. The server has closed the connection already though, so it sends a RST packet. This causes a broken pipe exception.

It's most likely NAT timeout or your network is going away and then coming back.

caleblloyd added a commit to caleblloyd/MySqlConnector that referenced this issue Dec 15, 2016
@alanprot
Copy link
Author

@alanprot alanprot commented Dec 15, 2016

Understood.

But the applications and mysql servers are inside the same network (AWS VPC) using only internal ips.
Maybe is something with the network then.

caleblloyd added a commit to caleblloyd/MySqlConnector that referenced this issue Dec 15, 2016
caleblloyd added a commit to caleblloyd/MySqlConnector that referenced this issue Dec 15, 2016
@bgrainger
Copy link
Member

@bgrainger bgrainger commented Dec 16, 2016

Please try 0.9.1 to confirm whether this is fixed for you.

@alanprot
Copy link
Author

@alanprot alanprot commented Dec 16, 2016

Ok ... I'll do it now and let you know if it happens again.

Thanks!
Obrigado

@naughtyGitCat
Copy link

@naughtyGitCat naughtyGitCat commented Mar 8, 2022

it happens again, during the last several versions

System.Net.Sockets.SocketException (32): Broken pipe
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.CreateException(SocketError error, Boolean forAsyncThrow)
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.SendAsync(Socket socket, CancellationToken cancellationToken)
   at System.Net.Sockets.Socket.SendAsync(ReadOnlyMemory`1 buffer, SocketFlags socketFlags, CancellationToken cancellationToken)
   at MySqlConnector.Protocol.Serialization.SocketByteHandler.DoWriteBytesAsync(ReadOnlyMemory`1 data)
   at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine)
   at MySqlConnector.Protocol.Serialization.SocketByteHandler.DoWriteBytesAsync(ReadOnlyMemory`1 data)
   at MySqlConnector.Protocol.Serialization.SocketByteHandler.WriteBytesAsync(ReadOnlyMemory`1 data, IOBehavior ioBehavior)
   at MySqlConnector.Protocol.Serialization.ProtocolUtility.WritePacketAsync(IByteHandler byteHandler, Int32 sequenceNumber, ReadOnlyMemory`1 contents, IOBehavior ioBehavior)
   at MySqlConnector.Protocol.Serialization.ProtocolUtility.WritePayloadAsync(IByteHandler byteHandler, Func`1 getNextSequenceNumber, ReadOnlyMemory`1 payload, IOBehavior ioBehavior)
   at MySqlConnector.Core.ServerSession.SendReplyAsync(PayloadData payload, IOBehavior ioBehavior, CancellationToken cancellationToken)
   at MySqlConnector.Core.CommandExecutor.ExecuteReaderAsync(IReadOnlyList`1 commands, ICommandPayloadCreator payloadCreator, CommandBehavior behavior, Activity activity, IOBehavior ioBehavior, CancellationToken cancellationToken)
   at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine)
   at MySqlConnector.Core.CommandExecutor.ExecuteReaderAsync(IReadOnlyList`1 commands, ICommandPayloadCreator payloadCreator, CommandBehavior behavior, Activity activity, IOBehavior ioBehavior, CancellationToken cancellationToken)
   at MySqlConnector.MySqlCommand.ExecuteReaderNoResetTimeoutAsync(CommandBehavior behavior, IOBehavior ioBehavior, CancellationToken cancellationToken)
   at MySqlConnector.MySqlCommand.ExecuteReaderAsync(CommandBehavior behavior, IOBehavior ioBehavior, CancellationToken cancellationToken)
   at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine)
   at MySqlConnector.MySqlCommand.ExecuteReaderAsync(CommandBehavior behavior, IOBehavior ioBehavior, CancellationToken cancellationToken)
   at MySqlConnector.MySqlCommand.ExecuteDbDataReaderAsync(CommandBehavior behavior, CancellationToken cancellationToken)
   at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine)
   at MySqlConnector.MySqlCommand.ExecuteDbDataReaderAsync(CommandBehavior behavior, CancellationToken cancellationToken)
   at NPoco.DatabaseType.ExecuteReaderAsync(Database database, DbCommand cmd)
   at NPoco.Database.ExecutionHookAsync[T](Func`1 action)
   at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine)
   at NPoco.Database.ExecutionHookAsync[T](Func`1 action)
   at NPoco.Database.ExecuteReaderHelperAsync(DbCommand cmd)
   at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine)
   at NPoco.Database.ExecuteReaderHelperAsync(DbCommand cmd)
   at NPoco.Database.ExecuteDataReader(DbCommand cmd, Boolean sync)
   at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine)
   at NPoco.Database.ExecuteDataReader(DbCommand cmd, Boolean sync)
   at NPoco.Database.QueryAsync[T](T instance, Expression`1 listExpression, Func`2 idFunc, Sql Sql, PocoData pocoData)+MoveNext()
   at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine)
   at NPoco.Database.QueryAsync[T](T instance, Expression`1 listExpression, Func`2 idFunc, Sql Sql, PocoData pocoData)+System.Collections.Generic.IAsyncEnumerator<T>.MoveNextAsync()
   at System.Runtime.CompilerServices.ConfiguredCancelableAsyncEnumerable`1.Enumerator.MoveNextAsync()
   at System.Linq.AsyncEnumerable.<ToListAsync>g__Core|620_0[TSource](IAsyncEnumerable`1 source, CancellationToken cancellationToken)
   at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine)
   at System.Linq.AsyncEnumerable.<ToListAsync>g__Core|620_0[TSource](IAsyncEnumerable`1 source, CancellationToken cancellationToken)
   at System.Linq.AsyncEnumerable.ToListAsync[TSource](IAsyncEnumerable`1 source, CancellationToken cancellationToken)
   at NPoco.Database.FetchAsync[T](String sql, Object[] args)
   at DBAAlertBroker.Services.BusinessService.GetRelatedServicesAsync(String ip, Int32 port, String dbName)
   at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine)
   at DBAAlertBroker.Services.BusinessService.GetRelatedServicesAsync(String ip, Int32 port, String dbName)
   at DBAAlertBroker.Services.BusinessService.OperateAsync(AlertObj alertObj)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.AsyncStateMachineBox`1.MoveNext(Thread threadPoolThread)
   at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(IAsyncStateMachineBox box, Boolean allowInlining)
   at System.Threading.Tasks.Task.RunContinuations(Object continuationObject)
   at System.Threading.Tasks.Task`1.TrySetResult(TResult result)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.SetExistingTaskResult(Task`1 task, TResult result)
   at DBAInternalCommon.Services.DBAMetaService.GetProcessInfoAsync(String url)

@naughtyGitCat
Copy link

@naughtyGitCat naughtyGitCat commented Mar 8, 2022

we do not use nat, but physical host with static ip , the latest version does not fix too

mysql conn string is

    "MySQL": {
      "ReadWriteURI": "server=XXXXXX;port=3306;user=XXXXX;password=XXXXX;database=XXXXX",
      "ReadOnlyURI": "server=YYYYY;port=3306;user=YYYY;password=YYYY;database=YYYY;DefaultCommandTimeout=188;SslMode=None;MaximumPoolsize=1588;MinimumPoolSize=8;ConnectionReset=false;ConnectionIdlePingTime=2;ConnectionIdleTimeout=88"
    },

@caleblloyd

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging a pull request may close this issue.

4 participants