MQTT communication exception while receiving packets #158
Comments
|
Hi @liuhongbo, Best regards |
|
I am on version 2.6.0 thank you. |
|
i fixed the issue, even though the fix is not perfect. the only fix i found is like this: in theory this fix is not good enough to be accepted as a fix, but this is the only fix that works for me so far. |
|
I see the same issue connecting to an AWS IOT over WebSockets. Specifically the "The remote party closed the WebSocket connection without completing the close handshake" error. Like @liuhongbo, setting the ServicePointManager to Infinite timeout seemed to address the issue. I'd like to understand better why this occurs and the best practice for avoiding it. |
|
@schenckg my understand is that (but not quite sure).
|
|
@liuhongbo @chkr1011 @schenckg @JTrotta |
|
The SSL stream disposed exception is fixed in latest dev branch. Please reopen this ticket if this still fails. |
|
I'm getting the exact same issue as described in the original post in 3.0.4. I'm also talking to an AWS IoT Message Broker. The workaround that @liuhongbo mentioned ( |
@chkr1011 You need to check this again... |
|
@jltjohanlindqvist So your transport is also WebSocket + TLS and you send NO messages? Just Pin/Pong? Or is keep alive disabled? |
|
@chkr1011 correct, I configure with proxy, client id and a wss url as the websocketserver. edit: I get this in the log when doing connect: |
|
The strange thing is, we have two addresses to our proxy at work. If I add the workaround it works with one of the proxy addresses, but not with the other proxy address, then I get the same issue of edit: So the workaround only works if I'm using the same proxy address that I've configured in the windows proxy settings. But I assume it should work without the workaround. |
|
Do you maybe use an UWP app? |
|
It's a WPF app. |
|
I'm pretty sure it has to do with the problem described in this comment https://github.com/dotnet/corefx/issues/31880#issuecomment-419269635 and the mentioned stackoverflow question. |
|
It is as I suspected, which is described in the comment previously mentioned, I get Which if I understand the corefx issue correctly is not handled well in .NET Framework. I'm gonna test and see if I have the same issue using a .NET Core-project. |
|
Tested using a simple project for both .NET Core 2.2 and .NET Framework 4.8, and I can only reproduce the issue in .NET Framework 4.8. So it seems like it's a bug in the .NET Framework that they've fixed in .NET Core. Guess there's not a lot to do in this project to fix this issue. |
|
I created an issue on the .NET Framework issue tracker, https://developercommunity.visualstudio.com/content/problem/629916/websocket-times-out-after-100-seconds.html |
|
@chkr1011 got this error tonight when running an MQTT WebSocket, as far as I understand this exception shouldn't happen in the Unhandled task exception branches-MQTT-0554, System.AggregateException: A Task's exception(s) were not observed either by Waiting on the Task or accessing its Exception property. As a result, the unobserved exception was rethrown by the finalizer thread. ---> MQTTnet.Exceptions.MqttCommunicationException: The remote party closed the WebSocket connection without completing the close handshake. ---> System.Net.WebSockets.WebSocketException: The remote party closed the WebSocket connection without completing the close handshake. ---> System.ObjectDisposedException: Cannot access a disposed object.
Object name: 'SslStream'.
at System.Net.Security.SslState.CheckThrow(Boolean authSuccessCheck, Boolean shutdownCheck)
at System.Net.Security.SslState.get_SecureStream()
at System.Net.TlsStream.EndRead(IAsyncResult asyncResult)
at System.Threading.Tasks.TaskFactory`1.FromAsyncTrimPromise`1.Complete(TInstance thisRef, Func`3 endMethod, IAsyncResult asyncResult, Boolean requiresSynchronization)
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Net.WebSockets.WebSocketConnectionStream.<ReadAsync>d__21.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Net.WebSockets.WebSocketBase.WebSocketOperation.<Process>d__19.MoveNext()
--- End of inner exception stack trace ---
at System.Net.WebSockets.WebSocketBase.WebSocketOperation.<Process>d__19.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Net.WebSockets.WebSocketBase.<ReceiveAsyncCore>d__45.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at MQTTnet.Implementations.MqttWebSocketChannel.<ReadAsync>d__18.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at MQTTnet.Formatter.MqttPacketReader.<ReadFixedHeaderAsync>d__3.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at MQTTnet.Adapter.MqttChannelAdapter.<ReceiveAsync>d__37.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at MQTTnet.Adapter.MqttChannelAdapter.<ReceivePacketAsync>d__35.MoveNext()
--- End of inner exception stack trace ---
at MQTTnet.Adapter.MqttChannelAdapter.WrapException(Exception exception)
at MQTTnet.Adapter.MqttChannelAdapter.<ReceivePacketAsync>d__35.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at MQTTnet.Client.MqttClient.<TryReceivePacketsAsync>d__48.MoveNext()
--- End of inner exception stack trace ---
---> (Inner Exception #0) MQTTnet.Exceptions.MqttCommunicationException: The remote party closed the WebSocket connection without completing the close handshake. ---> System.Net.WebSockets.WebSocketException: The remote party closed the WebSocket connection without completing the close handshake. ---> System.ObjectDisposedException: Cannot access a disposed object.
Object name: 'SslStream'.
at System.Net.Security.SslState.CheckThrow(Boolean authSuccessCheck, Boolean shutdownCheck)
at System.Net.Security.SslState.get_SecureStream()
at System.Net.TlsStream.EndRead(IAsyncResult asyncResult)
at System.Threading.Tasks.TaskFactory`1.FromAsyncTrimPromise`1.Complete(TInstance thisRef, Func`3 endMethod, IAsyncResult asyncResult, Boolean requiresSynchronization)
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Net.WebSockets.WebSocketConnectionStream.<ReadAsync>d__21.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Net.WebSockets.WebSocketBase.WebSocketOperation.<Process>d__19.MoveNext()
--- End of inner exception stack trace ---
at System.Net.WebSockets.WebSocketBase.WebSocketOperation.<Process>d__19.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Net.WebSockets.WebSocketBase.<ReceiveAsyncCore>d__45.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at MQTTnet.Implementations.MqttWebSocketChannel.<ReadAsync>d__18.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at MQTTnet.Formatter.MqttPacketReader.<ReadFixedHeaderAsync>d__3.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at MQTTnet.Adapter.MqttChannelAdapter.<ReceiveAsync>d__37.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at MQTTnet.Adapter.MqttChannelAdapter.<ReceivePacketAsync>d__35.MoveNext()
--- End of inner exception stack trace ---
at MQTTnet.Adapter.MqttChannelAdapter.WrapException(Exception exception)
at MQTTnet.Adapter.MqttChannelAdapter.<ReceivePacketAsync>d__35.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at MQTTnet.Client.MqttClient.<TryReceivePacketsAsync>d__48.MoveNext()<---I still have the 100 second timeout issue, but we're bypassing that issue by just reconnecting every 100 seconds, which is not optimal, but until Microsoft or Amazon fixes their issues there's not a lot more we can do about it. So this happened randomly while running a WebSocket during the night, with us using one MqttClient where we're reconnecting and subscribing each time we get the 100 second timeout. Glad to provide more information if you feel there's any missing. I unfortunately don't have any logs from the MQTTnet lib. |
|
@jltjohanlindqvist In order to avouid the disconnects you can try to download and use the WebSocket4Net extensions. Then the WebSocket layer is a different one and may not disconnect. Or is the issue caused by the server (wasn't following all the time)? Which version are you using? In theory this should be already fixed. |
|
We're using 3.0.5, and I saw that the WebSocket4Net doesn't support proxies which is why I'm not using it. We're not using the server at all, only the client parts of the MQTTnet lib. A colleague and I looked at the source yesterday and we think the reason is that when we get to line 420 in MqttClient.cs the adapter has been disposed and thus we get an object disposed exception. This is probably because we weren't re-initializing our MqttClient object every time we got a disconnect (which in my opinion shouldn't be needed), we do that now and I'll see if we get the crash again. |
|
Still get the same crash even though we're always creating a new MqttClient every time we disconnect. So my idea that recreating the MqttClient would help was obviously wrong and now I'm at a bit of a loss what to do from my side? |
|
@chkr1011 any idea why this would be happening even after the changes to |
|
The main thing I am wondering is why this is not covered in a try-catch block. I am searching for it but have to clue. Do you have any idea? |
|
Could it be because the task is started in a |
|
Or it could be something that goes wrong in the catch block in |
|
I have the same problem. Still can't find a workaround, same as @jltjohanlindqvist It looks like the last thing lib does is in MqttTcpChannel.ConnectAsync, and something fires: Object name: 'System.Net.Sockets.Socket'. ---> System.ObjectDisposedException: Cannot access a disposed object. I'm using WPF application targeting 4.7.2 Framework. |
|
It appears problem was on our side, if it helps @jltjohanlindqvist : To test disconnect, I was changing my internet connection, and waiting for ping timeout. But my other internet had other IP and couldn't connect to MQTT server (it wasn't whitelisted). |
|
@jltjohanlindqvist Do you have some disconnected handler attached? From the StackTrace it looks like that it comes from WebSockets. So nothing we can fix. The Task.Run is calling code which is wrapped in TryCatch only. |
|
Yeah, I'm using the |
|
Why do you ask, @chkr1011 ? |
|
New stack trace where we also crash because of an unhandled exception. |
|
@chkr1011 we're still getting this crash, and after looking at the MQTTnet code we've found something we think might be the cause of the issue. What happens if one of the tasks that are run in a Is that if-case really needed? Why don't just remove the checking of canceled, completed and faulted and just always await the task? That way there wont be unhandled tasks, and if the task is completed or canceled the await will just return right away. Or is there a reason for the if-case that we're not seeing? edit: added a pull request with the changes I suggested here edit 2: also, what's the idea behind the first if-clause in the |
|
The reason was that be found a dead lock when using await. @JanEggers and I spent a lot of time to fix this. But I am eager to try your solution because the current solution is only a workaround. Let me try your pull request... |
|
Ah, I see. If you need any help don't hesitate to ask. |
|
@chkr1011 you asked earlier if I was using a DisconnectedHandler, was there any specific reason why you asked this? Also, we've still not been able to fix the issue, still investigating. |
|
Please try 3.0.9-rc2. We again addressed some connection management issues. |
|
I close this ticket because I assume it is fixed. |
I use the MQTTnet client to connect to aws iot through a presigned url, it works, however, every about 100 seconds (it is pretty accurate every time) it will get disconnected. i am not sure why. here is the log i got:
I am not sure if this is a issue with aws or not, but just can not find the cause.
The text was updated successfully, but these errors were encountered: