<6> 2023-11-30 14:00:18.542 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 14:00:18.544 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 2.1815ms" <6> 2023-11-30 14:01:18.815 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 14:01:18.816 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 1.8306ms" <6> 2023-11-30 14:02:19.110 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 14:02:19.113 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 3.0590ms" <6> 2023-11-30 14:02:55.940 +00:00 [INF] - Entering periodic task to reauthenticate connected clients <6> 2023-11-30 14:03:19.425 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 14:03:19.431 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 6.3716ms" <6> 2023-11-30 14:03:49.669 +00:00 [INF] - Starting periodic operation Get EdgeHub config... <6> 2023-11-30 14:03:49.804 +00:00 [INF] - Obtained edge hub config from module twin <6> 2023-11-30 14:03:49.805 +00:00 [INF] - Set the following 3 route(s) in edge hub <6> 2023-11-30 14:03:49.805 +00:00 [INF] - plant_11-to-upstream: FROM /messages/modules/plant_11/outputs/upstream INTO $upstream <6> 2023-11-30 14:03:49.805 +00:00 [INF] - mtc_helper-to-upstream: FROM /messages/modules/mtc_helper/outputs/upstream INTO $upstream <6> 2023-11-30 14:03:49.805 +00:00 [INF] - metrics: FROM /messages/modules/metrics/outputs/metricOutput INTO $upstream <6> 2023-11-30 14:03:49.805 +00:00 [INF] - Updated message store TTL to 604800 seconds <6> 2023-11-30 14:03:49.805 +00:00 [INF] - Updated the edge hub store and forward configuration <6> 2023-11-30 14:03:49.805 +00:00 [INF] - Successfully completed periodic operation Get EdgeHub config <6> 2023-11-30 14:03:50.009 +00:00 [INF] - Updated reported properties for usea1svlimtcld1-vm/$edgeHub <6> 2023-11-30 14:04:19.983 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 14:04:19.984 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 1.5157ms" <6> 2023-11-30 14:05:20.349 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 14:05:20.351 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 2.3098ms" <6> 2023-11-30 14:06:20.771 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 14:06:20.774 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 3.4248ms" <6> 2023-11-30 14:07:21.054 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 14:07:21.057 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 3.2383ms" <6> 2023-11-30 14:07:49.241 +00:00 [INF] - Started task to cleanup processed and stale messages for endpoint iothub <6> 2023-11-30 14:07:50.256 +00:00 [INF] - Cleaned up 4623 messages from queue for endpoint iothub and 4623 messages from message store. <6> 2023-11-30 14:07:50.257 +00:00 [INF] - Started task to cleanup processed and stale messages for endpoint iothub_Pri2 <6> 2023-11-30 14:07:50.257 +00:00 [INF] - Cleaned up 0 messages from queue for endpoint iothub_Pri2 and 0 messages from message store. <6> 2023-11-30 14:07:55.940 +00:00 [INF] - Entering periodic task to reauthenticate connected clients <6> 2023-11-30 14:12:55.940 +00:00 [INF] - Entering periodic task to reauthenticate connected clients <6> 2023-11-30 14:13:21.089 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 14:13:21.092 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 3.1263ms" <6> 2023-11-30 14:17:55.944 +00:00 [INF] - Entering periodic task to reauthenticate connected clients <6> 2023-11-30 14:19:21.128 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 14:19:21.131 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 3.2047ms" <6> 2023-11-30 14:22:34.942 +00:00 [INF] - Closing cloud proxy c2e62221-c012-401e-8ff1-32e361180d5e for usea1svlimtcld1-vm/metrics because of inactivity <6> 2023-11-30 14:22:35.386 +00:00 [INF] - Starting refresh of device scope identities cache <6> 2023-11-30 14:22:55.941 +00:00 [INF] - Entering periodic task to reauthenticate connected clients <6> 2023-11-30 14:23:24.945 +00:00 [INF] - Closing receiver in cloud proxy c2e62221-c012-401e-8ff1-32e361180d5e for usea1svlimtcld1-vm/metrics <6> 2023-11-30 14:23:24.945 +00:00 [INF] - Closed cloud proxy c2e62221-c012-401e-8ff1-32e361180d5e for usea1svlimtcld1-vm/metrics <6> 2023-11-30 14:24:00.006 +00:00 [INF] - New token received on the Cbs link <6> 2023-11-30 14:24:00.006 +00:00 [INF] - Token updated for usea1svlimtcld1-vm/metrics <6> 2023-11-30 14:24:00.077 +00:00 [INF] - Not changed node: usea1svlimtcld1-vm <6> 2023-11-30 14:24:00.077 +00:00 [INF] - Not changed node: usea1svlimtcld1-vm/$edgeAgent <6> 2023-11-30 14:24:00.077 +00:00 [INF] - Not changed node: usea1svlimtcld1-vm/$edgeHub <6> 2023-11-30 14:24:00.077 +00:00 [INF] - Not changed node: usea1svlimtcld1-vm/mtc_helper <6> 2023-11-30 14:24:00.077 +00:00 [INF] - Not changed node: usea1svlimtcld1-vm/metrics <6> 2023-11-30 14:24:00.077 +00:00 [INF] - Not changed node: usea1svlimtcld1-vm/plant_11 <6> 2023-11-30 14:24:00.077 +00:00 [INF] - Done refreshing device scope identities cache. Waiting for 60 minutes. <6> 2023-11-30 14:25:21.170 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 14:25:21.171 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 1.5278ms" <6> 2023-11-30 14:26:21.570 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 14:26:21.571 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 1.1797ms" <6> 2023-11-30 14:27:21.911 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 14:27:21.913 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 1.2730ms" <6> 2023-11-30 14:27:55.942 +00:00 [INF] - Entering periodic task to reauthenticate connected clients <6> 2023-11-30 14:27:55.942 +00:00 [INF] - Error authenticating token for usea1svlimtcld1-vm/plant_11 because the token is expired or could not be parsed <6> 2023-11-30 14:27:55.943 +00:00 [INF] - Credentials for client usea1svlimtcld1-vm/plant_11 are not valid. <4> 2023-11-30 14:27:55.943 +00:00 [WRN] - Reauthenticating client usea1svlimtcld1-vm/plant_11 failed, removing client connection <6> 2023-11-30 14:27:55.943 +00:00 [INF] - Unable to re-authenticate usea1svlimtcld1-vm/plant_11, dropping client connection. <6> 2023-11-30 14:27:55.943 +00:00 [INF] - Removing device connection for device usea1svlimtcld1-vm/plant_11 with removeCloudConnection flag 'True'. <6> 2023-11-30 14:27:55.943 +00:00 [INF] - Closing AMQP device proxy for usea1svlimtcld1-vm/plant_11 because no handler was registered. Microsoft.Azure.Devices.Edge.Hub.Core.EdgeHubConnectionException: Connection closed for device usea1svlimtcld1-vm/plant_11. <6> 2023-11-30 14:27:55.949 +00:00 [INF] - Closing link Events for usea1svlimtcld1-vm/plant_11 <6> 2023-11-30 14:27:55.952 +00:00 [INF] - Closing link TwinSending for usea1svlimtcld1-vm/plant_11 <6> 2023-11-30 14:27:55.953 +00:00 [INF] - Closing link TwinReceiving for usea1svlimtcld1-vm/plant_11 <6> 2023-11-30 14:27:55.953 +00:00 [INF] - Closing link MethodSending for usea1svlimtcld1-vm/plant_11 <6> 2023-11-30 14:27:55.954 +00:00 [INF] - Closing link MethodReceiving for usea1svlimtcld1-vm/plant_11 <6> 2023-11-30 14:27:55.999 +00:00 [INF] - Closing receiver in cloud proxy 7970538e-e801-42da-b67d-274e452c0681 for usea1svlimtcld1-vm/plant_11 <6> 2023-11-30 14:27:55.999 +00:00 [INF] - Closed cloud proxy 7970538e-e801-42da-b67d-274e452c0681 for usea1svlimtcld1-vm/plant_11 <6> 2023-11-30 14:27:55.999 +00:00 [INF] - Device connection removed for device usea1svlimtcld1-vm/plant_11 <6> 2023-11-30 14:27:56.000 +00:00 [INF] - Error authenticating token for usea1svlimtcld1-vm/mtc_helper because the token is expired or could not be parsed <6> 2023-11-30 14:27:56.000 +00:00 [INF] - Credentials for client usea1svlimtcld1-vm/mtc_helper are not valid. <4> 2023-11-30 14:27:56.000 +00:00 [WRN] - Reauthenticating client usea1svlimtcld1-vm/mtc_helper failed, removing client connection <6> 2023-11-30 14:27:56.000 +00:00 [INF] - Unable to re-authenticate usea1svlimtcld1-vm/mtc_helper, dropping client connection. <6> 2023-11-30 14:27:56.000 +00:00 [INF] - Removing device connection for device usea1svlimtcld1-vm/mtc_helper with removeCloudConnection flag 'True'. <6> 2023-11-30 14:27:56.000 +00:00 [INF] - Closing AMQP device proxy for usea1svlimtcld1-vm/mtc_helper because no handler was registered. Microsoft.Azure.Devices.Edge.Hub.Core.EdgeHubConnectionException: Connection closed for device usea1svlimtcld1-vm/mtc_helper. <6> 2023-11-30 14:27:56.012 +00:00 [INF] - Closing link Events for usea1svlimtcld1-vm/mtc_helper <6> 2023-11-30 14:27:56.012 +00:00 [INF] - Closing link MethodSending for usea1svlimtcld1-vm/mtc_helper <6> 2023-11-30 14:27:56.043 +00:00 [INF] - Closing link MethodReceiving for usea1svlimtcld1-vm/mtc_helper <6> 2023-11-30 14:27:56.043 +00:00 [INF] - Closing link TwinSending for usea1svlimtcld1-vm/mtc_helper <6> 2023-11-30 14:27:56.043 +00:00 [INF] - Closing link TwinReceiving for usea1svlimtcld1-vm/mtc_helper <6> 2023-11-30 14:27:56.085 +00:00 [INF] - Closing receiver in cloud proxy d3ecc4ff-42eb-468f-b994-14efb8a9f921 for usea1svlimtcld1-vm/mtc_helper <6> 2023-11-30 14:27:56.085 +00:00 [INF] - Closed cloud proxy d3ecc4ff-42eb-468f-b994-14efb8a9f921 for usea1svlimtcld1-vm/mtc_helper <6> 2023-11-30 14:27:56.085 +00:00 [INF] - Device connection removed for device usea1svlimtcld1-vm/mtc_helper <6> 2023-11-30 14:27:56.295 +00:00 [INF] - Updated reported properties for usea1svlimtcld1-vm/$edgeHub <6> 2023-11-30 14:27:56.336 +00:00 [INF] - New token received on the Cbs link <6> 2023-11-30 14:27:56.338 +00:00 [INF] - Client usea1svlimtcld1-vm/plant_11 in device scope authenticated locally. <6> 2023-11-30 14:27:56.338 +00:00 [INF] - New device connection for device usea1svlimtcld1-vm/plant_11 <6> 2023-11-30 14:27:56.338 +00:00 [INF] - Client usea1svlimtcld1-vm/plant_11 connected to edgeHub, processing existing subscriptions. <6> 2023-11-30 14:27:56.338 +00:00 [INF] - Bind device proxy for device usea1svlimtcld1-vm/plant_11 <6> 2023-11-30 14:27:56.338 +00:00 [INF] - Initialized device listener in the AMQP protocol head for usea1svlimtcld1-vm/plant_11 <6> 2023-11-30 14:27:56.338 +00:00 [INF] - Opened link Events for usea1svlimtcld1-vm/plant_11 <6> 2023-11-30 14:27:56.339 +00:00 [INF] - Attempting to connect to IoT Hub for client usea1svlimtcld1-vm/plant_11 via AMQP... <6> 2023-11-30 14:27:56.342 +00:00 [INF] - Processing pending subscriptions for usea1svlimtcld1-vm/plant_11 <6> 2023-11-30 14:27:56.342 +00:00 [INF] - Opened link MethodSending for usea1svlimtcld1-vm/plant_11 <6> 2023-11-30 14:27:56.343 +00:00 [INF] - Opened link MethodReceiving for usea1svlimtcld1-vm/plant_11 <6> 2023-11-30 14:27:56.345 +00:00 [INF] - Opened link TwinSending for usea1svlimtcld1-vm/plant_11 <6> 2023-11-30 14:27:56.345 +00:00 [INF] - Opened link TwinReceiving for usea1svlimtcld1-vm/plant_11 <6> 2023-11-30 14:27:56.453 +00:00 [INF] - Cloud connection for usea1svlimtcld1-vm/plant_11 is True <6> 2023-11-30 14:27:56.453 +00:00 [INF] - Connection status for usea1svlimtcld1-vm/plant_11 changed to ConnectionEstablished <6> 2023-11-30 14:27:56.453 +00:00 [INF] - Client usea1svlimtcld1-vm/plant_11 connected to cloud, processing existing subscriptions. <6> 2023-11-30 14:27:56.453 +00:00 [INF] - Skipping usea1svlimtcld1-vm/plant_11 for subscription processing, as it is currently being processed. <6> 2023-11-30 14:27:56.453 +00:00 [INF] - Created cloud proxy for client usea1svlimtcld1-vm/plant_11 via AMQP, with client operation timeout 20 seconds. <6> 2023-11-30 14:27:56.453 +00:00 [INF] - Initialized cloud proxy f1c87086-8188-46ee-951a-58a724ad77ec for usea1svlimtcld1-vm/plant_11 <6> 2023-11-30 14:27:56.453 +00:00 [INF] - Created cloud connection for client usea1svlimtcld1-vm/plant_11 <6> 2023-11-30 14:27:56.499 +00:00 [INF] - Processing pending subscriptions for usea1svlimtcld1-vm/plant_11 <6> 2023-11-30 14:27:56.510 +00:00 [INF] - New token received on the Cbs link <6> 2023-11-30 14:27:56.512 +00:00 [INF] - Client usea1svlimtcld1-vm/mtc_helper in device scope authenticated locally. <6> 2023-11-30 14:27:56.513 +00:00 [INF] - New device connection for device usea1svlimtcld1-vm/mtc_helper <6> 2023-11-30 14:27:56.513 +00:00 [INF] - Client usea1svlimtcld1-vm/mtc_helper connected to edgeHub, processing existing subscriptions. <6> 2023-11-30 14:27:56.513 +00:00 [INF] - Bind device proxy for device usea1svlimtcld1-vm/mtc_helper <6> 2023-11-30 14:27:56.513 +00:00 [INF] - Initialized device listener in the AMQP protocol head for usea1svlimtcld1-vm/mtc_helper <6> 2023-11-30 14:27:56.513 +00:00 [INF] - Opened link Events for usea1svlimtcld1-vm/mtc_helper <6> 2023-11-30 14:27:56.513 +00:00 [INF] - Attempting to connect to IoT Hub for client usea1svlimtcld1-vm/mtc_helper via AMQP... <6> 2023-11-30 14:27:56.520 +00:00 [INF] - Opened link MethodSending for usea1svlimtcld1-vm/mtc_helper <6> 2023-11-30 14:27:56.520 +00:00 [INF] - Processing pending subscriptions for usea1svlimtcld1-vm/mtc_helper <6> 2023-11-30 14:27:56.520 +00:00 [INF] - Opened link MethodReceiving for usea1svlimtcld1-vm/mtc_helper <6> 2023-11-30 14:27:56.522 +00:00 [INF] - Opened link TwinSending for usea1svlimtcld1-vm/mtc_helper <6> 2023-11-30 14:27:56.522 +00:00 [INF] - Opened link TwinReceiving for usea1svlimtcld1-vm/mtc_helper <6> 2023-11-30 14:27:56.629 +00:00 [INF] - Cloud connection for usea1svlimtcld1-vm/mtc_helper is True <6> 2023-11-30 14:27:56.629 +00:00 [INF] - Connection status for usea1svlimtcld1-vm/mtc_helper changed to ConnectionEstablished <6> 2023-11-30 14:27:56.629 +00:00 [INF] - Client usea1svlimtcld1-vm/mtc_helper connected to cloud, processing existing subscriptions. <6> 2023-11-30 14:27:56.629 +00:00 [INF] - Skipping usea1svlimtcld1-vm/mtc_helper for subscription processing, as it is currently being processed. <6> 2023-11-30 14:27:56.629 +00:00 [INF] - Created cloud proxy for client usea1svlimtcld1-vm/mtc_helper via AMQP, with client operation timeout 20 seconds. <6> 2023-11-30 14:27:56.629 +00:00 [INF] - Initialized cloud proxy f3155d9b-f499-4026-89a5-3fcdc1c96fba for usea1svlimtcld1-vm/mtc_helper <6> 2023-11-30 14:27:56.629 +00:00 [INF] - Created cloud connection for client usea1svlimtcld1-vm/mtc_helper <6> 2023-11-30 14:27:56.688 +00:00 [INF] - Processing pending subscriptions for usea1svlimtcld1-vm/mtc_helper <6> 2023-11-30 14:28:01.603 +00:00 [INF] - Updated reported properties for usea1svlimtcld1-vm/$edgeHub <6> 2023-11-30 14:28:22.525 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 14:28:22.534 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 8.4093ms" <6> 2023-11-30 14:29:23.041 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 14:29:23.043 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 2.2944ms" <6> 2023-11-30 14:30:23.350 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 14:30:23.351 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 1.1066ms" <6> 2023-11-30 14:31:23.654 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 14:31:23.657 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 2.6142ms" <6> 2023-11-30 14:32:23.926 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 14:32:23.927 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 0.6759ms" <6> 2023-11-30 14:32:55.941 +00:00 [INF] - Entering periodic task to reauthenticate connected clients <6> 2023-11-30 14:33:24.214 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 14:33:24.218 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 3.9972ms" <6> 2023-11-30 14:34:24.670 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 14:34:24.673 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 3.2941ms" <6> 2023-11-30 14:35:24.957 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 14:35:24.959 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 2.1169ms" <6> 2023-11-30 14:36:25.221 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 14:36:25.222 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 1.6782ms" <6> 2023-11-30 14:37:25.541 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 14:37:25.546 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 4.9252ms" <6> 2023-11-30 14:37:37.920 +00:00 [INF] - "Request starting HTTP/1.1 GET http://172.18.0.6:9600/metrics - -" <6> 2023-11-30 14:37:37.922 +00:00 [INF] - "Request finished HTTP/1.1 GET http://172.18.0.6:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 1.6705ms" <6> 2023-11-30 14:37:42.970 +00:00 [INF] - Starting compaction of stores <6> 2023-11-30 14:37:42.970 +00:00 [INF] - Starting compaction of store checkpoints <6> 2023-11-30 14:37:42.970 +00:00 [INF] - Starting compaction of store sessions <6> 2023-11-30 14:37:42.970 +00:00 [INF] - Starting compaction of store messages <6> 2023-11-30 14:37:42.970 +00:00 [INF] - Starting compaction of store default <6> 2023-11-30 14:37:42.970 +00:00 [INF] - Starting compaction of store iothub <6> 2023-11-30 14:37:42.970 +00:00 [INF] - Starting compaction of store twins <6> 2023-11-30 14:37:42.970 +00:00 [INF] - Starting compaction of store EdgeTwin <6> 2023-11-30 14:37:42.970 +00:00 [INF] - Starting compaction of store DeviceScopeCache <6> 2023-11-30 14:37:42.970 +00:00 [INF] - Starting compaction of store MetadataStore <6> 2023-11-30 14:37:42.970 +00:00 [INF] - Starting compaction of store iothub_Pri2 <6> 2023-11-30 14:37:50.256 +00:00 [INF] - Started task to cleanup processed and stale messages for endpoint iothub <6> 2023-11-30 14:37:51.138 +00:00 [INF] - Cleaned up 5025 messages from queue for endpoint iothub and 5025 messages from message store. <6> 2023-11-30 14:37:51.138 +00:00 [INF] - Started task to cleanup processed and stale messages for endpoint iothub_Pri2 <6> 2023-11-30 14:37:51.139 +00:00 [INF] - Cleaned up 0 messages from queue for endpoint iothub_Pri2 and 0 messages from message store. <6> 2023-11-30 14:37:55.940 +00:00 [INF] - Entering periodic task to reauthenticate connected clients <6> 2023-11-30 14:38:25.874 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 14:38:25.881 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 6.4493ms" <6> 2023-11-30 14:39:26.214 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 14:39:26.215 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 1.6759ms" <6> 2023-11-30 14:40:26.513 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 14:40:26.515 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 1.5813ms" <6> 2023-11-30 14:41:26.946 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 14:41:26.949 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 3.3123ms" <6> 2023-11-30 14:42:27.241 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 14:42:27.243 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 2.2923ms" <6> 2023-11-30 14:42:55.942 +00:00 [INF] - Entering periodic task to reauthenticate connected clients <6> 2023-11-30 14:43:27.589 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 14:43:27.589 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 0.4678ms" <6> 2023-11-30 14:44:27.970 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 14:44:27.972 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 2.0300ms" <6> 2023-11-30 14:45:28.234 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 14:45:28.236 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 2.4072ms" <6> 2023-11-30 14:46:28.577 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 14:46:28.580 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 2.3338ms" <6> 2023-11-30 14:47:28.909 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 14:47:28.911 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 2.8228ms" <6> 2023-11-30 14:47:55.940 +00:00 [INF] - Entering periodic task to reauthenticate connected clients <6> 2023-11-30 14:48:29.294 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 14:48:29.298 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 3.8985ms" <6> 2023-11-30 14:49:29.745 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 14:49:29.747 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 2.4150ms" <6> 2023-11-30 14:50:30.069 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 14:50:30.071 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 1.3502ms" <6> 2023-11-30 14:51:30.357 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 14:51:30.360 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 3.0720ms" <6> 2023-11-30 14:52:30.686 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 14:52:30.692 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 5.4174ms" <6> 2023-11-30 14:52:55.940 +00:00 [INF] - Entering periodic task to reauthenticate connected clients <6> 2023-11-30 14:53:31.054 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 14:53:31.058 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 3.8986ms" <6> 2023-11-30 14:54:31.470 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 14:54:31.473 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 3.3485ms" <6> 2023-11-30 14:55:31.838 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 14:55:31.846 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 7.5566ms" <6> 2023-11-30 14:56:32.183 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 14:56:32.186 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 3.3498ms" <6> 2023-11-30 14:57:32.568 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 14:57:32.572 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 4.0957ms" <6> 2023-11-30 14:57:55.940 +00:00 [INF] - Entering periodic task to reauthenticate connected clients <6> 2023-11-30 14:58:33.087 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 14:58:33.092 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 5.0445ms" <6> 2023-11-30 14:59:33.435 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 14:59:33.443 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 8.3026ms" <6> 2023-11-30 15:00:33.826 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 15:00:33.840 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 14.0533ms" <6> 2023-11-30 15:01:34.128 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 15:01:34.133 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 4.4111ms" <6> 2023-11-30 15:02:34.407 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 15:02:34.410 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 3.5184ms" <6> 2023-11-30 15:02:55.941 +00:00 [INF] - Entering periodic task to reauthenticate connected clients <6> 2023-11-30 15:03:34.647 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 15:03:34.659 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 12.1188ms" <6> 2023-11-30 15:03:49.804 +00:00 [INF] - Starting periodic operation Get EdgeHub config... <6> 2023-11-30 15:03:49.955 +00:00 [INF] - Obtained edge hub config from module twin <6> 2023-11-30 15:03:49.956 +00:00 [INF] - Set the following 3 route(s) in edge hub <6> 2023-11-30 15:03:49.956 +00:00 [INF] - plant_11-to-upstream: FROM /messages/modules/plant_11/outputs/upstream INTO $upstream <6> 2023-11-30 15:03:49.956 +00:00 [INF] - mtc_helper-to-upstream: FROM /messages/modules/mtc_helper/outputs/upstream INTO $upstream <6> 2023-11-30 15:03:49.956 +00:00 [INF] - metrics: FROM /messages/modules/metrics/outputs/metricOutput INTO $upstream <6> 2023-11-30 15:03:49.956 +00:00 [INF] - Updated message store TTL to 604800 seconds <6> 2023-11-30 15:03:49.956 +00:00 [INF] - Updated the edge hub store and forward configuration <6> 2023-11-30 15:03:49.956 +00:00 [INF] - Successfully completed periodic operation Get EdgeHub config <6> 2023-11-30 15:03:50.133 +00:00 [INF] - Updated reported properties for usea1svlimtcld1-vm/$edgeHub <6> 2023-11-30 15:04:35.054 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 15:04:35.056 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 1.5750ms" <6> 2023-11-30 15:05:35.373 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 15:05:35.375 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 2.3780ms" <6> 2023-11-30 15:06:35.769 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 15:06:35.779 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 10.0983ms" <6> 2023-11-30 15:07:36.005 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 15:07:36.009 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 4.2993ms" <6> 2023-11-30 15:07:51.140 +00:00 [INF] - Started task to cleanup processed and stale messages for endpoint iothub <6> 2023-11-30 15:07:52.185 +00:00 [INF] - Cleaned up 8317 messages from queue for endpoint iothub and 8317 messages from message store. <6> 2023-11-30 15:07:52.185 +00:00 [INF] - Started task to cleanup processed and stale messages for endpoint iothub_Pri2 <6> 2023-11-30 15:07:52.186 +00:00 [INF] - Cleaned up 0 messages from queue for endpoint iothub_Pri2 and 0 messages from message store. <6> 2023-11-30 15:07:55.942 +00:00 [INF] - Entering periodic task to reauthenticate connected clients <6> 2023-11-30 15:08:36.295 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 15:08:36.298 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 3.1489ms" <6> 2023-11-30 15:09:36.794 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 15:09:36.796 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 1.7076ms" <6> 2023-11-30 15:10:37.324 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 15:10:37.327 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 2.4340ms" <6> 2023-11-30 15:11:37.687 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 15:11:37.692 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 4.9003ms" <6> 2023-11-30 15:12:38.025 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 15:12:38.027 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 2.4486ms" <6> 2023-11-30 15:12:55.940 +00:00 [INF] - Entering periodic task to reauthenticate connected clients <6> 2023-11-30 15:13:38.326 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 15:13:38.332 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 6.8127ms" <6> 2023-11-30 15:14:38.730 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 15:14:38.740 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 10.1887ms" <6> 2023-11-30 15:14:38.998 +00:00 [INF] - New token received on the Cbs link <6> 2023-11-30 15:14:38.998 +00:00 [INF] - Token updated for usea1svlimtcld1-vm/metrics <6> 2023-11-30 15:14:38.999 +00:00 [INF] - New token received on the Cbs link <6> 2023-11-30 15:14:38.999 +00:00 [INF] - Token updated for usea1svlimtcld1-vm/metrics <6> 2023-11-30 15:14:38.999 +00:00 [INF] - New token received on the Cbs link <6> 2023-11-30 15:14:38.999 +00:00 [INF] - Token updated for usea1svlimtcld1-vm/metrics <6> 2023-11-30 15:14:39.201 +00:00 [INF] - New token received on the Cbs link <6> 2023-11-30 15:14:39.201 +00:00 [INF] - Token updated for usea1svlimtcld1-vm/metrics <6> 2023-11-30 15:15:39.119 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 15:15:39.122 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 2.6616ms" <6> 2023-11-30 15:16:39.486 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 15:16:39.500 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 14.0381ms" <6> 2023-11-30 15:17:39.832 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 15:17:39.836 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 4.4115ms" <6> 2023-11-30 15:17:55.941 +00:00 [INF] - Entering periodic task to reauthenticate connected clients <6> 2023-11-30 15:18:40.141 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 15:18:40.145 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 3.5899ms" <6> 2023-11-30 15:18:57.182 +00:00 [INF] - New token received on the Cbs link <6> 2023-11-30 15:18:57.183 +00:00 [INF] - Token updated for usea1svlimtcld1-vm/plant_11 <6> 2023-11-30 15:18:57.342 +00:00 [INF] - New token received on the Cbs link <6> 2023-11-30 15:18:57.342 +00:00 [INF] - Token updated for usea1svlimtcld1-vm/mtc_helper <6> 2023-11-30 15:19:40.550 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 15:19:40.556 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 6.1404ms" <6> 2023-11-30 15:20:40.886 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 15:20:40.891 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 4.3291ms" <6> 2023-11-30 15:21:41.163 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 15:21:41.165 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 1.9971ms" <6> 2023-11-30 15:22:41.446 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 15:22:41.448 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 2.6745ms" <6> 2023-11-30 15:22:55.940 +00:00 [INF] - Entering periodic task to reauthenticate connected clients <6> 2023-11-30 15:23:41.792 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 15:23:41.798 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 6.9992ms" <6> 2023-11-30 15:24:00.078 +00:00 [INF] - Starting refresh of device scope identities cache <6> 2023-11-30 15:24:00.778 +00:00 [INF] - Not changed node: usea1svlimtcld1-vm <6> 2023-11-30 15:24:00.779 +00:00 [INF] - Not changed node: usea1svlimtcld1-vm/$edgeAgent <6> 2023-11-30 15:24:00.779 +00:00 [INF] - Not changed node: usea1svlimtcld1-vm/$edgeHub <6> 2023-11-30 15:24:00.779 +00:00 [INF] - Not changed node: usea1svlimtcld1-vm/mtc_helper <6> 2023-11-30 15:24:00.779 +00:00 [INF] - Not changed node: usea1svlimtcld1-vm/metrics <6> 2023-11-30 15:24:00.779 +00:00 [INF] - Not changed node: usea1svlimtcld1-vm/plant_11 <6> 2023-11-30 15:24:00.779 +00:00 [INF] - Done refreshing device scope identities cache. Waiting for 60 minutes. <6> 2023-11-30 15:24:42.200 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 15:24:42.206 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 6.0810ms" <6> 2023-11-30 15:25:42.506 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 15:25:42.509 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 3.2526ms" <6> 2023-11-30 15:26:42.913 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 15:26:42.915 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 1.3653ms" <6> 2023-11-30 15:27:43.358 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 15:27:43.361 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 3.3370ms" <6> 2023-11-30 15:27:55.942 +00:00 [INF] - Entering periodic task to reauthenticate connected clients <6> 2023-11-30 15:28:43.782 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 15:28:43.784 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 2.2495ms" <6> 2023-11-30 15:29:44.179 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 15:29:44.182 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 2.9362ms" <6> 2023-11-30 15:30:44.496 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 15:30:44.501 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 4.5262ms" <6> 2023-11-30 15:31:44.868 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 15:31:44.872 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 3.4194ms" <6> 2023-11-30 15:32:45.342 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 15:32:45.346 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 4.4643ms" <6> 2023-11-30 15:32:55.940 +00:00 [INF] - Entering periodic task to reauthenticate connected clients <6> 2023-11-30 15:33:45.835 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 15:33:45.836 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 1.1597ms" <6> 2023-11-30 15:34:46.210 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 15:34:46.213 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 3.3877ms" <6> 2023-11-30 15:35:46.553 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 15:35:46.554 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 0.5772ms" <6> 2023-11-30 15:36:46.893 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 15:36:46.895 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 1.4552ms" <6> 2023-11-30 15:37:37.930 +00:00 [INF] - "Request starting HTTP/1.1 GET http://172.18.0.6:9600/metrics - -" <6> 2023-11-30 15:37:37.934 +00:00 [INF] - "Request finished HTTP/1.1 GET http://172.18.0.6:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 4.2600ms" <6> 2023-11-30 15:37:47.137 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 15:37:47.139 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 1.9710ms" <6> 2023-11-30 15:37:52.189 +00:00 [INF] - Started task to cleanup processed and stale messages for endpoint iothub <6> 2023-11-30 15:37:53.192 +00:00 [INF] - Cleaned up 8153 messages from queue for endpoint iothub and 8153 messages from message store. <6> 2023-11-30 15:37:53.192 +00:00 [INF] - Started task to cleanup processed and stale messages for endpoint iothub_Pri2 <6> 2023-11-30 15:37:53.192 +00:00 [INF] - Cleaned up 0 messages from queue for endpoint iothub_Pri2 and 0 messages from message store. <6> 2023-11-30 15:37:55.941 +00:00 [INF] - Entering periodic task to reauthenticate connected clients <6> 2023-11-30 15:42:55.940 +00:00 [INF] - Entering periodic task to reauthenticate connected clients <6> 2023-11-30 15:43:47.174 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 15:43:47.177 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 2.7289ms" <6> 2023-11-30 15:47:55.944 +00:00 [INF] - Entering periodic task to reauthenticate connected clients <6> 2023-11-30 15:49:47.218 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 15:49:47.224 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 6.0986ms" <6> 2023-11-30 15:52:55.942 +00:00 [INF] - Entering periodic task to reauthenticate connected clients <6> 2023-11-30 15:55:47.263 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 15:55:47.266 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 2.8138ms" <6> 2023-11-30 15:57:55.940 +00:00 [INF] - Entering periodic task to reauthenticate connected clients <6> 2023-11-30 16:01:47.302 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 16:01:47.304 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 2.1841ms" <6> 2023-11-30 16:02:55.941 +00:00 [INF] - Entering periodic task to reauthenticate connected clients <6> 2023-11-30 16:03:49.956 +00:00 [INF] - Starting periodic operation Get EdgeHub config... <6> 2023-11-30 16:03:50.103 +00:00 [INF] - Obtained edge hub config from module twin <6> 2023-11-30 16:03:50.105 +00:00 [INF] - Set the following 3 route(s) in edge hub <6> 2023-11-30 16:03:50.105 +00:00 [INF] - plant_11-to-upstream: FROM /messages/modules/plant_11/outputs/upstream INTO $upstream <6> 2023-11-30 16:03:50.105 +00:00 [INF] - mtc_helper-to-upstream: FROM /messages/modules/mtc_helper/outputs/upstream INTO $upstream <6> 2023-11-30 16:03:50.105 +00:00 [INF] - metrics: FROM /messages/modules/metrics/outputs/metricOutput INTO $upstream <6> 2023-11-30 16:03:50.105 +00:00 [INF] - Updated message store TTL to 604800 seconds <6> 2023-11-30 16:03:50.105 +00:00 [INF] - Updated the edge hub store and forward configuration <6> 2023-11-30 16:03:50.105 +00:00 [INF] - Successfully completed periodic operation Get EdgeHub config <6> 2023-11-30 16:03:50.284 +00:00 [INF] - Updated reported properties for usea1svlimtcld1-vm/$edgeHub <6> 2023-11-30 16:07:47.344 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 16:07:47.355 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 11.1570ms" <6> 2023-11-30 16:07:53.193 +00:00 [INF] - Started task to cleanup processed and stale messages for endpoint iothub <6> 2023-11-30 16:07:54.340 +00:00 [INF] - Cleaned up 9481 messages from queue for endpoint iothub and 9481 messages from message store. <6> 2023-11-30 16:07:54.340 +00:00 [INF] - Started task to cleanup processed and stale messages for endpoint iothub_Pri2 <6> 2023-11-30 16:07:54.340 +00:00 [INF] - Cleaned up 0 messages from queue for endpoint iothub_Pri2 and 0 messages from message store. <6> 2023-11-30 16:07:55.940 +00:00 [INF] - Entering periodic task to reauthenticate connected clients <6> 2023-11-30 16:09:57.997 +00:00 [INF] - New token received on the Cbs link <6> 2023-11-30 16:09:57.997 +00:00 [INF] - Token updated for usea1svlimtcld1-vm/mtc_helper <6> 2023-11-30 16:12:55.942 +00:00 [INF] - Entering periodic task to reauthenticate connected clients <6> 2023-11-30 16:13:47.391 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 16:13:47.393 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 2.5865ms" <6> 2023-11-30 16:17:55.941 +00:00 [INF] - Entering periodic task to reauthenticate connected clients <6> 2023-11-30 16:19:47.430 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 16:19:47.435 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 4.6957ms" <6> 2023-11-30 16:22:55.943 +00:00 [INF] - Entering periodic task to reauthenticate connected clients <6> 2023-11-30 16:22:55.943 +00:00 [INF] - Error authenticating token for usea1svlimtcld1-vm/metrics because the token is expired or could not be parsed <6> 2023-11-30 16:22:55.943 +00:00 [INF] - Credentials for client usea1svlimtcld1-vm/metrics are not valid. <4> 2023-11-30 16:22:55.943 +00:00 [WRN] - Reauthenticating client usea1svlimtcld1-vm/metrics failed, removing client connection <6> 2023-11-30 16:22:55.943 +00:00 [INF] - Unable to re-authenticate usea1svlimtcld1-vm/metrics, dropping client connection. <6> 2023-11-30 16:22:55.943 +00:00 [INF] - Removing device connection for device usea1svlimtcld1-vm/metrics with removeCloudConnection flag 'True'. <6> 2023-11-30 16:22:55.943 +00:00 [INF] - Closing AMQP device proxy for usea1svlimtcld1-vm/metrics because no handler was registered. Microsoft.Azure.Devices.Edge.Hub.Core.EdgeHubConnectionException: Connection closed for device usea1svlimtcld1-vm/metrics. <6> 2023-11-30 16:22:55.944 +00:00 [INF] - Closing link Events for usea1svlimtcld1-vm/metrics <6> 2023-11-30 16:22:55.948 +00:00 [INF] - Device connection removed for device usea1svlimtcld1-vm/metrics <6> 2023-11-30 16:23:15.973 +00:00 [INF] - Exiting connected state <6> 2023-11-30 16:23:15.973 +00:00 [INF] - Entering unreachable state <6> 2023-11-30 16:23:16.058 +00:00 [INF] - Entering disconnected state <6> 2023-11-30 16:23:16.058 +00:00 [INF] - Cloud connection for usea1svlimtcld1-vm/$edgeHub is False <6> 2023-11-30 16:23:16.058 +00:00 [INF] - Connection status for usea1svlimtcld1-vm/$edgeHub changed to Disconnected <6> 2023-11-30 16:23:16.058 +00:00 [INF] - Cloud connection for usea1svlimtcld1-vm/plant_11 is False <6> 2023-11-30 16:23:16.058 +00:00 [INF] - Connection status for usea1svlimtcld1-vm/plant_11 changed to Disconnected <6> 2023-11-30 16:23:16.058 +00:00 [INF] - Cloud connection for usea1svlimtcld1-vm/mtc_helper is False <6> 2023-11-30 16:23:16.058 +00:00 [INF] - Connection status for usea1svlimtcld1-vm/mtc_helper changed to Disconnected <6> 2023-11-30 16:23:16.058 +00:00 [INF] - Cloud connection for usea1svlimtcld1-vm/$edgeHub is False <6> 2023-11-30 16:23:16.058 +00:00 [INF] - Connection status for usea1svlimtcld1-vm/$edgeHub changed to Disconnected <6> 2023-11-30 16:23:16.058 +00:00 [INF] - Cloud connection for usea1svlimtcld1-vm/plant_11 is False <6> 2023-11-30 16:23:16.058 +00:00 [INF] - Connection status for usea1svlimtcld1-vm/plant_11 changed to Disconnected <6> 2023-11-30 16:23:16.058 +00:00 [INF] - Cloud connection for usea1svlimtcld1-vm/mtc_helper is False <6> 2023-11-30 16:23:16.058 +00:00 [INF] - Connection status for usea1svlimtcld1-vm/mtc_helper changed to Disconnected <6> 2023-11-30 16:23:16.060 +00:00 [INF] - Retrying cloud proxy operation UpdateReportedPropertiesAsync for usea1svlimtcld1-vm/$edgeHub. System.TimeoutException: The operation timed out. ---> System.Threading.Tasks.TaskCanceledException: A task was canceled. at Microsoft.Azure.Devices.Client.Transport.RetryDelegatingHandler.SendTwinPatchAsync(TwinCollection reportedProperties, CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.InternalClient.UpdateReportedPropertiesAsync(TwinCollection reportedProperties) --- End of inner exception stack trace --- at Microsoft.Azure.Devices.Client.InternalClient.UpdateReportedPropertiesAsync(TwinCollection reportedProperties) at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.<>c__DisplayClass30_0.<b__0>d.MoveNext() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 196 --- End of stack trace from previous location --- at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 189 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 189 at Microsoft.Azure.Devices.Edge.Util.TaskEx.TimeoutAfter(Task task, TimeSpan timeout, Action action) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/TaskEx.cs:line 160 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudProxy.UpdateReportedPropertiesAsync(IMessage reportedPropertiesMessage) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudProxy.cs:line 218 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudProxy.UpdateReportedPropertiesAsync(IMessage reportedPropertiesMessage) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudProxy.cs:line 227 at Microsoft.Azure.Devices.Edge.Hub.Core.Cloud.RetryingCloudProxy.<>c__DisplayClass24_0.<b__0>d.MoveNext() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.Core/cloud/RetryingCloudProxy.cs:line 65 --- End of stack trace from previous location --- at Microsoft.Azure.Devices.Edge.Hub.Core.Cloud.RetryingCloudProxy.ExecuteOperation[T](Func`2 func, String operation) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.Core/cloud/RetryingCloudProxy.cs:line 76 <6> 2023-11-30 16:23:16.061 +00:00 [INF] - Retrying cloud proxy operation UpdateReportedPropertiesAsync for usea1svlimtcld1-vm/$edgeHub. System.TimeoutException: The operation timed out. ---> System.Threading.Tasks.TaskCanceledException: A task was canceled. at Microsoft.Azure.Devices.Client.Transport.RetryDelegatingHandler.SendTwinPatchAsync(TwinCollection reportedProperties, CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.InternalClient.UpdateReportedPropertiesAsync(TwinCollection reportedProperties) --- End of inner exception stack trace --- at Microsoft.Azure.Devices.Client.InternalClient.UpdateReportedPropertiesAsync(TwinCollection reportedProperties) at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.<>c__DisplayClass30_0.<b__0>d.MoveNext() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 196 --- End of stack trace from previous location --- at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 189 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 189 at Microsoft.Azure.Devices.Edge.Util.TaskEx.TimeoutAfter(Task task, TimeSpan timeout, Action action) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/TaskEx.cs:line 160 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudProxy.UpdateReportedPropertiesAsync(IMessage reportedPropertiesMessage) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudProxy.cs:line 218 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudProxy.UpdateReportedPropertiesAsync(IMessage reportedPropertiesMessage) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudProxy.cs:line 227 at Microsoft.Azure.Devices.Edge.Hub.Core.Cloud.RetryingCloudProxy.<>c__DisplayClass24_0.<b__0>d.MoveNext() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.Core/cloud/RetryingCloudProxy.cs:line 65 --- End of stack trace from previous location --- at Microsoft.Azure.Devices.Edge.Hub.Core.Cloud.RetryingCloudProxy.ExecuteOperation[T](Func`2 func, String operation) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.Core/cloud/RetryingCloudProxy.cs:line 76 <6> 2023-11-30 16:24:00.777 +00:00 [INF] - Starting refresh of device scope identities cache <6> 2023-11-30 16:24:06.061 +00:00 [INF] - Closing receiver in cloud proxy db42d2e4-d6e5-492b-a499-e91fe7dd1f74 for usea1svlimtcld1-vm/$edgeHub <6> 2023-11-30 16:24:06.061 +00:00 [INF] - Closed cloud proxy db42d2e4-d6e5-492b-a499-e91fe7dd1f74 for usea1svlimtcld1-vm/$edgeHub <6> 2023-11-30 16:24:06.062 +00:00 [INF] - Retrying cloud proxy operation SendMessageAsync for usea1svlimtcld1-vm/plant_11. System.TimeoutException: The operation timed out. ---> System.Threading.Tasks.TaskCanceledException: A task was canceled. at Microsoft.Azure.Devices.Client.Transport.RetryDelegatingHandler.SendEventAsync(Message message, CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.InternalClient.SendEventAsync(Message message) --- End of inner exception stack trace --- at Microsoft.Azure.Devices.Client.InternalClient.SendEventAsync(Message message) at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.<>c__DisplayClass30_0.<b__0>d.MoveNext() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 196 --- End of stack trace from previous location --- at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 189 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 189 at Microsoft.Azure.Devices.Edge.Util.TaskEx.TimeoutAfter(Task task, TimeSpan timeout, Action action) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/TaskEx.cs:line 160 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudProxy.SendMessageAsync(IMessage inputMessage) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudProxy.cs:line 170 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudProxy.SendMessageAsync(IMessage inputMessage) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudProxy.cs:line 170 at Microsoft.Azure.Devices.Edge.Hub.Core.Cloud.RetryingCloudProxy.<>c__DisplayClass24_0.<b__0>d.MoveNext() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.Core/cloud/RetryingCloudProxy.cs:line 65 --- End of stack trace from previous location --- at Microsoft.Azure.Devices.Edge.Hub.Core.Cloud.RetryingCloudProxy.ExecuteOperation[T](Func`2 func, String operation) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.Core/cloud/RetryingCloudProxy.cs:line 76 <6> 2023-11-30 16:24:56.063 +00:00 [INF] - Closing receiver in cloud proxy f1c87086-8188-46ee-951a-58a724ad77ec for usea1svlimtcld1-vm/plant_11 <6> 2023-11-30 16:24:56.063 +00:00 [INF] - Closed cloud proxy f1c87086-8188-46ee-951a-58a724ad77ec for usea1svlimtcld1-vm/plant_11 <4> 2023-11-30 16:25:40.782 +00:00 [WRN] - Encountered an error while refreshing the device scope identities cache. Will retry the operation in some time... <6> 2023-11-30 16:25:40.782 +00:00 [INF] - Done refreshing device scope identities cache. Waiting for 60 minutes. <6> 2023-11-30 16:25:46.065 +00:00 [INF] - Closing receiver in cloud proxy f3155d9b-f499-4026-89a5-3fcdc1c96fba for usea1svlimtcld1-vm/mtc_helper <6> 2023-11-30 16:25:46.065 +00:00 [INF] - Closed cloud proxy f3155d9b-f499-4026-89a5-3fcdc1c96fba for usea1svlimtcld1-vm/mtc_helper <6> 2023-11-30 16:25:46.065 +00:00 [INF] - Attempting to connect to IoT Hub for client usea1svlimtcld1-vm/plant_11 via AMQP... <6> 2023-11-30 16:25:46.065 +00:00 [INF] - Attempting to connect to IoT Hub for client usea1svlimtcld1-vm/$edgeHub via AMQP... <6> 2023-11-30 16:25:47.470 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 16:25:47.472 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 1.1494ms" <4> 2023-11-30 16:27:46.068 +00:00 [WRN] - Error creating cloud connection for client usea1svlimtcld1-vm/plant_11 System.TimeoutException: Operation timed out at Microsoft.Azure.Devices.Edge.Util.TaskEx.<>c.<.cctor>b__27_0() in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/TaskEx.cs:line 114 at Microsoft.Azure.Devices.Edge.Util.TaskEx.TimeoutAfter(Task task, TimeSpan timeout, Action action) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/TaskEx.cs:line 160 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ModuleClientWrapper.OpenAsync() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ModuleClientWrapper.cs:line 65 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.<>c__DisplayClass30_0.<b__0>d.MoveNext() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 196 --- End of stack trace from previous location --- at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 189 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 189 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.OpenAsync() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 65 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.ConnectToIoTHub(ITokenProvider newTokenProvider) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 140 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.CreateNewCloudProxyAsync(ITokenProvider newTokenProvider) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 123 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.Create(IIdentity identity, Action`2 connectionStatusChangedHandler, ITransportSettings[] transportSettings, IMessageConverterProvider messageConverterProvider, IClientProvider clientProvider, ICloudListener cloudListener, ITokenProvider tokenProvider, TimeSpan idleTimeout, Boolean closeOnIdleTimeout, TimeSpan operationTimeout, TimeSpan cloudConnectionHangingTimeout, String productInfo, Option`1 modelId) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 104 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.TryCreateCloudConnectionFromServiceIdentity(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshOutOfDateCache, CloudListener cloudListener, String authChain) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 306 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.ConnectInternalWithDeviceStateTracking(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshCachedIdentity) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 267 <4> 2023-11-30 16:27:46.068 +00:00 [WRN] - Error creating cloud connection for client usea1svlimtcld1-vm/$edgeHub System.TimeoutException: Operation timed out at Microsoft.Azure.Devices.Edge.Util.TaskEx.<>c.<.cctor>b__27_0() in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/TaskEx.cs:line 114 at Microsoft.Azure.Devices.Edge.Util.TaskEx.TimeoutAfter(Task task, TimeSpan timeout, Action action) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/TaskEx.cs:line 160 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ModuleClientWrapper.OpenAsync() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ModuleClientWrapper.cs:line 65 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.<>c__DisplayClass30_0.<b__0>d.MoveNext() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 196 --- End of stack trace from previous location --- at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 189 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 189 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.OpenAsync() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 65 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.ConnectToIoTHub(ITokenProvider newTokenProvider) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 140 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.CreateNewCloudProxyAsync(ITokenProvider newTokenProvider) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 123 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.Create(IIdentity identity, Action`2 connectionStatusChangedHandler, ITransportSettings[] transportSettings, IMessageConverterProvider messageConverterProvider, IClientProvider clientProvider, ICloudListener cloudListener, ITokenProvider tokenProvider, TimeSpan idleTimeout, Boolean closeOnIdleTimeout, TimeSpan operationTimeout, TimeSpan cloudConnectionHangingTimeout, String productInfo, Option`1 modelId) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 104 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.TryCreateCloudConnectionFromServiceIdentity(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshOutOfDateCache, CloudListener cloudListener, String authChain) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 306 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.ConnectInternalWithDeviceStateTracking(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshCachedIdentity) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 267 <6> 2023-11-30 16:27:46.069 +00:00 [INF] - Error getting cloud connection for device usea1svlimtcld1-vm/plant_11 System.TimeoutException: Operation timed out at Microsoft.Azure.Devices.Edge.Util.TaskEx.<>c.<.cctor>b__27_0() in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/TaskEx.cs:line 114 at Microsoft.Azure.Devices.Edge.Util.TaskEx.TimeoutAfter(Task task, TimeSpan timeout, Action action) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/TaskEx.cs:line 160 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ModuleClientWrapper.OpenAsync() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ModuleClientWrapper.cs:line 65 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.<>c__DisplayClass30_0.<b__0>d.MoveNext() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 196 --- End of stack trace from previous location --- at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 189 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 189 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.OpenAsync() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 65 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.ConnectToIoTHub(ITokenProvider newTokenProvider) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 140 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.CreateNewCloudProxyAsync(ITokenProvider newTokenProvider) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 123 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.Create(IIdentity identity, Action`2 connectionStatusChangedHandler, ITransportSettings[] transportSettings, IMessageConverterProvider messageConverterProvider, IClientProvider clientProvider, ICloudListener cloudListener, ITokenProvider tokenProvider, TimeSpan idleTimeout, Boolean closeOnIdleTimeout, TimeSpan operationTimeout, TimeSpan cloudConnectionHangingTimeout, String productInfo, Option`1 modelId) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 104 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.TryCreateCloudConnectionFromServiceIdentity(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshOutOfDateCache, CloudListener cloudListener, String authChain) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 306 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.ConnectInternalWithDeviceStateTracking(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshCachedIdentity) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 267 <6> 2023-11-30 16:27:46.069 +00:00 [INF] - Error getting cloud connection for device usea1svlimtcld1-vm/$edgeHub System.TimeoutException: Operation timed out at Microsoft.Azure.Devices.Edge.Util.TaskEx.<>c.<.cctor>b__27_0() in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/TaskEx.cs:line 114 at Microsoft.Azure.Devices.Edge.Util.TaskEx.TimeoutAfter(Task task, TimeSpan timeout, Action action) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/TaskEx.cs:line 160 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ModuleClientWrapper.OpenAsync() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ModuleClientWrapper.cs:line 65 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.<>c__DisplayClass30_0.<b__0>d.MoveNext() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 196 --- End of stack trace from previous location --- at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 189 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 189 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.OpenAsync() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 65 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.ConnectToIoTHub(ITokenProvider newTokenProvider) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 140 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.CreateNewCloudProxyAsync(ITokenProvider newTokenProvider) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 123 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.Create(IIdentity identity, Action`2 connectionStatusChangedHandler, ITransportSettings[] transportSettings, IMessageConverterProvider messageConverterProvider, IClientProvider clientProvider, ICloudListener cloudListener, ITokenProvider tokenProvider, TimeSpan idleTimeout, Boolean closeOnIdleTimeout, TimeSpan operationTimeout, TimeSpan cloudConnectionHangingTimeout, String productInfo, Option`1 modelId) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 104 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.TryCreateCloudConnectionFromServiceIdentity(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshOutOfDateCache, CloudListener cloudListener, String authChain) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 306 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.ConnectInternalWithDeviceStateTracking(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshCachedIdentity) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 267 <6> 2023-11-30 16:27:46.069 +00:00 [INF] - Error getting cloud connection for device usea1svlimtcld1-vm/$edgeHub System.TimeoutException: Operation timed out at Microsoft.Azure.Devices.Edge.Util.TaskEx.<>c.<.cctor>b__27_0() in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/TaskEx.cs:line 114 at Microsoft.Azure.Devices.Edge.Util.TaskEx.TimeoutAfter(Task task, TimeSpan timeout, Action action) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/TaskEx.cs:line 160 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ModuleClientWrapper.OpenAsync() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ModuleClientWrapper.cs:line 65 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.<>c__DisplayClass30_0.<b__0>d.MoveNext() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 196 --- End of stack trace from previous location --- at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 189 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 189 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.OpenAsync() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 65 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.ConnectToIoTHub(ITokenProvider newTokenProvider) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 140 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.CreateNewCloudProxyAsync(ITokenProvider newTokenProvider) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 123 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.Create(IIdentity identity, Action`2 connectionStatusChangedHandler, ITransportSettings[] transportSettings, IMessageConverterProvider messageConverterProvider, IClientProvider clientProvider, ICloudListener cloudListener, ITokenProvider tokenProvider, TimeSpan idleTimeout, Boolean closeOnIdleTimeout, TimeSpan operationTimeout, TimeSpan cloudConnectionHangingTimeout, String productInfo, Option`1 modelId) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 104 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.TryCreateCloudConnectionFromServiceIdentity(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshOutOfDateCache, CloudListener cloudListener, String authChain) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 306 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.ConnectInternalWithDeviceStateTracking(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshCachedIdentity) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 267 <4> 2023-11-30 16:27:46.070 +00:00 [WRN] - Updating reported properties failed usea1svlimtcld1-vm/$edgeHub <6> 2023-11-30 16:27:46.073 +00:00 [INF] - Attempting to connect to IoT Hub for client usea1svlimtcld1-vm/$edgeHub via AMQP... <4> 2023-11-30 16:27:46.071 +00:00 [WRN] - Error calling IotHub for connectivity test Microsoft.Azure.Devices.Edge.Hub.Core.EdgeHubIOException: Unable to create IoTHub connection ---> System.TimeoutException: Operation timed out at Microsoft.Azure.Devices.Edge.Util.TaskEx.<>c.<.cctor>b__27_0() in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/TaskEx.cs:line 114 at Microsoft.Azure.Devices.Edge.Util.TaskEx.TimeoutAfter(Task task, TimeSpan timeout, Action action) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/TaskEx.cs:line 160 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ModuleClientWrapper.OpenAsync() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ModuleClientWrapper.cs:line 65 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.<>c__DisplayClass30_0.<b__0>d.MoveNext() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 196 --- End of stack trace from previous location --- at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 189 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 189 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.OpenAsync() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 65 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.ConnectToIoTHub(ITokenProvider newTokenProvider) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 140 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.CreateNewCloudProxyAsync(ITokenProvider newTokenProvider) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 123 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.Create(IIdentity identity, Action`2 connectionStatusChangedHandler, ITransportSettings[] transportSettings, IMessageConverterProvider messageConverterProvider, IClientProvider clientProvider, ICloudListener cloudListener, ITokenProvider tokenProvider, TimeSpan idleTimeout, Boolean closeOnIdleTimeout, TimeSpan operationTimeout, TimeSpan cloudConnectionHangingTimeout, String productInfo, Option`1 modelId) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 104 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.TryCreateCloudConnectionFromServiceIdentity(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshOutOfDateCache, CloudListener cloudListener, String authChain) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 306 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.ConnectInternalWithDeviceStateTracking(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshCachedIdentity) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 267 --- End of inner exception stack trace --- at Microsoft.Azure.Devices.Edge.Hub.Core.Cloud.RetryingCloudProxy.GetCloudProxy() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.Core/cloud/RetryingCloudProxy.cs:line 120 at Microsoft.Azure.Devices.Edge.Hub.Core.Cloud.RetryingCloudProxy.ExecuteOperation[T](Func`2 func, String operation) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.Core/cloud/RetryingCloudProxy.cs:line 73 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.DeviceConnectivityManager.ConnectivityChecker.Check() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/DeviceConnectivityManager.cs:line 220 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.DeviceConnectivityManager.CheckConnectivity() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/DeviceConnectivityManager.cs:line 190 <6> 2023-11-30 16:27:54.811 +00:00 [INF] - Attempting to connect to IoT Hub for client usea1svlimtcld1-vm/plant_11 via AMQP... <6> 2023-11-30 16:27:55.942 +00:00 [INF] - Entering periodic task to reauthenticate connected clients <6> 2023-11-30 16:27:55.942 +00:00 [INF] - Error authenticating token for usea1svlimtcld1-vm/plant_11 because the token is expired or could not be parsed <6> 2023-11-30 16:27:55.942 +00:00 [INF] - Credentials for client usea1svlimtcld1-vm/plant_11 are not valid. <4> 2023-11-30 16:27:55.942 +00:00 [WRN] - Reauthenticating client usea1svlimtcld1-vm/plant_11 failed, removing client connection <6> 2023-11-30 16:27:55.942 +00:00 [INF] - Unable to re-authenticate usea1svlimtcld1-vm/plant_11, dropping client connection. <6> 2023-11-30 16:27:55.942 +00:00 [INF] - Removing device connection for device usea1svlimtcld1-vm/plant_11 with removeCloudConnection flag 'True'. <6> 2023-11-30 16:27:55.942 +00:00 [INF] - Closing AMQP device proxy for usea1svlimtcld1-vm/plant_11 because no handler was registered. Microsoft.Azure.Devices.Edge.Hub.Core.EdgeHubConnectionException: Connection closed for device usea1svlimtcld1-vm/plant_11. <6> 2023-11-30 16:27:55.943 +00:00 [INF] - Closing link Events for usea1svlimtcld1-vm/plant_11 <6> 2023-11-30 16:27:55.943 +00:00 [INF] - Closing link MethodSending for usea1svlimtcld1-vm/plant_11 <6> 2023-11-30 16:27:55.943 +00:00 [INF] - Closing link MethodReceiving for usea1svlimtcld1-vm/plant_11 <6> 2023-11-30 16:27:55.943 +00:00 [INF] - Closing link TwinSending for usea1svlimtcld1-vm/plant_11 <6> 2023-11-30 16:27:55.943 +00:00 [INF] - Closing link TwinReceiving for usea1svlimtcld1-vm/plant_11 <6> 2023-11-30 16:27:55.948 +00:00 [INF] - Device connection removed for device usea1svlimtcld1-vm/plant_11 <6> 2023-11-30 16:27:55.948 +00:00 [INF] - Error authenticating token for usea1svlimtcld1-vm/mtc_helper because the token is expired or could not be parsed <6> 2023-11-30 16:27:55.948 +00:00 [INF] - Credentials for client usea1svlimtcld1-vm/mtc_helper are not valid. <4> 2023-11-30 16:27:55.948 +00:00 [WRN] - Reauthenticating client usea1svlimtcld1-vm/mtc_helper failed, removing client connection <6> 2023-11-30 16:27:55.948 +00:00 [INF] - Unable to re-authenticate usea1svlimtcld1-vm/mtc_helper, dropping client connection. <6> 2023-11-30 16:27:55.948 +00:00 [INF] - Removing device connection for device usea1svlimtcld1-vm/mtc_helper with removeCloudConnection flag 'True'. <6> 2023-11-30 16:27:55.948 +00:00 [INF] - Closing AMQP device proxy for usea1svlimtcld1-vm/mtc_helper because no handler was registered. Microsoft.Azure.Devices.Edge.Hub.Core.EdgeHubConnectionException: Connection closed for device usea1svlimtcld1-vm/mtc_helper. <6> 2023-11-30 16:27:55.955 +00:00 [INF] - Closing link Events for usea1svlimtcld1-vm/mtc_helper <6> 2023-11-30 16:27:55.956 +00:00 [INF] - Closing link MethodSending for usea1svlimtcld1-vm/mtc_helper <6> 2023-11-30 16:27:55.956 +00:00 [INF] - Closing link MethodReceiving for usea1svlimtcld1-vm/mtc_helper <6> 2023-11-30 16:27:55.956 +00:00 [INF] - Closing link TwinSending for usea1svlimtcld1-vm/mtc_helper <6> 2023-11-30 16:27:55.956 +00:00 [INF] - Closing link TwinReceiving for usea1svlimtcld1-vm/mtc_helper <6> 2023-11-30 16:27:55.956 +00:00 [INF] - Device connection removed for device usea1svlimtcld1-vm/mtc_helper <4> 2023-11-30 16:29:46.073 +00:00 [WRN] - Error creating cloud connection for client usea1svlimtcld1-vm/$edgeHub System.TimeoutException: Operation timed out at Microsoft.Azure.Devices.Edge.Util.TaskEx.<>c.<.cctor>b__27_0() in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/TaskEx.cs:line 114 at Microsoft.Azure.Devices.Edge.Util.TaskEx.TimeoutAfter(Task task, TimeSpan timeout, Action action) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/TaskEx.cs:line 160 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ModuleClientWrapper.OpenAsync() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ModuleClientWrapper.cs:line 65 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.<>c__DisplayClass30_0.<b__0>d.MoveNext() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 196 --- End of stack trace from previous location --- at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 189 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 189 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.OpenAsync() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 65 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.ConnectToIoTHub(ITokenProvider newTokenProvider) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 140 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.CreateNewCloudProxyAsync(ITokenProvider newTokenProvider) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 123 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.Create(IIdentity identity, Action`2 connectionStatusChangedHandler, ITransportSettings[] transportSettings, IMessageConverterProvider messageConverterProvider, IClientProvider clientProvider, ICloudListener cloudListener, ITokenProvider tokenProvider, TimeSpan idleTimeout, Boolean closeOnIdleTimeout, TimeSpan operationTimeout, TimeSpan cloudConnectionHangingTimeout, String productInfo, Option`1 modelId) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 104 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.TryCreateCloudConnectionFromServiceIdentity(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshOutOfDateCache, CloudListener cloudListener, String authChain) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 306 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.ConnectInternalWithDeviceStateTracking(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshCachedIdentity) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 267 <6> 2023-11-30 16:29:46.073 +00:00 [INF] - Error getting cloud connection for device usea1svlimtcld1-vm/$edgeHub System.TimeoutException: Operation timed out at Microsoft.Azure.Devices.Edge.Util.TaskEx.<>c.<.cctor>b__27_0() in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/TaskEx.cs:line 114 at Microsoft.Azure.Devices.Edge.Util.TaskEx.TimeoutAfter(Task task, TimeSpan timeout, Action action) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/TaskEx.cs:line 160 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ModuleClientWrapper.OpenAsync() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ModuleClientWrapper.cs:line 65 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.<>c__DisplayClass30_0.<b__0>d.MoveNext() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 196 --- End of stack trace from previous location --- at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 189 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 189 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.OpenAsync() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 65 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.ConnectToIoTHub(ITokenProvider newTokenProvider) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 140 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.CreateNewCloudProxyAsync(ITokenProvider newTokenProvider) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 123 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.Create(IIdentity identity, Action`2 connectionStatusChangedHandler, ITransportSettings[] transportSettings, IMessageConverterProvider messageConverterProvider, IClientProvider clientProvider, ICloudListener cloudListener, ITokenProvider tokenProvider, TimeSpan idleTimeout, Boolean closeOnIdleTimeout, TimeSpan operationTimeout, TimeSpan cloudConnectionHangingTimeout, String productInfo, Option`1 modelId) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 104 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.TryCreateCloudConnectionFromServiceIdentity(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshOutOfDateCache, CloudListener cloudListener, String authChain) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 306 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.ConnectInternalWithDeviceStateTracking(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshCachedIdentity) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 267 <6> 2023-11-30 16:29:46.073 +00:00 [INF] - Error getting cloud connection for device usea1svlimtcld1-vm/$edgeHub System.TimeoutException: Operation timed out at Microsoft.Azure.Devices.Edge.Util.TaskEx.<>c.<.cctor>b__27_0() in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/TaskEx.cs:line 114 at Microsoft.Azure.Devices.Edge.Util.TaskEx.TimeoutAfter(Task task, TimeSpan timeout, Action action) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/TaskEx.cs:line 160 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ModuleClientWrapper.OpenAsync() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ModuleClientWrapper.cs:line 65 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.<>c__DisplayClass30_0.<b__0>d.MoveNext() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 196 --- End of stack trace from previous location --- at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 189 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 189 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.OpenAsync() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 65 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.ConnectToIoTHub(ITokenProvider newTokenProvider) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 140 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.CreateNewCloudProxyAsync(ITokenProvider newTokenProvider) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 123 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.Create(IIdentity identity, Action`2 connectionStatusChangedHandler, ITransportSettings[] transportSettings, IMessageConverterProvider messageConverterProvider, IClientProvider clientProvider, ICloudListener cloudListener, ITokenProvider tokenProvider, TimeSpan idleTimeout, Boolean closeOnIdleTimeout, TimeSpan operationTimeout, TimeSpan cloudConnectionHangingTimeout, String productInfo, Option`1 modelId) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 104 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.TryCreateCloudConnectionFromServiceIdentity(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshOutOfDateCache, CloudListener cloudListener, String authChain) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 306 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.ConnectInternalWithDeviceStateTracking(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshCachedIdentity) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 267 <4> 2023-11-30 16:29:46.074 +00:00 [WRN] - Updating reported properties failed usea1svlimtcld1-vm/$edgeHub <6> 2023-11-30 16:29:46.074 +00:00 [INF] - Attempting to connect to IoT Hub for client usea1svlimtcld1-vm/$edgeHub via AMQP... <4> 2023-11-30 16:29:54.810 +00:00 [WRN] - Error creating cloud connection for client usea1svlimtcld1-vm/plant_11 System.TimeoutException: Operation timed out at Microsoft.Azure.Devices.Edge.Util.TaskEx.<>c.<.cctor>b__27_0() in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/TaskEx.cs:line 114 at Microsoft.Azure.Devices.Edge.Util.TaskEx.TimeoutAfter(Task task, TimeSpan timeout, Action action) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/TaskEx.cs:line 160 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ModuleClientWrapper.OpenAsync() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ModuleClientWrapper.cs:line 65 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.<>c__DisplayClass30_0.<b__0>d.MoveNext() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 196 --- End of stack trace from previous location --- at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 189 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 189 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.OpenAsync() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 65 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.ConnectToIoTHub(ITokenProvider newTokenProvider) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 140 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.CreateNewCloudProxyAsync(ITokenProvider newTokenProvider) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 123 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.Create(IIdentity identity, Action`2 connectionStatusChangedHandler, ITransportSettings[] transportSettings, IMessageConverterProvider messageConverterProvider, IClientProvider clientProvider, ICloudListener cloudListener, ITokenProvider tokenProvider, TimeSpan idleTimeout, Boolean closeOnIdleTimeout, TimeSpan operationTimeout, TimeSpan cloudConnectionHangingTimeout, String productInfo, Option`1 modelId) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 104 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.TryCreateCloudConnectionFromServiceIdentity(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshOutOfDateCache, CloudListener cloudListener, String authChain) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 306 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.ConnectInternalWithDeviceStateTracking(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshCachedIdentity) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 267 <6> 2023-11-30 16:29:54.810 +00:00 [INF] - Error getting cloud connection for device usea1svlimtcld1-vm/plant_11 System.TimeoutException: Operation timed out at Microsoft.Azure.Devices.Edge.Util.TaskEx.<>c.<.cctor>b__27_0() in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/TaskEx.cs:line 114 at Microsoft.Azure.Devices.Edge.Util.TaskEx.TimeoutAfter(Task task, TimeSpan timeout, Action action) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/TaskEx.cs:line 160 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ModuleClientWrapper.OpenAsync() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ModuleClientWrapper.cs:line 65 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.<>c__DisplayClass30_0.<b__0>d.MoveNext() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 196 --- End of stack trace from previous location --- at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 189 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 189 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.OpenAsync() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 65 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.ConnectToIoTHub(ITokenProvider newTokenProvider) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 140 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.CreateNewCloudProxyAsync(ITokenProvider newTokenProvider) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 123 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.Create(IIdentity identity, Action`2 connectionStatusChangedHandler, ITransportSettings[] transportSettings, IMessageConverterProvider messageConverterProvider, IClientProvider clientProvider, ICloudListener cloudListener, ITokenProvider tokenProvider, TimeSpan idleTimeout, Boolean closeOnIdleTimeout, TimeSpan operationTimeout, TimeSpan cloudConnectionHangingTimeout, String productInfo, Option`1 modelId) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 104 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.TryCreateCloudConnectionFromServiceIdentity(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshOutOfDateCache, CloudListener cloudListener, String authChain) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 306 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.ConnectInternalWithDeviceStateTracking(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshCachedIdentity) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 267 <4> 2023-11-30 16:29:54.811 +00:00 [WRN] - Could not get an active Iot Hub connection for client usea1svlimtcld1-vm/plant_11 <6> 2023-11-30 16:30:13.349 +00:00 [INF] - Attempting to connect to IoT Hub for client usea1svlimtcld1-vm/plant_11 via AMQP... <4> 2023-11-30 16:31:46.074 +00:00 [WRN] - Error creating cloud connection for client usea1svlimtcld1-vm/$edgeHub System.TimeoutException: Operation timed out at Microsoft.Azure.Devices.Edge.Util.TaskEx.<>c.<.cctor>b__27_0() in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/TaskEx.cs:line 114 at Microsoft.Azure.Devices.Edge.Util.TaskEx.TimeoutAfter(Task task, TimeSpan timeout, Action action) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/TaskEx.cs:line 160 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ModuleClientWrapper.OpenAsync() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ModuleClientWrapper.cs:line 65 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.<>c__DisplayClass30_0.<b__0>d.MoveNext() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 196 --- End of stack trace from previous location --- at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 189 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 189 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.OpenAsync() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 65 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.ConnectToIoTHub(ITokenProvider newTokenProvider) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 140 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.CreateNewCloudProxyAsync(ITokenProvider newTokenProvider) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 123 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.Create(IIdentity identity, Action`2 connectionStatusChangedHandler, ITransportSettings[] transportSettings, IMessageConverterProvider messageConverterProvider, IClientProvider clientProvider, ICloudListener cloudListener, ITokenProvider tokenProvider, TimeSpan idleTimeout, Boolean closeOnIdleTimeout, TimeSpan operationTimeout, TimeSpan cloudConnectionHangingTimeout, String productInfo, Option`1 modelId) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 104 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.TryCreateCloudConnectionFromServiceIdentity(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshOutOfDateCache, CloudListener cloudListener, String authChain) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 306 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.ConnectInternalWithDeviceStateTracking(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshCachedIdentity) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 267 <6> 2023-11-30 16:31:46.074 +00:00 [INF] - Error getting cloud connection for device usea1svlimtcld1-vm/$edgeHub System.TimeoutException: Operation timed out at Microsoft.Azure.Devices.Edge.Util.TaskEx.<>c.<.cctor>b__27_0() in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/TaskEx.cs:line 114 at Microsoft.Azure.Devices.Edge.Util.TaskEx.TimeoutAfter(Task task, TimeSpan timeout, Action action) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/TaskEx.cs:line 160 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ModuleClientWrapper.OpenAsync() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ModuleClientWrapper.cs:line 65 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.<>c__DisplayClass30_0.<b__0>d.MoveNext() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 196 --- End of stack trace from previous location --- at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 189 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 189 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.OpenAsync() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 65 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.ConnectToIoTHub(ITokenProvider newTokenProvider) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 140 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.CreateNewCloudProxyAsync(ITokenProvider newTokenProvider) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 123 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.Create(IIdentity identity, Action`2 connectionStatusChangedHandler, ITransportSettings[] transportSettings, IMessageConverterProvider messageConverterProvider, IClientProvider clientProvider, ICloudListener cloudListener, ITokenProvider tokenProvider, TimeSpan idleTimeout, Boolean closeOnIdleTimeout, TimeSpan operationTimeout, TimeSpan cloudConnectionHangingTimeout, String productInfo, Option`1 modelId) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 104 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.TryCreateCloudConnectionFromServiceIdentity(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshOutOfDateCache, CloudListener cloudListener, String authChain) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 306 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.ConnectInternalWithDeviceStateTracking(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshCachedIdentity) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 267 <4> 2023-11-30 16:31:46.075 +00:00 [WRN] - Updating reported properties failed usea1svlimtcld1-vm/$edgeHub <6> 2023-11-30 16:31:46.075 +00:00 [INF] - Error getting cloud connection for device usea1svlimtcld1-vm/$edgeHub System.TimeoutException: Operation timed out at Microsoft.Azure.Devices.Edge.Util.TaskEx.<>c.<.cctor>b__27_0() in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/TaskEx.cs:line 114 at Microsoft.Azure.Devices.Edge.Util.TaskEx.TimeoutAfter(Task task, TimeSpan timeout, Action action) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/TaskEx.cs:line 160 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ModuleClientWrapper.OpenAsync() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ModuleClientWrapper.cs:line 65 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.<>c__DisplayClass30_0.<b__0>d.MoveNext() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 196 --- End of stack trace from previous location --- at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 189 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 189 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.OpenAsync() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 65 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.ConnectToIoTHub(ITokenProvider newTokenProvider) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 140 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.CreateNewCloudProxyAsync(ITokenProvider newTokenProvider) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 123 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.Create(IIdentity identity, Action`2 connectionStatusChangedHandler, ITransportSettings[] transportSettings, IMessageConverterProvider messageConverterProvider, IClientProvider clientProvider, ICloudListener cloudListener, ITokenProvider tokenProvider, TimeSpan idleTimeout, Boolean closeOnIdleTimeout, TimeSpan operationTimeout, TimeSpan cloudConnectionHangingTimeout, String productInfo, Option`1 modelId) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 104 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.TryCreateCloudConnectionFromServiceIdentity(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshOutOfDateCache, CloudListener cloudListener, String authChain) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 306 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.ConnectInternalWithDeviceStateTracking(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshCachedIdentity) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 267 <6> 2023-11-30 16:31:47.499 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 16:31:47.502 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 2.9106ms" <4> 2023-11-30 16:32:13.347 +00:00 [WRN] - Error creating cloud connection for client usea1svlimtcld1-vm/plant_11 System.TimeoutException: Operation timed out at Microsoft.Azure.Devices.Edge.Util.TaskEx.<>c.<.cctor>b__27_0() in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/TaskEx.cs:line 114 at Microsoft.Azure.Devices.Edge.Util.TaskEx.TimeoutAfter(Task task, TimeSpan timeout, Action action) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/TaskEx.cs:line 160 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ModuleClientWrapper.OpenAsync() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ModuleClientWrapper.cs:line 65 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.<>c__DisplayClass30_0.<b__0>d.MoveNext() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 196 --- End of stack trace from previous location --- at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 189 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 189 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.OpenAsync() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 65 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.ConnectToIoTHub(ITokenProvider newTokenProvider) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 140 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.CreateNewCloudProxyAsync(ITokenProvider newTokenProvider) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 123 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.Create(IIdentity identity, Action`2 connectionStatusChangedHandler, ITransportSettings[] transportSettings, IMessageConverterProvider messageConverterProvider, IClientProvider clientProvider, ICloudListener cloudListener, ITokenProvider tokenProvider, TimeSpan idleTimeout, Boolean closeOnIdleTimeout, TimeSpan operationTimeout, TimeSpan cloudConnectionHangingTimeout, String productInfo, Option`1 modelId) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 104 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.TryCreateCloudConnectionFromServiceIdentity(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshOutOfDateCache, CloudListener cloudListener, String authChain) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 306 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.ConnectInternalWithDeviceStateTracking(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshCachedIdentity) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 267 <6> 2023-11-30 16:32:13.348 +00:00 [INF] - Error getting cloud connection for device usea1svlimtcld1-vm/plant_11 System.TimeoutException: Operation timed out at Microsoft.Azure.Devices.Edge.Util.TaskEx.<>c.<.cctor>b__27_0() in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/TaskEx.cs:line 114 at Microsoft.Azure.Devices.Edge.Util.TaskEx.TimeoutAfter(Task task, TimeSpan timeout, Action action) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/TaskEx.cs:line 160 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ModuleClientWrapper.OpenAsync() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ModuleClientWrapper.cs:line 65 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.<>c__DisplayClass30_0.<b__0>d.MoveNext() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 196 --- End of stack trace from previous location --- at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 189 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 189 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.OpenAsync() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 65 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.ConnectToIoTHub(ITokenProvider newTokenProvider) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 140 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.CreateNewCloudProxyAsync(ITokenProvider newTokenProvider) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 123 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.Create(IIdentity identity, Action`2 connectionStatusChangedHandler, ITransportSettings[] transportSettings, IMessageConverterProvider messageConverterProvider, IClientProvider clientProvider, ICloudListener cloudListener, ITokenProvider tokenProvider, TimeSpan idleTimeout, Boolean closeOnIdleTimeout, TimeSpan operationTimeout, TimeSpan cloudConnectionHangingTimeout, String productInfo, Option`1 modelId) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 104 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.TryCreateCloudConnectionFromServiceIdentity(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshOutOfDateCache, CloudListener cloudListener, String authChain) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 306 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.ConnectInternalWithDeviceStateTracking(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshCachedIdentity) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 267 <4> 2023-11-30 16:32:13.348 +00:00 [WRN] - Could not get an active Iot Hub connection for client usea1svlimtcld1-vm/plant_11 <6> 2023-11-30 16:32:44.826 +00:00 [INF] - Attempting to connect to IoT Hub for client usea1svlimtcld1-vm/plant_11 via AMQP... <6> 2023-11-30 16:32:55.940 +00:00 [INF] - Entering periodic task to reauthenticate connected clients <6> 2023-11-30 16:34:13.350 +00:00 [INF] - Attempting to connect to IoT Hub for client usea1svlimtcld1-vm/$edgeHub via AMQP... <4> 2023-11-30 16:34:44.825 +00:00 [WRN] - Error creating cloud connection for client usea1svlimtcld1-vm/plant_11 System.TimeoutException: Operation timed out at Microsoft.Azure.Devices.Edge.Util.TaskEx.<>c.<.cctor>b__27_0() in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/TaskEx.cs:line 114 at Microsoft.Azure.Devices.Edge.Util.TaskEx.TimeoutAfter(Task task, TimeSpan timeout, Action action) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/TaskEx.cs:line 160 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ModuleClientWrapper.OpenAsync() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ModuleClientWrapper.cs:line 65 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.<>c__DisplayClass30_0.<b__0>d.MoveNext() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 196 --- End of stack trace from previous location --- at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 189 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 189 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.OpenAsync() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 65 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.ConnectToIoTHub(ITokenProvider newTokenProvider) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 140 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.CreateNewCloudProxyAsync(ITokenProvider newTokenProvider) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 123 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.Create(IIdentity identity, Action`2 connectionStatusChangedHandler, ITransportSettings[] transportSettings, IMessageConverterProvider messageConverterProvider, IClientProvider clientProvider, ICloudListener cloudListener, ITokenProvider tokenProvider, TimeSpan idleTimeout, Boolean closeOnIdleTimeout, TimeSpan operationTimeout, TimeSpan cloudConnectionHangingTimeout, String productInfo, Option`1 modelId) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 104 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.TryCreateCloudConnectionFromServiceIdentity(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshOutOfDateCache, CloudListener cloudListener, String authChain) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 306 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.ConnectInternalWithDeviceStateTracking(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshCachedIdentity) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 267 <6> 2023-11-30 16:34:44.826 +00:00 [INF] - Error getting cloud connection for device usea1svlimtcld1-vm/plant_11 System.TimeoutException: Operation timed out at Microsoft.Azure.Devices.Edge.Util.TaskEx.<>c.<.cctor>b__27_0() in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/TaskEx.cs:line 114 at Microsoft.Azure.Devices.Edge.Util.TaskEx.TimeoutAfter(Task task, TimeSpan timeout, Action action) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/TaskEx.cs:line 160 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ModuleClientWrapper.OpenAsync() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ModuleClientWrapper.cs:line 65 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.<>c__DisplayClass30_0.<b__0>d.MoveNext() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 196 --- End of stack trace from previous location --- at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 189 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 189 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.OpenAsync() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 65 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.ConnectToIoTHub(ITokenProvider newTokenProvider) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 140 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.CreateNewCloudProxyAsync(ITokenProvider newTokenProvider) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 123 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.Create(IIdentity identity, Action`2 connectionStatusChangedHandler, ITransportSettings[] transportSettings, IMessageConverterProvider messageConverterProvider, IClientProvider clientProvider, ICloudListener cloudListener, ITokenProvider tokenProvider, TimeSpan idleTimeout, Boolean closeOnIdleTimeout, TimeSpan operationTimeout, TimeSpan cloudConnectionHangingTimeout, String productInfo, Option`1 modelId) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 104 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.TryCreateCloudConnectionFromServiceIdentity(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshOutOfDateCache, CloudListener cloudListener, String authChain) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 306 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.ConnectInternalWithDeviceStateTracking(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshCachedIdentity) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 267 <4> 2023-11-30 16:34:44.827 +00:00 [WRN] - Could not get an active Iot Hub connection for client usea1svlimtcld1-vm/plant_11 <6> 2023-11-30 16:35:44.829 +00:00 [INF] - Attempting to connect to IoT Hub for client usea1svlimtcld1-vm/plant_11 via AMQP... <4> 2023-11-30 16:36:13.351 +00:00 [WRN] - Error creating cloud connection for client usea1svlimtcld1-vm/$edgeHub System.TimeoutException: Operation timed out at Microsoft.Azure.Devices.Edge.Util.TaskEx.<>c.<.cctor>b__27_0() in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/TaskEx.cs:line 114 at Microsoft.Azure.Devices.Edge.Util.TaskEx.TimeoutAfter(Task task, TimeSpan timeout, Action action) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/TaskEx.cs:line 160 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ModuleClientWrapper.OpenAsync() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ModuleClientWrapper.cs:line 65 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.<>c__DisplayClass30_0.<b__0>d.MoveNext() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 196 --- End of stack trace from previous location --- at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 189 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 189 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.OpenAsync() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 65 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.ConnectToIoTHub(ITokenProvider newTokenProvider) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 140 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.CreateNewCloudProxyAsync(ITokenProvider newTokenProvider) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 123 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.Create(IIdentity identity, Action`2 connectionStatusChangedHandler, ITransportSettings[] transportSettings, IMessageConverterProvider messageConverterProvider, IClientProvider clientProvider, ICloudListener cloudListener, ITokenProvider tokenProvider, TimeSpan idleTimeout, Boolean closeOnIdleTimeout, TimeSpan operationTimeout, TimeSpan cloudConnectionHangingTimeout, String productInfo, Option`1 modelId) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 104 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.TryCreateCloudConnectionFromServiceIdentity(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshOutOfDateCache, CloudListener cloudListener, String authChain) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 306 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.ConnectInternalWithDeviceStateTracking(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshCachedIdentity) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 267 <6> 2023-11-30 16:36:13.352 +00:00 [INF] - Error getting cloud connection for device usea1svlimtcld1-vm/$edgeHub System.TimeoutException: Operation timed out at Microsoft.Azure.Devices.Edge.Util.TaskEx.<>c.<.cctor>b__27_0() in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/TaskEx.cs:line 114 at Microsoft.Azure.Devices.Edge.Util.TaskEx.TimeoutAfter(Task task, TimeSpan timeout, Action action) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/TaskEx.cs:line 160 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ModuleClientWrapper.OpenAsync() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ModuleClientWrapper.cs:line 65 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.<>c__DisplayClass30_0.<b__0>d.MoveNext() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 196 --- End of stack trace from previous location --- at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 189 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 189 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.OpenAsync() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 65 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.ConnectToIoTHub(ITokenProvider newTokenProvider) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 140 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.CreateNewCloudProxyAsync(ITokenProvider newTokenProvider) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 123 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.Create(IIdentity identity, Action`2 connectionStatusChangedHandler, ITransportSettings[] transportSettings, IMessageConverterProvider messageConverterProvider, IClientProvider clientProvider, ICloudListener cloudListener, ITokenProvider tokenProvider, TimeSpan idleTimeout, Boolean closeOnIdleTimeout, TimeSpan operationTimeout, TimeSpan cloudConnectionHangingTimeout, String productInfo, Option`1 modelId) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 104 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.TryCreateCloudConnectionFromServiceIdentity(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshOutOfDateCache, CloudListener cloudListener, String authChain) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 306 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.ConnectInternalWithDeviceStateTracking(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshCachedIdentity) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 267 <6> 2023-11-30 16:37:37.948 +00:00 [INF] - "Request starting HTTP/1.1 GET http://172.18.0.6:9600/metrics - -" <6> 2023-11-30 16:37:37.949 +00:00 [INF] - "Request finished HTTP/1.1 GET http://172.18.0.6:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 0.7523ms" <6> 2023-11-30 16:37:42.972 +00:00 [INF] - Starting compaction of stores <6> 2023-11-30 16:37:42.972 +00:00 [INF] - Starting compaction of store checkpoints <6> 2023-11-30 16:37:42.972 +00:00 [INF] - Starting compaction of store sessions <6> 2023-11-30 16:37:42.972 +00:00 [INF] - Starting compaction of store messages <6> 2023-11-30 16:37:42.972 +00:00 [INF] - Starting compaction of store default <6> 2023-11-30 16:37:42.972 +00:00 [INF] - Starting compaction of store iothub <6> 2023-11-30 16:37:42.972 +00:00 [INF] - Starting compaction of store twins <6> 2023-11-30 16:37:42.972 +00:00 [INF] - Starting compaction of store EdgeTwin <6> 2023-11-30 16:37:42.972 +00:00 [INF] - Starting compaction of store DeviceScopeCache <6> 2023-11-30 16:37:42.972 +00:00 [INF] - Starting compaction of store MetadataStore <6> 2023-11-30 16:37:42.972 +00:00 [INF] - Starting compaction of store iothub_Pri2 <4> 2023-11-30 16:37:44.829 +00:00 [WRN] - Error creating cloud connection for client usea1svlimtcld1-vm/plant_11 System.TimeoutException: Operation timed out at Microsoft.Azure.Devices.Edge.Util.TaskEx.<>c.<.cctor>b__27_0() in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/TaskEx.cs:line 114 at Microsoft.Azure.Devices.Edge.Util.TaskEx.TimeoutAfter(Task task, TimeSpan timeout, Action action) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/TaskEx.cs:line 160 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ModuleClientWrapper.OpenAsync() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ModuleClientWrapper.cs:line 65 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.<>c__DisplayClass30_0.<b__0>d.MoveNext() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 196 --- End of stack trace from previous location --- at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 189 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 189 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.OpenAsync() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 65 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.ConnectToIoTHub(ITokenProvider newTokenProvider) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 140 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.CreateNewCloudProxyAsync(ITokenProvider newTokenProvider) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 123 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.Create(IIdentity identity, Action`2 connectionStatusChangedHandler, ITransportSettings[] transportSettings, IMessageConverterProvider messageConverterProvider, IClientProvider clientProvider, ICloudListener cloudListener, ITokenProvider tokenProvider, TimeSpan idleTimeout, Boolean closeOnIdleTimeout, TimeSpan operationTimeout, TimeSpan cloudConnectionHangingTimeout, String productInfo, Option`1 modelId) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 104 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.TryCreateCloudConnectionFromServiceIdentity(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshOutOfDateCache, CloudListener cloudListener, String authChain) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 306 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.ConnectInternalWithDeviceStateTracking(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshCachedIdentity) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 267 <6> 2023-11-30 16:37:44.833 +00:00 [INF] - Error getting cloud connection for device usea1svlimtcld1-vm/plant_11 System.TimeoutException: Operation timed out at Microsoft.Azure.Devices.Edge.Util.TaskEx.<>c.<.cctor>b__27_0() in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/TaskEx.cs:line 114 at Microsoft.Azure.Devices.Edge.Util.TaskEx.TimeoutAfter(Task task, TimeSpan timeout, Action action) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/TaskEx.cs:line 160 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ModuleClientWrapper.OpenAsync() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ModuleClientWrapper.cs:line 65 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.<>c__DisplayClass30_0.<b__0>d.MoveNext() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 196 --- End of stack trace from previous location --- at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 189 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 189 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.OpenAsync() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 65 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.ConnectToIoTHub(ITokenProvider newTokenProvider) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 140 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.CreateNewCloudProxyAsync(ITokenProvider newTokenProvider) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 123 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.Create(IIdentity identity, Action`2 connectionStatusChangedHandler, ITransportSettings[] transportSettings, IMessageConverterProvider messageConverterProvider, IClientProvider clientProvider, ICloudListener cloudListener, ITokenProvider tokenProvider, TimeSpan idleTimeout, Boolean closeOnIdleTimeout, TimeSpan operationTimeout, TimeSpan cloudConnectionHangingTimeout, String productInfo, Option`1 modelId) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 104 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.TryCreateCloudConnectionFromServiceIdentity(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshOutOfDateCache, CloudListener cloudListener, String authChain) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 306 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.ConnectInternalWithDeviceStateTracking(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshCachedIdentity) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 267 <4> 2023-11-30 16:37:44.834 +00:00 [WRN] - Could not get an active Iot Hub connection for client usea1svlimtcld1-vm/plant_11 <6> 2023-11-30 16:37:47.554 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 16:37:47.555 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 0.9094ms" <6> 2023-11-30 16:37:54.344 +00:00 [INF] - Started task to cleanup processed and stale messages for endpoint iothub <6> 2023-11-30 16:37:54.808 +00:00 [INF] - Cleaned up 2523 messages from queue for endpoint iothub and 2523 messages from message store. <6> 2023-11-30 16:37:54.808 +00:00 [INF] - Started task to cleanup processed and stale messages for endpoint iothub_Pri2 <6> 2023-11-30 16:37:54.808 +00:00 [INF] - Cleaned up 0 messages from queue for endpoint iothub_Pri2 and 0 messages from message store. <6> 2023-11-30 16:37:55.940 +00:00 [INF] - Entering periodic task to reauthenticate connected clients <6> 2023-11-30 16:38:44.834 +00:00 [INF] - Attempting to connect to IoT Hub for client usea1svlimtcld1-vm/plant_11 via AMQP... <6> 2023-11-30 16:39:44.829 +00:00 [INF] - Attempting to connect to IoT Hub for client usea1svlimtcld1-vm/$edgeHub via AMQP... <4> 2023-11-30 16:40:44.833 +00:00 [WRN] - Error creating cloud connection for client usea1svlimtcld1-vm/plant_11 System.TimeoutException: Operation timed out at Microsoft.Azure.Devices.Edge.Util.TaskEx.<>c.<.cctor>b__27_0() in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/TaskEx.cs:line 114 at Microsoft.Azure.Devices.Edge.Util.TaskEx.TimeoutAfter(Task task, TimeSpan timeout, Action action) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/TaskEx.cs:line 160 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ModuleClientWrapper.OpenAsync() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ModuleClientWrapper.cs:line 65 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.<>c__DisplayClass30_0.<b__0>d.MoveNext() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 196 --- End of stack trace from previous location --- at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 189 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 189 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.OpenAsync() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 65 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.ConnectToIoTHub(ITokenProvider newTokenProvider) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 140 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.CreateNewCloudProxyAsync(ITokenProvider newTokenProvider) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 123 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.Create(IIdentity identity, Action`2 connectionStatusChangedHandler, ITransportSettings[] transportSettings, IMessageConverterProvider messageConverterProvider, IClientProvider clientProvider, ICloudListener cloudListener, ITokenProvider tokenProvider, TimeSpan idleTimeout, Boolean closeOnIdleTimeout, TimeSpan operationTimeout, TimeSpan cloudConnectionHangingTimeout, String productInfo, Option`1 modelId) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 104 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.TryCreateCloudConnectionFromServiceIdentity(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshOutOfDateCache, CloudListener cloudListener, String authChain) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 306 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.ConnectInternalWithDeviceStateTracking(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshCachedIdentity) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 267 <6> 2023-11-30 16:40:44.834 +00:00 [INF] - Error getting cloud connection for device usea1svlimtcld1-vm/plant_11 System.TimeoutException: Operation timed out at Microsoft.Azure.Devices.Edge.Util.TaskEx.<>c.<.cctor>b__27_0() in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/TaskEx.cs:line 114 at Microsoft.Azure.Devices.Edge.Util.TaskEx.TimeoutAfter(Task task, TimeSpan timeout, Action action) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/TaskEx.cs:line 160 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ModuleClientWrapper.OpenAsync() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ModuleClientWrapper.cs:line 65 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.<>c__DisplayClass30_0.<b__0>d.MoveNext() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 196 --- End of stack trace from previous location --- at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 189 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 189 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.OpenAsync() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 65 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.ConnectToIoTHub(ITokenProvider newTokenProvider) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 140 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.CreateNewCloudProxyAsync(ITokenProvider newTokenProvider) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 123 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.Create(IIdentity identity, Action`2 connectionStatusChangedHandler, ITransportSettings[] transportSettings, IMessageConverterProvider messageConverterProvider, IClientProvider clientProvider, ICloudListener cloudListener, ITokenProvider tokenProvider, TimeSpan idleTimeout, Boolean closeOnIdleTimeout, TimeSpan operationTimeout, TimeSpan cloudConnectionHangingTimeout, String productInfo, Option`1 modelId) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 104 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.TryCreateCloudConnectionFromServiceIdentity(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshOutOfDateCache, CloudListener cloudListener, String authChain) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 306 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.ConnectInternalWithDeviceStateTracking(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshCachedIdentity) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 267 <4> 2023-11-30 16:40:44.834 +00:00 [WRN] - Could not get an active Iot Hub connection for client usea1svlimtcld1-vm/plant_11 <4> 2023-11-30 16:41:44.836 +00:00 [WRN] - Error creating cloud connection for client usea1svlimtcld1-vm/$edgeHub System.TimeoutException: Operation timed out at Microsoft.Azure.Devices.Edge.Util.TaskEx.<>c.<.cctor>b__27_0() in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/TaskEx.cs:line 114 at Microsoft.Azure.Devices.Edge.Util.TaskEx.TimeoutAfter(Task task, TimeSpan timeout, Action action) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/TaskEx.cs:line 160 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ModuleClientWrapper.OpenAsync() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ModuleClientWrapper.cs:line 65 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.<>c__DisplayClass30_0.<b__0>d.MoveNext() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 196 --- End of stack trace from previous location --- at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 189 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 189 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.OpenAsync() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 65 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.ConnectToIoTHub(ITokenProvider newTokenProvider) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 140 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.CreateNewCloudProxyAsync(ITokenProvider newTokenProvider) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 123 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.Create(IIdentity identity, Action`2 connectionStatusChangedHandler, ITransportSettings[] transportSettings, IMessageConverterProvider messageConverterProvider, IClientProvider clientProvider, ICloudListener cloudListener, ITokenProvider tokenProvider, TimeSpan idleTimeout, Boolean closeOnIdleTimeout, TimeSpan operationTimeout, TimeSpan cloudConnectionHangingTimeout, String productInfo, Option`1 modelId) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 104 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.TryCreateCloudConnectionFromServiceIdentity(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshOutOfDateCache, CloudListener cloudListener, String authChain) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 306 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.ConnectInternalWithDeviceStateTracking(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshCachedIdentity) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 267 <6> 2023-11-30 16:41:44.836 +00:00 [INF] - Error getting cloud connection for device usea1svlimtcld1-vm/$edgeHub System.TimeoutException: Operation timed out at Microsoft.Azure.Devices.Edge.Util.TaskEx.<>c.<.cctor>b__27_0() in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/TaskEx.cs:line 114 at Microsoft.Azure.Devices.Edge.Util.TaskEx.TimeoutAfter(Task task, TimeSpan timeout, Action action) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/TaskEx.cs:line 160 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ModuleClientWrapper.OpenAsync() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ModuleClientWrapper.cs:line 65 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.<>c__DisplayClass30_0.<b__0>d.MoveNext() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 196 --- End of stack trace from previous location --- at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 189 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 189 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.OpenAsync() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 65 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.ConnectToIoTHub(ITokenProvider newTokenProvider) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 140 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.CreateNewCloudProxyAsync(ITokenProvider newTokenProvider) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 123 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.Create(IIdentity identity, Action`2 connectionStatusChangedHandler, ITransportSettings[] transportSettings, IMessageConverterProvider messageConverterProvider, IClientProvider clientProvider, ICloudListener cloudListener, ITokenProvider tokenProvider, TimeSpan idleTimeout, Boolean closeOnIdleTimeout, TimeSpan operationTimeout, TimeSpan cloudConnectionHangingTimeout, String productInfo, Option`1 modelId) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 104 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.TryCreateCloudConnectionFromServiceIdentity(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshOutOfDateCache, CloudListener cloudListener, String authChain) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 306 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.ConnectInternalWithDeviceStateTracking(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshCachedIdentity) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 267 <6> 2023-11-30 16:41:44.839 +00:00 [INF] - Attempting to connect to IoT Hub for client usea1svlimtcld1-vm/plant_11 via AMQP... <6> 2023-11-30 16:42:55.941 +00:00 [INF] - Entering periodic task to reauthenticate connected clients <4> 2023-11-30 16:43:44.839 +00:00 [WRN] - Error creating cloud connection for client usea1svlimtcld1-vm/plant_11 System.TimeoutException: Operation timed out at Microsoft.Azure.Devices.Edge.Util.TaskEx.<>c.<.cctor>b__27_0() in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/TaskEx.cs:line 114 at Microsoft.Azure.Devices.Edge.Util.TaskEx.TimeoutAfter(Task task, TimeSpan timeout, Action action) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/TaskEx.cs:line 160 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ModuleClientWrapper.OpenAsync() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ModuleClientWrapper.cs:line 65 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.<>c__DisplayClass30_0.<b__0>d.MoveNext() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 196 --- End of stack trace from previous location --- at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 189 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 189 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.OpenAsync() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 65 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.ConnectToIoTHub(ITokenProvider newTokenProvider) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 140 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.CreateNewCloudProxyAsync(ITokenProvider newTokenProvider) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 123 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.Create(IIdentity identity, Action`2 connectionStatusChangedHandler, ITransportSettings[] transportSettings, IMessageConverterProvider messageConverterProvider, IClientProvider clientProvider, ICloudListener cloudListener, ITokenProvider tokenProvider, TimeSpan idleTimeout, Boolean closeOnIdleTimeout, TimeSpan operationTimeout, TimeSpan cloudConnectionHangingTimeout, String productInfo, Option`1 modelId) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 104 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.TryCreateCloudConnectionFromServiceIdentity(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshOutOfDateCache, CloudListener cloudListener, String authChain) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 306 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.ConnectInternalWithDeviceStateTracking(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshCachedIdentity) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 267 <6> 2023-11-30 16:43:44.840 +00:00 [INF] - Error getting cloud connection for device usea1svlimtcld1-vm/plant_11 System.TimeoutException: Operation timed out at Microsoft.Azure.Devices.Edge.Util.TaskEx.<>c.<.cctor>b__27_0() in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/TaskEx.cs:line 114 at Microsoft.Azure.Devices.Edge.Util.TaskEx.TimeoutAfter(Task task, TimeSpan timeout, Action action) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/TaskEx.cs:line 160 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ModuleClientWrapper.OpenAsync() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ModuleClientWrapper.cs:line 65 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.<>c__DisplayClass30_0.<b__0>d.MoveNext() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 196 --- End of stack trace from previous location --- at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 189 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 189 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.OpenAsync() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 65 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.ConnectToIoTHub(ITokenProvider newTokenProvider) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 140 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.CreateNewCloudProxyAsync(ITokenProvider newTokenProvider) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 123 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.Create(IIdentity identity, Action`2 connectionStatusChangedHandler, ITransportSettings[] transportSettings, IMessageConverterProvider messageConverterProvider, IClientProvider clientProvider, ICloudListener cloudListener, ITokenProvider tokenProvider, TimeSpan idleTimeout, Boolean closeOnIdleTimeout, TimeSpan operationTimeout, TimeSpan cloudConnectionHangingTimeout, String productInfo, Option`1 modelId) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 104 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.TryCreateCloudConnectionFromServiceIdentity(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshOutOfDateCache, CloudListener cloudListener, String authChain) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 306 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.ConnectInternalWithDeviceStateTracking(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshCachedIdentity) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 267 <4> 2023-11-30 16:43:44.841 +00:00 [WRN] - Could not get an active Iot Hub connection for client usea1svlimtcld1-vm/plant_11 <6> 2023-11-30 16:43:47.586 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 16:43:47.587 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 1.0910ms" <6> 2023-11-30 16:44:44.842 +00:00 [INF] - Attempting to connect to IoT Hub for client usea1svlimtcld1-vm/plant_11 via AMQP... <6> 2023-11-30 16:45:44.840 +00:00 [INF] - Attempting to connect to IoT Hub for client usea1svlimtcld1-vm/$edgeHub via AMQP... <4> 2023-11-30 16:46:44.842 +00:00 [WRN] - Error creating cloud connection for client usea1svlimtcld1-vm/plant_11 System.TimeoutException: Operation timed out at Microsoft.Azure.Devices.Edge.Util.TaskEx.<>c.<.cctor>b__27_0() in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/TaskEx.cs:line 114 at Microsoft.Azure.Devices.Edge.Util.TaskEx.TimeoutAfter(Task task, TimeSpan timeout, Action action) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/TaskEx.cs:line 160 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ModuleClientWrapper.OpenAsync() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ModuleClientWrapper.cs:line 65 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.<>c__DisplayClass30_0.<b__0>d.MoveNext() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 196 --- End of stack trace from previous location --- at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 189 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 189 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.OpenAsync() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 65 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.ConnectToIoTHub(ITokenProvider newTokenProvider) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 140 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.CreateNewCloudProxyAsync(ITokenProvider newTokenProvider) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 123 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.Create(IIdentity identity, Action`2 connectionStatusChangedHandler, ITransportSettings[] transportSettings, IMessageConverterProvider messageConverterProvider, IClientProvider clientProvider, ICloudListener cloudListener, ITokenProvider tokenProvider, TimeSpan idleTimeout, Boolean closeOnIdleTimeout, TimeSpan operationTimeout, TimeSpan cloudConnectionHangingTimeout, String productInfo, Option`1 modelId) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 104 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.TryCreateCloudConnectionFromServiceIdentity(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshOutOfDateCache, CloudListener cloudListener, String authChain) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 306 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.ConnectInternalWithDeviceStateTracking(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshCachedIdentity) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 267 <6> 2023-11-30 16:46:44.843 +00:00 [INF] - Error getting cloud connection for device usea1svlimtcld1-vm/plant_11 System.TimeoutException: Operation timed out at Microsoft.Azure.Devices.Edge.Util.TaskEx.<>c.<.cctor>b__27_0() in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/TaskEx.cs:line 114 at Microsoft.Azure.Devices.Edge.Util.TaskEx.TimeoutAfter(Task task, TimeSpan timeout, Action action) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/TaskEx.cs:line 160 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ModuleClientWrapper.OpenAsync() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ModuleClientWrapper.cs:line 65 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.<>c__DisplayClass30_0.<b__0>d.MoveNext() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 196 --- End of stack trace from previous location --- at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 189 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 189 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.OpenAsync() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 65 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.ConnectToIoTHub(ITokenProvider newTokenProvider) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 140 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.CreateNewCloudProxyAsync(ITokenProvider newTokenProvider) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 123 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.Create(IIdentity identity, Action`2 connectionStatusChangedHandler, ITransportSettings[] transportSettings, IMessageConverterProvider messageConverterProvider, IClientProvider clientProvider, ICloudListener cloudListener, ITokenProvider tokenProvider, TimeSpan idleTimeout, Boolean closeOnIdleTimeout, TimeSpan operationTimeout, TimeSpan cloudConnectionHangingTimeout, String productInfo, Option`1 modelId) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 104 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.TryCreateCloudConnectionFromServiceIdentity(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshOutOfDateCache, CloudListener cloudListener, String authChain) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 306 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.ConnectInternalWithDeviceStateTracking(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshCachedIdentity) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 267 <4> 2023-11-30 16:46:44.843 +00:00 [WRN] - Could not get an active Iot Hub connection for client usea1svlimtcld1-vm/plant_11 <4> 2023-11-30 16:47:24.066 +00:00 [WRN] - Error creating cloud connection for client usea1svlimtcld1-vm/$edgeHub Microsoft.Azure.Devices.Client.Exceptions.IotHubCommunicationException: Transient network error occurred, please retry. ---> System.IO.IOException: Unable to read data from the transport connection: Connection reset by peer. ---> System.Net.Sockets.SocketException (104): Connection reset by peer --- End of inner exception stack trace --- at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken) at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token) at System.IO.BufferedStream.ReadFromUnderlyingStreamAsync(Memory`1 buffer, CancellationToken cancellationToken, Int32 bytesAlreadySatisfied, Task semaphoreLockTask) at Microsoft.Azure.Devices.Edge.Util.Uds.HttpBufferedStream.ReadLineAsync(CancellationToken cancellationToken) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/uds/HttpBufferedStream.cs:line 62 at Microsoft.Azure.Devices.Edge.Util.Uds.HttpRequestResponseSerializer.SetResponseStatusLine(HttpResponseMessage httpResponse, HttpBufferedStream bufferedStream, CancellationToken cancellationToken) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/uds/HttpRequestResponseSerializer.cs:line 158 at Microsoft.Azure.Devices.Edge.Util.Uds.HttpRequestResponseSerializer.DeserializeResponse(HttpBufferedStream bufferedStream, CancellationToken cancellationToken) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/uds/HttpRequestResponseSerializer.cs:line 69 at Microsoft.Azure.Devices.Edge.Util.Uds.HttpUdsMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/uds/HttpUdsMessageHandler.cs:line 48 at System.Net.Http.HttpClient.g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken) at Microsoft.Azure.Devices.Edge.Util.Edged.Version_2019_01_30.GeneratedCode.HttpWorkloadClient.SignAsync(String api_version, String name, String genid, SignRequest payload, CancellationToken cancellationToken) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/edged/version_2019_01_30/generatedCode/HttpWorkloadClient.cs:line 106 at Microsoft.Azure.Devices.Edge.Util.TaskEx.TimeoutAfter[T](Task`1 task, TimeSpan timeout, Action action) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/TaskEx.cs:line 137 at Microsoft.Azure.Devices.Edge.Util.Edged.WorkloadClientVersioned.Execute[T](Func`1 func, String operation) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/edged/WorkloadClientVersioned.cs:line 81 at Microsoft.Azure.Devices.Edge.Util.Edged.WorkloadClientVersioned.Execute[T](Func`1 func, String operation) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/edged/WorkloadClientVersioned.cs:line 81 at Microsoft.Azure.Devices.Edge.Util.Edged.Version_2019_01_30.WorkloadClient.SignAsync(String keyId, String algorithm, String data) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/edged/version_2019_01_30/WorkloadClient.cs:line 101 at Microsoft.Azure.Devices.Edge.Util.ClientTokenProvider.GetTokenAsyncInternal(DateTime startTime, TimeSpan ttl) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/ClientTokenProvider.cs:line 89 at Microsoft.Azure.Devices.Edge.Util.ClientTokenProvider.<>c__DisplayClass9_0.<b__3>d.MoveNext() in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/ClientTokenProvider.cs:line 62 --- End of stack trace from previous location --- at Microsoft.Azure.Devices.Edge.Util.ClientTokenProvider.<>c__DisplayClass9_0.<b__1>d.MoveNext() in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/ClientTokenProvider.cs:line 64 --- End of stack trace from previous location --- at Microsoft.Azure.Devices.Edge.Util.ClientTokenProvider.GetTokenAsync(Option`1 ttl) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/ClientTokenProvider.cs:line 65 at Microsoft.Azure.Devices.Client.AuthenticationWithTokenRefresh.GetTokenAsync(String iotHub) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpIoTCbsTokenProvider.GetTokenAsync(Uri namespaceAddress, String appliesTo, String[] requiredClaims) at Microsoft.Azure.Amqp.TaskHelpers.EndAsyncResult(IAsyncResult asyncResult) at Microsoft.Azure.Amqp.IteratorAsyncResult`1.StepCallback(IAsyncResult result) --- End of stack trace from previous location --- at Microsoft.Azure.Amqp.AsyncResult.End[TAsyncResult](IAsyncResult result) at Microsoft.Azure.Amqp.AmqpCbsLink.<>c.b__4_1(IAsyncResult r) at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization) --- End of stack trace from previous location --- at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpIoTCbsLink.SendTokenAsync(ICbsTokenProvider tokenProvider, Uri namespaceAddress, String audience, String resource, String[] requiredClaims, TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpAuthenticationRefresher.InitLoopAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpIoTConnection.CreateRefresherAsync(DeviceIdentity deviceIdentity, TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpConnectionHolder.CreateRefresherAsync(DeviceIdentity deviceIdentity, TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpUnit.EnsureSessionIsOpenAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpUnit.EnsureSessionIsOpenAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpUnit.OpenAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpTransportHandler.OpenAsync(CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.Transport.ProtocolRoutingDelegatingHandler.OpenAsync(CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.<>c__DisplayClass27_0.<b__0>d.MoveNext() --- End of stack trace from previous location --- at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.ExecuteWithErrorHandlingAsync[T](Func`1 asyncOperation) --- End of inner exception stack trace --- at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.ExecuteWithErrorHandlingAsync[T](Func`1 asyncOperation) at Microsoft.Azure.Devices.Client.Transport.RetryDelegatingHandler.<>c__DisplayClass44_0.<b__0>d.MoveNext() --- End of stack trace from previous location --- at Microsoft.Azure.Devices.Client.Transport.RetryDelegatingHandler.OpenInternalAsync(Boolean withRetry, CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.Transport.RetryDelegatingHandler.EnsureOpenedAsync(Boolean withRetry, CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.InternalClient.OpenAsync() at Microsoft.Azure.Devices.Edge.Util.TaskEx.TimeoutAfter(Task task, TimeSpan timeout, Action action) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/TaskEx.cs:line 160 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ModuleClientWrapper.OpenAsync() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ModuleClientWrapper.cs:line 65 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.<>c__DisplayClass30_0.<b__0>d.MoveNext() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 196 --- End of stack trace from previous location --- at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 189 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 189 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.OpenAsync() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 65 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.ConnectToIoTHub(ITokenProvider newTokenProvider) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 140 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.CreateNewCloudProxyAsync(ITokenProvider newTokenProvider) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 123 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.Create(IIdentity identity, Action`2 connectionStatusChangedHandler, ITransportSettings[] transportSettings, IMessageConverterProvider messageConverterProvider, IClientProvider clientProvider, ICloudListener cloudListener, ITokenProvider tokenProvider, TimeSpan idleTimeout, Boolean closeOnIdleTimeout, TimeSpan operationTimeout, TimeSpan cloudConnectionHangingTimeout, String productInfo, Option`1 modelId) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 104 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.TryCreateCloudConnectionFromServiceIdentity(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshOutOfDateCache, CloudListener cloudListener, String authChain) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 306 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.ConnectInternalWithDeviceStateTracking(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshCachedIdentity) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 267 <6> 2023-11-30 16:47:24.069 +00:00 [INF] - Error getting cloud connection for device usea1svlimtcld1-vm/$edgeHub Microsoft.Azure.Devices.Client.Exceptions.IotHubCommunicationException: Transient network error occurred, please retry. ---> System.IO.IOException: Unable to read data from the transport connection: Connection reset by peer. ---> System.Net.Sockets.SocketException (104): Connection reset by peer --- End of inner exception stack trace --- at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken) at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token) at System.IO.BufferedStream.ReadFromUnderlyingStreamAsync(Memory`1 buffer, CancellationToken cancellationToken, Int32 bytesAlreadySatisfied, Task semaphoreLockTask) at Microsoft.Azure.Devices.Edge.Util.Uds.HttpBufferedStream.ReadLineAsync(CancellationToken cancellationToken) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/uds/HttpBufferedStream.cs:line 62 at Microsoft.Azure.Devices.Edge.Util.Uds.HttpRequestResponseSerializer.SetResponseStatusLine(HttpResponseMessage httpResponse, HttpBufferedStream bufferedStream, CancellationToken cancellationToken) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/uds/HttpRequestResponseSerializer.cs:line 158 at Microsoft.Azure.Devices.Edge.Util.Uds.HttpRequestResponseSerializer.DeserializeResponse(HttpBufferedStream bufferedStream, CancellationToken cancellationToken) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/uds/HttpRequestResponseSerializer.cs:line 69 at Microsoft.Azure.Devices.Edge.Util.Uds.HttpUdsMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/uds/HttpUdsMessageHandler.cs:line 48 at System.Net.Http.HttpClient.g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken) at Microsoft.Azure.Devices.Edge.Util.Edged.Version_2019_01_30.GeneratedCode.HttpWorkloadClient.SignAsync(String api_version, String name, String genid, SignRequest payload, CancellationToken cancellationToken) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/edged/version_2019_01_30/generatedCode/HttpWorkloadClient.cs:line 106 at Microsoft.Azure.Devices.Edge.Util.TaskEx.TimeoutAfter[T](Task`1 task, TimeSpan timeout, Action action) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/TaskEx.cs:line 137 at Microsoft.Azure.Devices.Edge.Util.Edged.WorkloadClientVersioned.Execute[T](Func`1 func, String operation) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/edged/WorkloadClientVersioned.cs:line 81 at Microsoft.Azure.Devices.Edge.Util.Edged.WorkloadClientVersioned.Execute[T](Func`1 func, String operation) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/edged/WorkloadClientVersioned.cs:line 81 at Microsoft.Azure.Devices.Edge.Util.Edged.Version_2019_01_30.WorkloadClient.SignAsync(String keyId, String algorithm, String data) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/edged/version_2019_01_30/WorkloadClient.cs:line 101 at Microsoft.Azure.Devices.Edge.Util.ClientTokenProvider.GetTokenAsyncInternal(DateTime startTime, TimeSpan ttl) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/ClientTokenProvider.cs:line 89 at Microsoft.Azure.Devices.Edge.Util.ClientTokenProvider.<>c__DisplayClass9_0.<b__3>d.MoveNext() in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/ClientTokenProvider.cs:line 62 --- End of stack trace from previous location --- at Microsoft.Azure.Devices.Edge.Util.ClientTokenProvider.<>c__DisplayClass9_0.<b__1>d.MoveNext() in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/ClientTokenProvider.cs:line 64 --- End of stack trace from previous location --- at Microsoft.Azure.Devices.Edge.Util.ClientTokenProvider.GetTokenAsync(Option`1 ttl) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/ClientTokenProvider.cs:line 65 at Microsoft.Azure.Devices.Client.AuthenticationWithTokenRefresh.GetTokenAsync(String iotHub) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpIoTCbsTokenProvider.GetTokenAsync(Uri namespaceAddress, String appliesTo, String[] requiredClaims) at Microsoft.Azure.Amqp.TaskHelpers.EndAsyncResult(IAsyncResult asyncResult) at Microsoft.Azure.Amqp.IteratorAsyncResult`1.StepCallback(IAsyncResult result) --- End of stack trace from previous location --- at Microsoft.Azure.Amqp.AsyncResult.End[TAsyncResult](IAsyncResult result) at Microsoft.Azure.Amqp.AmqpCbsLink.<>c.b__4_1(IAsyncResult r) at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization) --- End of stack trace from previous location --- at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpIoTCbsLink.SendTokenAsync(ICbsTokenProvider tokenProvider, Uri namespaceAddress, String audience, String resource, String[] requiredClaims, TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpAuthenticationRefresher.InitLoopAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpIoTConnection.CreateRefresherAsync(DeviceIdentity deviceIdentity, TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpConnectionHolder.CreateRefresherAsync(DeviceIdentity deviceIdentity, TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpUnit.EnsureSessionIsOpenAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpUnit.EnsureSessionIsOpenAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpUnit.OpenAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpTransportHandler.OpenAsync(CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.Transport.ProtocolRoutingDelegatingHandler.OpenAsync(CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.<>c__DisplayClass27_0.<b__0>d.MoveNext() --- End of stack trace from previous location --- at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.ExecuteWithErrorHandlingAsync[T](Func`1 asyncOperation) --- End of inner exception stack trace --- at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.ExecuteWithErrorHandlingAsync[T](Func`1 asyncOperation) at Microsoft.Azure.Devices.Client.Transport.RetryDelegatingHandler.<>c__DisplayClass44_0.<b__0>d.MoveNext() --- End of stack trace from previous location --- at Microsoft.Azure.Devices.Client.Transport.RetryDelegatingHandler.OpenInternalAsync(Boolean withRetry, CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.Transport.RetryDelegatingHandler.EnsureOpenedAsync(Boolean withRetry, CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.InternalClient.OpenAsync() at Microsoft.Azure.Devices.Edge.Util.TaskEx.TimeoutAfter(Task task, TimeSpan timeout, Action action) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/TaskEx.cs:line 160 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ModuleClientWrapper.OpenAsync() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ModuleClientWrapper.cs:line 65 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.<>c__DisplayClass30_0.<b__0>d.MoveNext() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 196 --- End of stack trace from previous location --- at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 189 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 189 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.OpenAsync() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 65 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.ConnectToIoTHub(ITokenProvider newTokenProvider) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 140 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.CreateNewCloudProxyAsync(ITokenProvider newTokenProvider) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 123 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.Create(IIdentity identity, Action`2 connectionStatusChangedHandler, ITransportSettings[] transportSettings, IMessageConverterProvider messageConverterProvider, IClientProvider clientProvider, ICloudListener cloudListener, ITokenProvider tokenProvider, TimeSpan idleTimeout, Boolean closeOnIdleTimeout, TimeSpan operationTimeout, TimeSpan cloudConnectionHangingTimeout, String productInfo, Option`1 modelId) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 104 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.TryCreateCloudConnectionFromServiceIdentity(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshOutOfDateCache, CloudListener cloudListener, String authChain) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 306 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.ConnectInternalWithDeviceStateTracking(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshCachedIdentity) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 267 <6> 2023-11-30 16:47:44.845 +00:00 [INF] - Attempting to connect to IoT Hub for client usea1svlimtcld1-vm/plant_11 via AMQP... <6> 2023-11-30 16:47:55.940 +00:00 [INF] - Entering periodic task to reauthenticate connected clients <6> 2023-11-30 16:48:44.842 +00:00 [INF] - Attempting to connect to IoT Hub for client usea1svlimtcld1-vm/$edgeHub via AMQP... <4> 2023-11-30 16:49:33.854 +00:00 [WRN] - Error creating cloud connection for client usea1svlimtcld1-vm/plant_11 System.TimeoutException: The operation timed out. ---> Microsoft.Azure.Devices.Client.Exceptions.IotHubCommunicationException: Transient network error occurred, please retry. ---> System.Threading.Tasks.TaskCanceledException: A task was canceled. at Microsoft.Azure.Amqp.TaskHelpers.EndAsyncResult(IAsyncResult asyncResult) at Microsoft.Azure.Amqp.IteratorAsyncResult`1.StepCallback(IAsyncResult result) --- End of stack trace from previous location --- at Microsoft.Azure.Amqp.AsyncResult.End[TAsyncResult](IAsyncResult result) at Microsoft.Azure.Amqp.AmqpCbsLink.<>c.b__4_1(IAsyncResult r) at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization) --- End of stack trace from previous location --- at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpIoTCbsLink.SendTokenAsync(ICbsTokenProvider tokenProvider, Uri namespaceAddress, String audience, String resource, String[] requiredClaims, TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpAuthenticationRefresher.InitLoopAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpIoTConnection.CreateRefresherAsync(DeviceIdentity deviceIdentity, TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpConnectionHolder.CreateRefresherAsync(DeviceIdentity deviceIdentity, TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpUnit.EnsureSessionIsOpenAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpUnit.EnsureSessionIsOpenAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpUnit.OpenAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpTransportHandler.OpenAsync(CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.Transport.ProtocolRoutingDelegatingHandler.OpenAsync(CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.<>c__DisplayClass27_0.<b__0>d.MoveNext() --- End of stack trace from previous location --- at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.ExecuteWithErrorHandlingAsync[T](Func`1 asyncOperation) --- End of inner exception stack trace --- at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.ExecuteWithErrorHandlingAsync[T](Func`1 asyncOperation) at Microsoft.Azure.Devices.Client.Transport.RetryDelegatingHandler.<>c__DisplayClass44_0.<b__0>d.MoveNext() --- End of stack trace from previous location --- at Microsoft.Azure.Devices.Client.Transport.RetryDelegatingHandler.OpenInternalAsync(Boolean withRetry, CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.Transport.RetryDelegatingHandler.EnsureOpenedAsync(Boolean withRetry, CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.InternalClient.OpenAsync() --- End of inner exception stack trace --- at Microsoft.Azure.Devices.Client.InternalClient.OpenAsync() at Microsoft.Azure.Devices.Edge.Util.TaskEx.TimeoutAfter(Task task, TimeSpan timeout, Action action) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/TaskEx.cs:line 160 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ModuleClientWrapper.OpenAsync() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ModuleClientWrapper.cs:line 65 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.<>c__DisplayClass30_0.<b__0>d.MoveNext() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 196 --- End of stack trace from previous location --- at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 189 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 189 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.OpenAsync() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 65 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.ConnectToIoTHub(ITokenProvider newTokenProvider) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 140 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.CreateNewCloudProxyAsync(ITokenProvider newTokenProvider) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 123 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.Create(IIdentity identity, Action`2 connectionStatusChangedHandler, ITransportSettings[] transportSettings, IMessageConverterProvider messageConverterProvider, IClientProvider clientProvider, ICloudListener cloudListener, ITokenProvider tokenProvider, TimeSpan idleTimeout, Boolean closeOnIdleTimeout, TimeSpan operationTimeout, TimeSpan cloudConnectionHangingTimeout, String productInfo, Option`1 modelId) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 104 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.TryCreateCloudConnectionFromServiceIdentity(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshOutOfDateCache, CloudListener cloudListener, String authChain) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 306 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.ConnectInternalWithDeviceStateTracking(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshCachedIdentity) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 267 <6> 2023-11-30 16:49:33.858 +00:00 [INF] - Error getting cloud connection for device usea1svlimtcld1-vm/plant_11 System.TimeoutException: The operation timed out. ---> Microsoft.Azure.Devices.Client.Exceptions.IotHubCommunicationException: Transient network error occurred, please retry. ---> System.Threading.Tasks.TaskCanceledException: A task was canceled. at Microsoft.Azure.Amqp.TaskHelpers.EndAsyncResult(IAsyncResult asyncResult) at Microsoft.Azure.Amqp.IteratorAsyncResult`1.StepCallback(IAsyncResult result) --- End of stack trace from previous location --- at Microsoft.Azure.Amqp.AsyncResult.End[TAsyncResult](IAsyncResult result) at Microsoft.Azure.Amqp.AmqpCbsLink.<>c.b__4_1(IAsyncResult r) at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization) --- End of stack trace from previous location --- at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpIoTCbsLink.SendTokenAsync(ICbsTokenProvider tokenProvider, Uri namespaceAddress, String audience, String resource, String[] requiredClaims, TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpAuthenticationRefresher.InitLoopAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpIoTConnection.CreateRefresherAsync(DeviceIdentity deviceIdentity, TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpConnectionHolder.CreateRefresherAsync(DeviceIdentity deviceIdentity, TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpUnit.EnsureSessionIsOpenAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpUnit.EnsureSessionIsOpenAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpUnit.OpenAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpTransportHandler.OpenAsync(CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.Transport.ProtocolRoutingDelegatingHandler.OpenAsync(CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.<>c__DisplayClass27_0.<b__0>d.MoveNext() --- End of stack trace from previous location --- at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.ExecuteWithErrorHandlingAsync[T](Func`1 asyncOperation) --- End of inner exception stack trace --- at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.ExecuteWithErrorHandlingAsync[T](Func`1 asyncOperation) at Microsoft.Azure.Devices.Client.Transport.RetryDelegatingHandler.<>c__DisplayClass44_0.<b__0>d.MoveNext() --- End of stack trace from previous location --- at Microsoft.Azure.Devices.Client.Transport.RetryDelegatingHandler.OpenInternalAsync(Boolean withRetry, CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.Transport.RetryDelegatingHandler.EnsureOpenedAsync(Boolean withRetry, CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.InternalClient.OpenAsync() --- End of inner exception stack trace --- at Microsoft.Azure.Devices.Client.InternalClient.OpenAsync() at Microsoft.Azure.Devices.Edge.Util.TaskEx.TimeoutAfter(Task task, TimeSpan timeout, Action action) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/TaskEx.cs:line 160 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ModuleClientWrapper.OpenAsync() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ModuleClientWrapper.cs:line 65 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.<>c__DisplayClass30_0.<b__0>d.MoveNext() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 196 --- End of stack trace from previous location --- at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 189 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 189 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.OpenAsync() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 65 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.ConnectToIoTHub(ITokenProvider newTokenProvider) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 140 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.CreateNewCloudProxyAsync(ITokenProvider newTokenProvider) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 123 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.Create(IIdentity identity, Action`2 connectionStatusChangedHandler, ITransportSettings[] transportSettings, IMessageConverterProvider messageConverterProvider, IClientProvider clientProvider, ICloudListener cloudListener, ITokenProvider tokenProvider, TimeSpan idleTimeout, Boolean closeOnIdleTimeout, TimeSpan operationTimeout, TimeSpan cloudConnectionHangingTimeout, String productInfo, Option`1 modelId) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 104 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.TryCreateCloudConnectionFromServiceIdentity(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshOutOfDateCache, CloudListener cloudListener, String authChain) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 306 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.ConnectInternalWithDeviceStateTracking(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshCachedIdentity) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 267 <4> 2023-11-30 16:49:33.860 +00:00 [WRN] - Could not get an active Iot Hub connection for client usea1svlimtcld1-vm/plant_11 <4> 2023-11-30 16:49:33.859 +00:00 [WRN] - Error creating cloud connection for client usea1svlimtcld1-vm/$edgeHub Microsoft.Azure.Devices.Client.Exceptions.IotHubCommunicationException: Transient network error occurred, please retry. ---> System.IO.IOException: Unable to write data to the transport connection: Broken pipe. ---> System.Net.Sockets.SocketException (32): Broken pipe at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.CreateException(SocketError error, Boolean forAsyncThrow) at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.SendAsyncForNetworkStream(Socket socket, CancellationToken cancellationToken) at System.Net.Sockets.NetworkStream.WriteAsync(ReadOnlyMemory`1 buffer, CancellationToken cancellationToken) at System.IO.BufferedStream.FlushWriteAsync(CancellationToken cancellationToken) at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine) at System.IO.BufferedStream.FlushWriteAsync(CancellationToken cancellationToken) at System.IO.BufferedStream.ReadFromUnderlyingStreamAsync(Memory`1 buffer, CancellationToken cancellationToken, Int32 bytesAlreadySatisfied, Task semaphoreLockTask) at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine) at System.IO.BufferedStream.ReadFromUnderlyingStreamAsync(Memory`1 buffer, CancellationToken cancellationToken, Int32 bytesAlreadySatisfied, Task semaphoreLockTask) at System.IO.BufferedStream.ReadAsync(Byte[] buffer, Int32 offset, Int32 count, CancellationToken cancellationToken) at Microsoft.Azure.Devices.Edge.Util.Uds.HttpBufferedStream.ReadLineAsync(CancellationToken cancellationToken) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/uds/HttpBufferedStream.cs:line 62 at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine) at Microsoft.Azure.Devices.Edge.Util.Uds.HttpBufferedStream.ReadLineAsync(CancellationToken cancellationToken) at Microsoft.Azure.Devices.Edge.Util.Uds.HttpRequestResponseSerializer.SetResponseStatusLine(HttpResponseMessage httpResponse, HttpBufferedStream bufferedStream, CancellationToken cancellationToken) at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine) at Microsoft.Azure.Devices.Edge.Util.Uds.HttpRequestResponseSerializer.SetResponseStatusLine(HttpResponseMessage httpResponse, HttpBufferedStream bufferedStream, CancellationToken cancellationToken) at Microsoft.Azure.Devices.Edge.Util.Uds.HttpRequestResponseSerializer.DeserializeResponse(HttpBufferedStream bufferedStream, CancellationToken cancellationToken) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/uds/HttpRequestResponseSerializer.cs:line 63 at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine) at Microsoft.Azure.Devices.Edge.Util.Uds.HttpRequestResponseSerializer.DeserializeResponse(HttpBufferedStream bufferedStream, CancellationToken cancellationToken) at Microsoft.Azure.Devices.Edge.Util.Uds.HttpUdsMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/uds/HttpUdsMessageHandler.cs:line 41 at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine) at Microsoft.Azure.Devices.Edge.Util.Uds.HttpUdsMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) at System.Net.Http.HttpMessageInvoker.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) at System.Net.Http.HttpClient.g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken) at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine) at Microsoft.Azure.Devices.Edge.Util.Edged.Version_2019_01_30.GeneratedCode.HttpWorkloadClient.SignAsync(String api_version, String name, String genid, SignRequest payload, CancellationToken cancellationToken) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/edged/version_2019_01_30/generatedCode/HttpWorkloadClient.cs:line 106 at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine) at Microsoft.Azure.Devices.Edge.Util.Edged.Version_2019_01_30.GeneratedCode.HttpWorkloadClient.SignAsync(String api_version, String name, String genid, SignRequest payload, CancellationToken cancellationToken) at Microsoft.Azure.Devices.Edge.Util.Edged.Version_2019_01_30.WorkloadClient.<>c__DisplayClass7_0.b__0() in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/edged/version_2019_01_30/WorkloadClient.cs:line 98 at Microsoft.Azure.Devices.Edge.Util.TransientFaultHandling.AsyncExecution`1.ExecuteAsyncImpl(Task ignore) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/transientFaultHandling/AsyncExecution[T].cs:line 64 at Microsoft.Azure.Devices.Edge.Util.TransientFaultHandling.RetryPolicy.ExecuteAsync[TResult](Func`1 taskFunc, CancellationToken cancellationToken) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/transientFaultHandling/RetryPolicy.cs:line 249 at Microsoft.Azure.Devices.Edge.Util.TransientFaultHandling.RetryPolicy.ExecuteAsync[TResult](Func`1 taskFunc) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/transientFaultHandling/RetryPolicy.cs:line 233 at Microsoft.Azure.Devices.Edge.Util.Edged.WorkloadClientVersioned.Execute[T](Func`1 func, String operation) at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine) at Microsoft.Azure.Devices.Edge.Util.Edged.Version_2019_01_30.WorkloadClient.SignAsync(String keyId, String algorithm, String data) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/edged/version_2019_01_30/WorkloadClient.cs:line 98 at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine) at Microsoft.Azure.Devices.Edge.Util.Edged.Version_2019_01_30.WorkloadClient.SignAsync(String keyId, String algorithm, String data) at Microsoft.Azure.Devices.Edge.Util.Edged.HttpHsmSignatureProvider.SignAsync(String data) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/edged/HttpHsmSignatureProvider.cs:line 29 at Microsoft.Azure.Devices.Edge.Util.ClientTokenProvider.GetTokenAsyncInternal(DateTime startTime, TimeSpan ttl) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/ClientTokenProvider.cs:line 82 at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine) at Microsoft.Azure.Devices.Edge.Util.ClientTokenProvider.GetTokenAsyncInternal(DateTime startTime, TimeSpan ttl) at Microsoft.Azure.Devices.Edge.Util.ClientTokenProvider.<>c__DisplayClass9_0.<b__3>d.MoveNext() in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/ClientTokenProvider.cs:line 58 at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine) at Microsoft.Azure.Devices.Edge.Util.ClientTokenProvider.<>c__DisplayClass9_0.b__3() at Microsoft.Azure.Devices.Edge.Util.ClientTokenProvider.<>c__DisplayClass9_0.<b__1>d.MoveNext() at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state) at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.AsyncStateMachineBox`1.MoveNext(Thread threadPoolThread) at System.Threading.ThreadPoolWorkQueue.Dispatch() at System.Threading.PortableThreadPool.WorkerThread.WorkerThreadStart() --- End of stack trace from previous location --- --- End of inner exception stack trace --- at System.IO.BufferedStream.FlushWriteAsync(CancellationToken cancellationToken) at System.IO.BufferedStream.ReadFromUnderlyingStreamAsync(Memory`1 buffer, CancellationToken cancellationToken, Int32 bytesAlreadySatisfied, Task semaphoreLockTask) at Microsoft.Azure.Devices.Edge.Util.Uds.HttpBufferedStream.ReadLineAsync(CancellationToken cancellationToken) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/uds/HttpBufferedStream.cs:line 62 at Microsoft.Azure.Devices.Edge.Util.Uds.HttpRequestResponseSerializer.SetResponseStatusLine(HttpResponseMessage httpResponse, HttpBufferedStream bufferedStream, CancellationToken cancellationToken) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/uds/HttpRequestResponseSerializer.cs:line 158 at Microsoft.Azure.Devices.Edge.Util.Uds.HttpRequestResponseSerializer.DeserializeResponse(HttpBufferedStream bufferedStream, CancellationToken cancellationToken) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/uds/HttpRequestResponseSerializer.cs:line 69 at Microsoft.Azure.Devices.Edge.Util.Uds.HttpUdsMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/uds/HttpUdsMessageHandler.cs:line 48 at System.Net.Http.HttpClient.g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken) at Microsoft.Azure.Devices.Edge.Util.Edged.Version_2019_01_30.GeneratedCode.HttpWorkloadClient.SignAsync(String api_version, String name, String genid, SignRequest payload, CancellationToken cancellationToken) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/edged/version_2019_01_30/generatedCode/HttpWorkloadClient.cs:line 106 at Microsoft.Azure.Devices.Edge.Util.TaskEx.TimeoutAfter[T](Task`1 task, TimeSpan timeout, Action action) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/TaskEx.cs:line 137 at Microsoft.Azure.Devices.Edge.Util.Edged.WorkloadClientVersioned.Execute[T](Func`1 func, String operation) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/edged/WorkloadClientVersioned.cs:line 81 at Microsoft.Azure.Devices.Edge.Util.Edged.WorkloadClientVersioned.Execute[T](Func`1 func, String operation) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/edged/WorkloadClientVersioned.cs:line 81 at Microsoft.Azure.Devices.Edge.Util.Edged.Version_2019_01_30.WorkloadClient.SignAsync(String keyId, String algorithm, String data) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/edged/version_2019_01_30/WorkloadClient.cs:line 101 at Microsoft.Azure.Devices.Edge.Util.ClientTokenProvider.GetTokenAsyncInternal(DateTime startTime, TimeSpan ttl) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/ClientTokenProvider.cs:line 89 at Microsoft.Azure.Devices.Edge.Util.ClientTokenProvider.<>c__DisplayClass9_0.<b__3>d.MoveNext() in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/ClientTokenProvider.cs:line 62 --- End of stack trace from previous location --- at Microsoft.Azure.Devices.Edge.Util.ClientTokenProvider.<>c__DisplayClass9_0.<b__1>d.MoveNext() in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/ClientTokenProvider.cs:line 64 --- End of stack trace from previous location --- at Microsoft.Azure.Devices.Edge.Util.ClientTokenProvider.GetTokenAsync(Option`1 ttl) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/ClientTokenProvider.cs:line 65 at Microsoft.Azure.Devices.Client.AuthenticationWithTokenRefresh.GetTokenAsync(String iotHub) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpIoTCbsTokenProvider.GetTokenAsync(Uri namespaceAddress, String appliesTo, String[] requiredClaims) at Microsoft.Azure.Amqp.TaskHelpers.EndAsyncResult(IAsyncResult asyncResult) at Microsoft.Azure.Amqp.IteratorAsyncResult`1.StepCallback(IAsyncResult result) --- End of stack trace from previous location --- at Microsoft.Azure.Amqp.AsyncResult.End[TAsyncResult](IAsyncResult result) at Microsoft.Azure.Amqp.AmqpCbsLink.<>c.b__4_1(IAsyncResult r) at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization) --- End of stack trace from previous location --- at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpIoTCbsLink.SendTokenAsync(ICbsTokenProvider tokenProvider, Uri namespaceAddress, String audience, String resource, String[] requiredClaims, TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpAuthenticationRefresher.InitLoopAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpIoTConnection.CreateRefresherAsync(DeviceIdentity deviceIdentity, TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpConnectionHolder.CreateRefresherAsync(DeviceIdentity deviceIdentity, TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpUnit.EnsureSessionIsOpenAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpUnit.EnsureSessionIsOpenAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpUnit.OpenAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpTransportHandler.OpenAsync(CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.Transport.ProtocolRoutingDelegatingHandler.OpenAsync(CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.<>c__DisplayClass27_0.<b__0>d.MoveNext() --- End of stack trace from previous location --- at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.ExecuteWithErrorHandlingAsync[T](Func`1 asyncOperation) --- End of inner exception stack trace --- at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.ExecuteWithErrorHandlingAsync[T](Func`1 asyncOperation) at Microsoft.Azure.Devices.Client.Transport.RetryDelegatingHandler.<>c__DisplayClass44_0.<b__0>d.MoveNext() --- End of stack trace from previous location --- at Microsoft.Azure.Devices.Client.Transport.RetryDelegatingHandler.OpenInternalAsync(Boolean withRetry, CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.Transport.RetryDelegatingHandler.EnsureOpenedAsync(Boolean withRetry, CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.InternalClient.OpenAsync() at Microsoft.Azure.Devices.Edge.Util.TaskEx.TimeoutAfter(Task task, TimeSpan timeout, Action action) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/TaskEx.cs:line 160 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ModuleClientWrapper.OpenAsync() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ModuleClientWrapper.cs:line 65 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.<>c__DisplayClass30_0.<b__0>d.MoveNext() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 196 --- End of stack trace from previous location --- at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 189 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 189 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.OpenAsync() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 65 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.ConnectToIoTHub(ITokenProvider newTokenProvider) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 140 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.CreateNewCloudProxyAsync(ITokenProvider newTokenProvider) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 123 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.Create(IIdentity identity, Action`2 connectionStatusChangedHandler, ITransportSettings[] transportSettings, IMessageConverterProvider messageConverterProvider, IClientProvider clientProvider, ICloudListener cloudListener, ITokenProvider tokenProvider, TimeSpan idleTimeout, Boolean closeOnIdleTimeout, TimeSpan operationTimeout, TimeSpan cloudConnectionHangingTimeout, String productInfo, Option`1 modelId) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 104 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.TryCreateCloudConnectionFromServiceIdentity(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshOutOfDateCache, CloudListener cloudListener, String authChain) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 306 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.ConnectInternalWithDeviceStateTracking(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshCachedIdentity) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 267 <6> 2023-11-30 16:49:33.862 +00:00 [INF] - Error getting cloud connection for device usea1svlimtcld1-vm/$edgeHub Microsoft.Azure.Devices.Client.Exceptions.IotHubCommunicationException: Transient network error occurred, please retry. ---> System.IO.IOException: Unable to write data to the transport connection: Broken pipe. ---> System.Net.Sockets.SocketException (32): Broken pipe at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.CreateException(SocketError error, Boolean forAsyncThrow) at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.SendAsyncForNetworkStream(Socket socket, CancellationToken cancellationToken) at System.Net.Sockets.NetworkStream.WriteAsync(ReadOnlyMemory`1 buffer, CancellationToken cancellationToken) at System.IO.BufferedStream.FlushWriteAsync(CancellationToken cancellationToken) at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine) at System.IO.BufferedStream.FlushWriteAsync(CancellationToken cancellationToken) at System.IO.BufferedStream.ReadFromUnderlyingStreamAsync(Memory`1 buffer, CancellationToken cancellationToken, Int32 bytesAlreadySatisfied, Task semaphoreLockTask) at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine) at System.IO.BufferedStream.ReadFromUnderlyingStreamAsync(Memory`1 buffer, CancellationToken cancellationToken, Int32 bytesAlreadySatisfied, Task semaphoreLockTask) at System.IO.BufferedStream.ReadAsync(Byte[] buffer, Int32 offset, Int32 count, CancellationToken cancellationToken) at Microsoft.Azure.Devices.Edge.Util.Uds.HttpBufferedStream.ReadLineAsync(CancellationToken cancellationToken) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/uds/HttpBufferedStream.cs:line 62 at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine) at Microsoft.Azure.Devices.Edge.Util.Uds.HttpBufferedStream.ReadLineAsync(CancellationToken cancellationToken) at Microsoft.Azure.Devices.Edge.Util.Uds.HttpRequestResponseSerializer.SetResponseStatusLine(HttpResponseMessage httpResponse, HttpBufferedStream bufferedStream, CancellationToken cancellationToken) at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine) at Microsoft.Azure.Devices.Edge.Util.Uds.HttpRequestResponseSerializer.SetResponseStatusLine(HttpResponseMessage httpResponse, HttpBufferedStream bufferedStream, CancellationToken cancellationToken) at Microsoft.Azure.Devices.Edge.Util.Uds.HttpRequestResponseSerializer.DeserializeResponse(HttpBufferedStream bufferedStream, CancellationToken cancellationToken) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/uds/HttpRequestResponseSerializer.cs:line 63 at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine) at Microsoft.Azure.Devices.Edge.Util.Uds.HttpRequestResponseSerializer.DeserializeResponse(HttpBufferedStream bufferedStream, CancellationToken cancellationToken) at Microsoft.Azure.Devices.Edge.Util.Uds.HttpUdsMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/uds/HttpUdsMessageHandler.cs:line 41 at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine) at Microsoft.Azure.Devices.Edge.Util.Uds.HttpUdsMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) at System.Net.Http.HttpMessageInvoker.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) at System.Net.Http.HttpClient.g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken) at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine) at Microsoft.Azure.Devices.Edge.Util.Edged.Version_2019_01_30.GeneratedCode.HttpWorkloadClient.SignAsync(String api_version, String name, String genid, SignRequest payload, CancellationToken cancellationToken) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/edged/version_2019_01_30/generatedCode/HttpWorkloadClient.cs:line 106 at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine) at Microsoft.Azure.Devices.Edge.Util.Edged.Version_2019_01_30.GeneratedCode.HttpWorkloadClient.SignAsync(String api_version, String name, String genid, SignRequest payload, CancellationToken cancellationToken) at Microsoft.Azure.Devices.Edge.Util.Edged.Version_2019_01_30.WorkloadClient.<>c__DisplayClass7_0.b__0() in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/edged/version_2019_01_30/WorkloadClient.cs:line 98 at Microsoft.Azure.Devices.Edge.Util.TransientFaultHandling.AsyncExecution`1.ExecuteAsyncImpl(Task ignore) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/transientFaultHandling/AsyncExecution[T].cs:line 64 at Microsoft.Azure.Devices.Edge.Util.TransientFaultHandling.RetryPolicy.ExecuteAsync[TResult](Func`1 taskFunc, CancellationToken cancellationToken) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/transientFaultHandling/RetryPolicy.cs:line 249 at Microsoft.Azure.Devices.Edge.Util.TransientFaultHandling.RetryPolicy.ExecuteAsync[TResult](Func`1 taskFunc) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/transientFaultHandling/RetryPolicy.cs:line 233 at Microsoft.Azure.Devices.Edge.Util.Edged.WorkloadClientVersioned.Execute[T](Func`1 func, String operation) at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine) at Microsoft.Azure.Devices.Edge.Util.Edged.Version_2019_01_30.WorkloadClient.SignAsync(String keyId, String algorithm, String data) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/edged/version_2019_01_30/WorkloadClient.cs:line 98 at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine) at Microsoft.Azure.Devices.Edge.Util.Edged.Version_2019_01_30.WorkloadClient.SignAsync(String keyId, String algorithm, String data) at Microsoft.Azure.Devices.Edge.Util.Edged.HttpHsmSignatureProvider.SignAsync(String data) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/edged/HttpHsmSignatureProvider.cs:line 29 at Microsoft.Azure.Devices.Edge.Util.ClientTokenProvider.GetTokenAsyncInternal(DateTime startTime, TimeSpan ttl) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/ClientTokenProvider.cs:line 82 at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine) at Microsoft.Azure.Devices.Edge.Util.ClientTokenProvider.GetTokenAsyncInternal(DateTime startTime, TimeSpan ttl) at Microsoft.Azure.Devices.Edge.Util.ClientTokenProvider.<>c__DisplayClass9_0.<b__3>d.MoveNext() in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/ClientTokenProvider.cs:line 58 at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine) at Microsoft.Azure.Devices.Edge.Util.ClientTokenProvider.<>c__DisplayClass9_0.b__3() at Microsoft.Azure.Devices.Edge.Util.ClientTokenProvider.<>c__DisplayClass9_0.<b__1>d.MoveNext() at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state) at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.AsyncStateMachineBox`1.MoveNext(Thread threadPoolThread) at System.Threading.ThreadPoolWorkQueue.Dispatch() at System.Threading.PortableThreadPool.WorkerThread.WorkerThreadStart() --- End of stack trace from previous location --- --- End of inner exception stack trace --- at System.IO.BufferedStream.FlushWriteAsync(CancellationToken cancellationToken) at System.IO.BufferedStream.ReadFromUnderlyingStreamAsync(Memory`1 buffer, CancellationToken cancellationToken, Int32 bytesAlreadySatisfied, Task semaphoreLockTask) at Microsoft.Azure.Devices.Edge.Util.Uds.HttpBufferedStream.ReadLineAsync(CancellationToken cancellationToken) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/uds/HttpBufferedStream.cs:line 62 at Microsoft.Azure.Devices.Edge.Util.Uds.HttpRequestResponseSerializer.SetResponseStatusLine(HttpResponseMessage httpResponse, HttpBufferedStream bufferedStream, CancellationToken cancellationToken) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/uds/HttpRequestResponseSerializer.cs:line 158 at Microsoft.Azure.Devices.Edge.Util.Uds.HttpRequestResponseSerializer.DeserializeResponse(HttpBufferedStream bufferedStream, CancellationToken cancellationToken) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/uds/HttpRequestResponseSerializer.cs:line 69 at Microsoft.Azure.Devices.Edge.Util.Uds.HttpUdsMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/uds/HttpUdsMessageHandler.cs:line 48 at System.Net.Http.HttpClient.g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken) at Microsoft.Azure.Devices.Edge.Util.Edged.Version_2019_01_30.GeneratedCode.HttpWorkloadClient.SignAsync(String api_version, String name, String genid, SignRequest payload, CancellationToken cancellationToken) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/edged/version_2019_01_30/generatedCode/HttpWorkloadClient.cs:line 106 at Microsoft.Azure.Devices.Edge.Util.TaskEx.TimeoutAfter[T](Task`1 task, TimeSpan timeout, Action action) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/TaskEx.cs:line 137 at Microsoft.Azure.Devices.Edge.Util.Edged.WorkloadClientVersioned.Execute[T](Func`1 func, String operation) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/edged/WorkloadClientVersioned.cs:line 81 at Microsoft.Azure.Devices.Edge.Util.Edged.WorkloadClientVersioned.Execute[T](Func`1 func, String operation) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/edged/WorkloadClientVersioned.cs:line 81 at Microsoft.Azure.Devices.Edge.Util.Edged.Version_2019_01_30.WorkloadClient.SignAsync(String keyId, String algorithm, String data) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/edged/version_2019_01_30/WorkloadClient.cs:line 101 at Microsoft.Azure.Devices.Edge.Util.ClientTokenProvider.GetTokenAsyncInternal(DateTime startTime, TimeSpan ttl) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/ClientTokenProvider.cs:line 89 at Microsoft.Azure.Devices.Edge.Util.ClientTokenProvider.<>c__DisplayClass9_0.<b__3>d.MoveNext() in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/ClientTokenProvider.cs:line 62 --- End of stack trace from previous location --- at Microsoft.Azure.Devices.Edge.Util.ClientTokenProvider.<>c__DisplayClass9_0.<b__1>d.MoveNext() in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/ClientTokenProvider.cs:line 64 --- End of stack trace from previous location --- at Microsoft.Azure.Devices.Edge.Util.ClientTokenProvider.GetTokenAsync(Option`1 ttl) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/ClientTokenProvider.cs:line 65 at Microsoft.Azure.Devices.Client.AuthenticationWithTokenRefresh.GetTokenAsync(String iotHub) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpIoTCbsTokenProvider.GetTokenAsync(Uri namespaceAddress, String appliesTo, String[] requiredClaims) at Microsoft.Azure.Amqp.TaskHelpers.EndAsyncResult(IAsyncResult asyncResult) at Microsoft.Azure.Amqp.IteratorAsyncResult`1.StepCallback(IAsyncResult result) --- End of stack trace from previous location --- at Microsoft.Azure.Amqp.AsyncResult.End[TAsyncResult](IAsyncResult result) at Microsoft.Azure.Amqp.AmqpCbsLink.<>c.b__4_1(IAsyncResult r) at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization) --- End of stack trace from previous location --- at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpIoTCbsLink.SendTokenAsync(ICbsTokenProvider tokenProvider, Uri namespaceAddress, String audience, String resource, String[] requiredClaims, TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpAuthenticationRefresher.InitLoopAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpIoTConnection.CreateRefresherAsync(DeviceIdentity deviceIdentity, TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpConnectionHolder.CreateRefresherAsync(DeviceIdentity deviceIdentity, TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpUnit.EnsureSessionIsOpenAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpUnit.EnsureSessionIsOpenAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpUnit.OpenAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpTransportHandler.OpenAsync(CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.Transport.ProtocolRoutingDelegatingHandler.OpenAsync(CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.<>c__DisplayClass27_0.<b__0>d.MoveNext() --- End of stack trace from previous location --- at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.ExecuteWithErrorHandlingAsync[T](Func`1 asyncOperation) --- End of inner exception stack trace --- at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.ExecuteWithErrorHandlingAsync[T](Func`1 asyncOperation) at Microsoft.Azure.Devices.Client.Transport.RetryDelegatingHandler.<>c__DisplayClass44_0.<b__0>d.MoveNext() --- End of stack trace from previous location --- at Microsoft.Azure.Devices.Client.Transport.RetryDelegatingHandler.OpenInternalAsync(Boolean withRetry, CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.Transport.RetryDelegatingHandler.EnsureOpenedAsync(Boolean withRetry, CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.InternalClient.OpenAsync() at Microsoft.Azure.Devices.Edge.Util.TaskEx.TimeoutAfter(Task task, TimeSpan timeout, Action action) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/TaskEx.cs:line 160 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ModuleClientWrapper.OpenAsync() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ModuleClientWrapper.cs:line 65 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.<>c__DisplayClass30_0.<b__0>d.MoveNext() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 196 --- End of stack trace from previous location --- at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 189 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 189 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.OpenAsync() in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 65 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.ConnectToIoTHub(ITokenProvider newTokenProvider) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 140 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.CreateNewCloudProxyAsync(ITokenProvider newTokenProvider) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 123 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.Create(IIdentity identity, Action`2 connectionStatusChangedHandler, ITransportSettings[] transportSettings, IMessageConverterProvider messageConverterProvider, IClientProvider clientProvider, ICloudListener cloudListener, ITokenProvider tokenProvider, TimeSpan idleTimeout, Boolean closeOnIdleTimeout, TimeSpan operationTimeout, TimeSpan cloudConnectionHangingTimeout, String productInfo, Option`1 modelId) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 104 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.TryCreateCloudConnectionFromServiceIdentity(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshOutOfDateCache, CloudListener cloudListener, String authChain) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 306 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.ConnectInternalWithDeviceStateTracking(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshCachedIdentity) in /mnt/vss/_work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 267 <6> 2023-11-30 16:49:47.633 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 16:49:47.634 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 0.4796ms" <6> 2023-11-30 16:50:33.861 +00:00 [INF] - Attempting to connect to IoT Hub for client usea1svlimtcld1-vm/plant_11 via AMQP... <6> 2023-11-30 16:51:29.096 +00:00 [INF] - New token received on the Cbs link <6> 2023-11-30 16:51:29.098 +00:00 [INF] - Client usea1svlimtcld1-vm/metrics in device scope authenticated locally. <6> 2023-11-30 16:51:29.098 +00:00 [INF] - New device connection for device usea1svlimtcld1-vm/metrics <6> 2023-11-30 16:51:29.098 +00:00 [INF] - Client usea1svlimtcld1-vm/metrics connected to edgeHub, processing existing subscriptions. <6> 2023-11-30 16:51:29.099 +00:00 [INF] - Bind device proxy for device usea1svlimtcld1-vm/metrics <6> 2023-11-30 16:51:29.099 +00:00 [INF] - Initialized device listener in the AMQP protocol head for usea1svlimtcld1-vm/metrics <6> 2023-11-30 16:51:29.099 +00:00 [INF] - Opened link Events for usea1svlimtcld1-vm/metrics <6> 2023-11-30 16:51:29.100 +00:00 [INF] - Attempting to connect to IoT Hub for client usea1svlimtcld1-vm/metrics via AMQP... <6> 2023-11-30 16:51:29.184 +00:00 [INF] - Attempting to connect to IoT Hub for client usea1svlimtcld1-vm/$edgeHub via AMQP... <6> 2023-11-30 16:51:29.403 +00:00 [INF] - Exiting disconnected state <6> 2023-11-30 16:51:29.403 +00:00 [INF] - Received device connected callback <6> 2023-11-30 16:51:29.403 +00:00 [INF] - Device connected to cloud, processing subscriptions for connected clients. <6> 2023-11-30 16:51:29.403 +00:00 [INF] - Processing subscriptions for client usea1svlimtcld1-vm/$edgeHub on device connected to cloud. <6> 2023-11-30 16:51:29.403 +00:00 [INF] - Processing subscriptions for client usea1svlimtcld1-vm/metrics on device connected to cloud. <6> 2023-11-30 16:51:29.403 +00:00 [INF] - Skipping usea1svlimtcld1-vm/metrics for subscription processing, as it is currently being processed. <6> 2023-11-30 16:51:29.403 +00:00 [INF] - Cloud connection for usea1svlimtcld1-vm/$edgeHub is True <6> 2023-11-30 16:51:29.403 +00:00 [INF] - Connection status for usea1svlimtcld1-vm/$edgeHub changed to ConnectionEstablished <6> 2023-11-30 16:51:29.403 +00:00 [INF] - Client usea1svlimtcld1-vm/$edgeHub connected to cloud, processing existing subscriptions. <6> 2023-11-30 16:51:29.403 +00:00 [INF] - Skipping usea1svlimtcld1-vm/$edgeHub for subscription processing, as it is currently being processed. <6> 2023-11-30 16:51:29.403 +00:00 [INF] - Cloud connection for usea1svlimtcld1-vm/plant_11 is True <6> 2023-11-30 16:51:29.403 +00:00 [INF] - Connection status for usea1svlimtcld1-vm/plant_11 changed to ConnectionEstablished <6> 2023-11-30 16:51:29.403 +00:00 [INF] - Client usea1svlimtcld1-vm/plant_11 connected to cloud, processing existing subscriptions. <6> 2023-11-30 16:51:29.403 +00:00 [INF] - Cloud connection for usea1svlimtcld1-vm/mtc_helper is True <6> 2023-11-30 16:51:29.403 +00:00 [INF] - Connection status for usea1svlimtcld1-vm/mtc_helper changed to ConnectionEstablished <6> 2023-11-30 16:51:29.403 +00:00 [INF] - Client usea1svlimtcld1-vm/mtc_helper connected to cloud, processing existing subscriptions. <6> 2023-11-30 16:51:29.403 +00:00 [INF] - Cloud connection for usea1svlimtcld1-vm/$edgeHub is True <6> 2023-11-30 16:51:29.403 +00:00 [INF] - Connection status for usea1svlimtcld1-vm/$edgeHub changed to ConnectionEstablished <6> 2023-11-30 16:51:29.403 +00:00 [INF] - Client usea1svlimtcld1-vm/$edgeHub connected to cloud, processing existing subscriptions. <6> 2023-11-30 16:51:29.403 +00:00 [INF] - Device scope identities refresh is ready because a refresh was signalled. <6> 2023-11-30 16:51:29.403 +00:00 [INF] - Skipping usea1svlimtcld1-vm/$edgeHub for subscription processing, as it is currently being processed. <6> 2023-11-30 16:51:29.403 +00:00 [INF] - Cloud connection for usea1svlimtcld1-vm/plant_11 is True <6> 2023-11-30 16:51:29.403 +00:00 [INF] - Starting refresh of device scope identities cache <6> 2023-11-30 16:51:29.403 +00:00 [INF] - Connection status for usea1svlimtcld1-vm/plant_11 changed to ConnectionEstablished <6> 2023-11-30 16:51:29.403 +00:00 [INF] - Client usea1svlimtcld1-vm/plant_11 connected to cloud, processing existing subscriptions. <6> 2023-11-30 16:51:29.403 +00:00 [INF] - Skipping usea1svlimtcld1-vm/plant_11 for subscription processing, as it is currently being processed. <6> 2023-11-30 16:51:29.403 +00:00 [INF] - Cloud connection for usea1svlimtcld1-vm/mtc_helper is True <6> 2023-11-30 16:51:29.403 +00:00 [INF] - Connection status for usea1svlimtcld1-vm/mtc_helper changed to ConnectionEstablished <6> 2023-11-30 16:51:29.403 +00:00 [INF] - Client usea1svlimtcld1-vm/mtc_helper connected to cloud, processing existing subscriptions. <6> 2023-11-30 16:51:29.403 +00:00 [INF] - Skipping usea1svlimtcld1-vm/mtc_helper for subscription processing, as it is currently being processed. <6> 2023-11-30 16:51:29.403 +00:00 [INF] - Entering connected state <6> 2023-11-30 16:51:29.403 +00:00 [INF] - Cloud connection for usea1svlimtcld1-vm/metrics is True <6> 2023-11-30 16:51:29.403 +00:00 [INF] - Connection status for usea1svlimtcld1-vm/metrics changed to ConnectionEstablished <6> 2023-11-30 16:51:29.403 +00:00 [INF] - Client usea1svlimtcld1-vm/metrics connected to cloud, processing existing subscriptions. <6> 2023-11-30 16:51:29.403 +00:00 [INF] - Skipping usea1svlimtcld1-vm/metrics for subscription processing, as it is currently being processed. <6> 2023-11-30 16:51:29.403 +00:00 [INF] - Created cloud proxy for client usea1svlimtcld1-vm/metrics via AMQP, with client operation timeout 20 seconds. <6> 2023-11-30 16:51:29.404 +00:00 [INF] - Initialized cloud proxy 28eed42f-aa2d-40d7-b1ab-f4926c512202 for usea1svlimtcld1-vm/metrics <6> 2023-11-30 16:51:29.404 +00:00 [INF] - Created cloud connection for client usea1svlimtcld1-vm/metrics <6> 2023-11-30 16:51:29.404 +00:00 [INF] - Cloud connection for usea1svlimtcld1-vm/$edgeHub is True <6> 2023-11-30 16:51:29.404 +00:00 [INF] - Connection status for usea1svlimtcld1-vm/$edgeHub changed to ConnectionEstablished <6> 2023-11-30 16:51:29.404 +00:00 [INF] - Client usea1svlimtcld1-vm/$edgeHub connected to cloud, processing existing subscriptions. <6> 2023-11-30 16:51:29.404 +00:00 [INF] - Skipping usea1svlimtcld1-vm/$edgeHub for subscription processing, as it is currently being processed. <6> 2023-11-30 16:51:29.404 +00:00 [INF] - Created cloud proxy for client usea1svlimtcld1-vm/$edgeHub via AMQP, with client operation timeout 20 seconds. <6> 2023-11-30 16:51:29.404 +00:00 [INF] - Initialized cloud proxy b23457a3-7774-47a5-9f08-65da792a1a7a for usea1svlimtcld1-vm/$edgeHub <6> 2023-11-30 16:51:29.404 +00:00 [INF] - Created cloud connection for client usea1svlimtcld1-vm/$edgeHub <6> 2023-11-30 16:51:29.404 +00:00 [INF] - Cloud connection for usea1svlimtcld1-vm/plant_11 is True <6> 2023-11-30 16:51:29.404 +00:00 [INF] - Connection status for usea1svlimtcld1-vm/plant_11 changed to ConnectionEstablished <6> 2023-11-30 16:51:29.404 +00:00 [INF] - Client usea1svlimtcld1-vm/plant_11 connected to cloud, processing existing subscriptions. <6> 2023-11-30 16:51:29.404 +00:00 [INF] - Skipping usea1svlimtcld1-vm/plant_11 for subscription processing, as it is currently being processed. <6> 2023-11-30 16:51:29.404 +00:00 [INF] - Created cloud proxy for client usea1svlimtcld1-vm/plant_11 via AMQP, with client operation timeout 20 seconds. <6> 2023-11-30 16:51:29.404 +00:00 [INF] - Initialized cloud proxy d5ea2cc4-ab18-4352-85be-fe44aad0326b for usea1svlimtcld1-vm/plant_11 <6> 2023-11-30 16:51:29.404 +00:00 [INF] - Created cloud connection for client usea1svlimtcld1-vm/plant_11 <6> 2023-11-30 16:51:29.405 +00:00 [INF] - Attempting to connect to IoT Hub for client usea1svlimtcld1-vm/mtc_helper via AMQP... <6> 2023-11-30 16:51:29.417 +00:00 [INF] - New token received on the Cbs link <6> 2023-11-30 16:51:29.419 +00:00 [INF] - Client usea1svlimtcld1-vm/mtc_helper in device scope authenticated locally. <6> 2023-11-30 16:51:29.419 +00:00 [INF] - New device connection for device usea1svlimtcld1-vm/mtc_helper <6> 2023-11-30 16:51:29.419 +00:00 [INF] - Client usea1svlimtcld1-vm/mtc_helper connected to edgeHub, processing existing subscriptions. <6> 2023-11-30 16:51:29.419 +00:00 [INF] - Skipping usea1svlimtcld1-vm/mtc_helper for subscription processing, as it is currently being processed. <6> 2023-11-30 16:51:29.419 +00:00 [INF] - Bind device proxy for device usea1svlimtcld1-vm/mtc_helper <6> 2023-11-30 16:51:29.419 +00:00 [INF] - Initialized device listener in the AMQP protocol head for usea1svlimtcld1-vm/mtc_helper <6> 2023-11-30 16:51:29.419 +00:00 [INF] - Opened link Events for usea1svlimtcld1-vm/mtc_helper <6> 2023-11-30 16:51:29.422 +00:00 [INF] - Opened link MethodSending for usea1svlimtcld1-vm/mtc_helper <6> 2023-11-30 16:51:29.423 +00:00 [INF] - Processing pending subscriptions for usea1svlimtcld1-vm/mtc_helper <6> 2023-11-30 16:51:29.423 +00:00 [INF] - Opened link MethodReceiving for usea1svlimtcld1-vm/mtc_helper <6> 2023-11-30 16:51:29.424 +00:00 [INF] - Opened link TwinSending for usea1svlimtcld1-vm/mtc_helper <6> 2023-11-30 16:51:29.424 +00:00 [INF] - Opened link TwinReceiving for usea1svlimtcld1-vm/mtc_helper <6> 2023-11-30 16:51:29.526 +00:00 [INF] - Cloud connection for usea1svlimtcld1-vm/mtc_helper is True <6> 2023-11-30 16:51:29.526 +00:00 [INF] - Connection status for usea1svlimtcld1-vm/mtc_helper changed to ConnectionEstablished <6> 2023-11-30 16:51:29.526 +00:00 [INF] - Client usea1svlimtcld1-vm/mtc_helper connected to cloud, processing existing subscriptions. <6> 2023-11-30 16:51:29.526 +00:00 [INF] - Skipping usea1svlimtcld1-vm/mtc_helper for subscription processing, as it is currently being processed. <6> 2023-11-30 16:51:29.526 +00:00 [INF] - Created cloud proxy for client usea1svlimtcld1-vm/mtc_helper via AMQP, with client operation timeout 20 seconds. <6> 2023-11-30 16:51:29.527 +00:00 [INF] - Initialized cloud proxy 23ef79a8-ad9b-4e38-8d9d-35ea4e30bb64 for usea1svlimtcld1-vm/mtc_helper <6> 2023-11-30 16:51:29.527 +00:00 [INF] - Created cloud connection for client usea1svlimtcld1-vm/mtc_helper <6> 2023-11-30 16:51:29.601 +00:00 [INF] - Processing pending subscriptions for usea1svlimtcld1-vm/mtc_helper <6> 2023-11-30 16:51:29.684 +00:00 [INF] - Updated reported properties for usea1svlimtcld1-vm/$edgeHub <6> 2023-11-30 16:51:29.824 +00:00 [INF] - Done syncing reported properties for usea1svlimtcld1-vm/metrics <6> 2023-11-30 16:51:29.838 +00:00 [INF] - Done syncing reported properties for usea1svlimtcld1-vm/mtc_helper <6> 2023-11-30 16:51:30.010 +00:00 [INF] - Not changed node: usea1svlimtcld1-vm <6> 2023-11-30 16:51:30.010 +00:00 [INF] - Not changed node: usea1svlimtcld1-vm/$edgeAgent <6> 2023-11-30 16:51:30.010 +00:00 [INF] - Not changed node: usea1svlimtcld1-vm/$edgeHub <6> 2023-11-30 16:51:30.010 +00:00 [INF] - Not changed node: usea1svlimtcld1-vm/mtc_helper <6> 2023-11-30 16:51:30.010 +00:00 [INF] - Not changed node: usea1svlimtcld1-vm/metrics <6> 2023-11-30 16:51:30.010 +00:00 [INF] - Not changed node: usea1svlimtcld1-vm/plant_11 <6> 2023-11-30 16:51:30.010 +00:00 [INF] - Done refreshing device scope identities cache. Waiting for 60 minutes. <6> 2023-11-30 16:51:34.922 +00:00 [INF] - Updated reported properties for usea1svlimtcld1-vm/$edgeHub <6> 2023-11-30 16:52:27.947 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 16:52:27.948 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 1.0267ms" <6> 2023-11-30 16:52:28.876 +00:00 [INF] - New token received on the Cbs link <6> 2023-11-30 16:52:28.905 +00:00 [INF] - Client usea1svlimtcld1-vm/plant_11 in device scope authenticated locally. <6> 2023-11-30 16:52:28.906 +00:00 [INF] - New device connection for device usea1svlimtcld1-vm/plant_11 <6> 2023-11-30 16:52:28.906 +00:00 [INF] - Client usea1svlimtcld1-vm/plant_11 connected to edgeHub, processing existing subscriptions. <6> 2023-11-30 16:52:28.906 +00:00 [INF] - Bind device proxy for device usea1svlimtcld1-vm/plant_11 <6> 2023-11-30 16:52:28.907 +00:00 [INF] - Initialized device listener in the AMQP protocol head for usea1svlimtcld1-vm/plant_11 <6> 2023-11-30 16:52:28.907 +00:00 [INF] - Opened link Events for usea1svlimtcld1-vm/plant_11 <6> 2023-11-30 16:52:28.936 +00:00 [INF] - Opened link TwinSending for usea1svlimtcld1-vm/plant_11 <6> 2023-11-30 16:52:28.936 +00:00 [INF] - Processing pending subscriptions for usea1svlimtcld1-vm/plant_11 <6> 2023-11-30 16:52:28.937 +00:00 [INF] - Opened link TwinReceiving for usea1svlimtcld1-vm/plant_11 <6> 2023-11-30 16:52:29.123 +00:00 [INF] - Processing pending subscriptions for usea1svlimtcld1-vm/plant_11 <6> 2023-11-30 16:52:29.123 +00:00 [INF] - Opened link MethodSending for usea1svlimtcld1-vm/plant_11 <6> 2023-11-30 16:52:29.127 +00:00 [INF] - Opened link MethodReceiving for usea1svlimtcld1-vm/plant_11 <6> 2023-11-30 16:52:29.173 +00:00 [INF] - Updated reported properties for usea1svlimtcld1-vm/$edgeHub <6> 2023-11-30 16:52:55.941 +00:00 [INF] - Entering periodic task to reauthenticate connected clients <6> 2023-11-30 16:53:28.273 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 16:53:28.277 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 3.9816ms" <6> 2023-11-30 16:54:28.619 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 16:54:28.622 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 2.0490ms" <6> 2023-11-30 16:55:29.095 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 16:55:29.103 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 8.8453ms" <6> 2023-11-30 16:56:29.470 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 16:56:29.473 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 3.0262ms" <6> 2023-11-30 16:57:29.826 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 16:57:29.835 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 9.3879ms" <6> 2023-11-30 16:57:55.941 +00:00 [INF] - Entering periodic task to reauthenticate connected clients <6> 2023-11-30 16:58:30.286 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 16:58:30.289 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 2.5636ms" <6> 2023-11-30 16:59:30.591 +00:00 [INF] - "Request starting HTTP/1.1 GET http://edgehub:9600/metrics - -" <6> 2023-11-30 16:59:30.597 +00:00 [INF] - "Request finished HTTP/1.1 GET http://edgehub:9600/metrics - - - 200 - text/plain;+version=0.0.4;+charset=utf-8 6.1038ms"