Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

EdgeAgent The X509 certificate could not be added to the store. #6005

Closed
harounshehata opened this issue Jan 17, 2022 · 22 comments
Closed

EdgeAgent The X509 certificate could not be added to the store. #6005

harounshehata opened this issue Jan 17, 2022 · 22 comments
Assignees
Labels
customer-reported iotedge needinfo Needs more info from issue reporter / updates from PR submitter

Comments

@harounshehata
Copy link

harounshehata commented Jan 17, 2022

Expected Behavior

Edge Agent should be able to setup correctly and connect to the IoT Hub.

Current Behavior

After the provisioning process (https://docs.microsoft.com/en-us/azure/iot-edge/how-to-provision-single-device-linux-symmetric?view=iotedge-2020-11&tabs=azure-portal#install-iot-edge), I noticed that the EdgeAgent is failing immediately.
The EdgeAgent Docker logs show:

System.Security.Cryptography.CryptographicException: The X509 certificate could not be added to the store.
 ---> System.UnauthorizedAccessException: Access to the path '/home/edgeagentuser/.dotnet/corefx/cryptography/x509stores/root' is denied.
 ---> System.IO.IOException: Permission denied
   --- End of inner exception stack trace ---
   at System.IO.FileSystem.CreateDirectory(String fullPath)
   at System.IO.Directory.CreateDirectory(String path)
   at Internal.Cryptography.Pal.DirectoryBasedStoreProvider.AddCertToStore(ICertificatePal certPal)
   at Internal.Cryptography.Pal.DirectoryBasedStoreProvider.Add(ICertificatePal certPal)
   --- End of inner exception stack trace ---
   at Internal.Cryptography.Pal.DirectoryBasedStoreProvider.Add(ICertificatePal certPal)
   at System.Security.Cryptography.X509Certificates.X509Store.Add(X509Certificate2 certificate)
   at Microsoft.Azure.Devices.Edge.Util.CertificateHelper.InstallCertificates(IEnumerable`1 certificateChain, ILogger logger) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/CertificateHelper.cs:line 228
   at Microsoft.Azure.Devices.Edge.Agent.Service.Program.MainAsync(IConfiguration configuration) in /mnt/vss/_work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.Service/Program.cs:line 187

The file in question does not exist, no directory "/home/edgeagentuser"

Running sudo iotedge check, gets stuck after:
image

What kind of permissions do I need so that the edgeAgent can add the certificate to the store?

Steps to Reproduce

Provide a detailed set of steps to reproduce the bug.

  1. Follow IotEdge Linux Provision Guide
  2. Apply configuration
  3. Check EdgeAgent status
  4. Permission Issue arises

Context (Environment)

Public access to IoTHub is disabled and device accesses iothub through private endpoint. Connection to hostname is successfull.

Output of iotedge check

Click here
√ keyd configuration is well-formed - OK
√ certd configuration is well-formed - OK
√ tpmd configuration is well-formed - OK
√ identityd configuration is well-formed - OK
√ daemon configurations up-to-date with config.toml - OK
√ identityd config toml file specifies a valid hostname - OK

Device Information

  • Host OS [e.g. Ubuntu 18.04, Windows Server IoT 2019]: Ubuntu 18.04
  • Architecture [e.g. amd64, arm32, arm64]: amd64
  • Container OS [e.g. Linux containers, Windows containers]: Linux containers

Runtime Versions

  • aziot-edged [run iotedge version]: 1.2.5
  • Edge Agent [image tag (e.g. 1.0.0)]: 1.2
  • Edge Hub [image tag (e.g. 1.0.0)]: Not Started
  • Docker/Moby [run docker version]: 20.10.11+azure-3

Logs

aziot-edged logs
-- Logs begin at Thu 2021-09-02 09:54:11 EEST, end at Mon 2022-01-17 14:40:47 EET. --
Jan 17 14:01:31 l31lnx813 systemd[1]: Started Azure IoT Identity Service.
Jan 17 14:01:31 l31lnx813 aziot-identityd[26930]: 2022-01-17T12:01:31Z [INFO] - Starting service...
Jan 17 14:01:31 l31lnx813 aziot-identityd[26930]: 2022-01-17T12:01:31Z [INFO] - Version - 1.2.4
Jan 17 14:01:31 l31lnx813 aziot-identityd[26930]: 2022-01-17T12:01:31Z [INFO] - Provisioning starting. Reason: Startup
Jan 17 14:01:31 l31lnx813 aziot-identityd[26930]: 2022-01-17T12:01:31Z [WARN] - Failed to clear old module backup state: Permission denied (os error 13)
Jan 17 14:01:31 l31lnx813 aziot-identityd[26930]: 2022-01-17T12:01:31Z [WARN] - Failed to create module information backup state folder: internal error
Jan 17 14:01:31 l31lnx813 aziot-identityd[26930]: 2022-01-17T12:01:31Z [INFO] - Updated device info for cdp-apps-dev-nordland-iot-edge.
Jan 17 14:01:31 l31lnx813 aziot-identityd[26930]: 2022-01-17T12:01:31Z [INFO] - Provisioning complete.
Jan 17 14:01:31 l31lnx813 aziot-identityd[26930]: 2022-01-17T12:01:31Z [INFO] - Identity reconciliation started. Reason: Startup
Jan 17 14:01:31 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:01:31Z [INFO] - <-- GET /key/device-id?api-version=2020-09-01 {"host": "keyd.sock"}
Jan 17 14:01:31 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:01:31Z [ERR!] - Permission denied (os error 13)
Jan 17 14:01:31 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:01:31Z [ERR!] - !!! internal error
Jan 17 14:01:31 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:01:31Z [ERR!] - !!! caused by: could not generate key
Jan 17 14:01:31 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:01:31Z [ERR!] - !!! caused by: could not create key: AZIOT_KEYS_RC_ERR_EXTERNAL
Jan 17 14:01:31 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:01:31Z [INFO] - --> 500 {"content-type": "application/json"}
Jan 17 14:01:31 l31lnx813 aziot-identityd[26930]: 2022-01-17T12:01:31Z [INFO] - Could not reconcile Identities with current device data. Reprovisioning.
Jan 17 14:01:31 l31lnx813 aziot-identityd[26930]: 2022-01-17T12:01:31Z [WARN] - Failed to clear old module backup state: Permission denied (os error 13)
Jan 17 14:01:31 l31lnx813 aziot-identityd[26930]: 2022-01-17T12:01:31Z [WARN] - Failed to create module information backup state folder: internal error
Jan 17 14:01:31 l31lnx813 aziot-identityd[26930]: 2022-01-17T12:01:31Z [INFO] - Updated device info for cdp-apps-dev-nordland-iot-edge.
Jan 17 14:01:31 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:01:31Z [INFO] - <-- GET /key/device-id?api-version=2020-09-01 {"host": "keyd.sock"}
Jan 17 14:01:31 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:01:31Z [ERR!] - Permission denied (os error 13)
Jan 17 14:01:31 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:01:31Z [ERR!] - !!! internal error
Jan 17 14:01:31 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:01:31Z [ERR!] - !!! caused by: could not generate key
Jan 17 14:01:31 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:01:31Z [ERR!] - !!! caused by: could not create key: AZIOT_KEYS_RC_ERR_EXTERNAL
Jan 17 14:01:31 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:01:31Z [INFO] - --> 500 {"content-type": "application/json"}
Jan 17 14:01:31 l31lnx813 aziot-identityd[26930]: 2022-01-17T12:01:31Z [ERR!] - Failed to provision with IoT Hub, and no valid device backup was found: Hub client error
Jan 17 14:01:31 l31lnx813 aziot-identityd[26930]: 2022-01-17T12:01:31Z [ERR!] - service encountered an error
Jan 17 14:01:31 l31lnx813 aziot-identityd[26930]: 2022-01-17T12:01:31Z [ERR!] - caused by: Hub client error
Jan 17 14:01:31 l31lnx813 aziot-identityd[26930]: 2022-01-17T12:01:31Z [ERR!] - caused by: internal error
Jan 17 14:01:31 l31lnx813 aziot-identityd[26930]: 2022-01-17T12:01:31Z [ERR!] -    0: <unknown>
Jan 17 14:01:31 l31lnx813 aziot-identityd[26930]:    1: <unknown>
Jan 17 14:01:31 l31lnx813 systemd[1]: aziot-identityd.service: Main process exited, code=exited, status=1/FAILURE
Jan 17 14:01:31 l31lnx813 systemd[1]: aziot-identityd.service: Failed with result 'exit-code'.
Jan 17 14:01:36 l31lnx813 aziot-edged[22058]: 2022-01-17T12:01:36Z [INFO] - Obtaining edge device provisioning data...
Jan 17 14:01:36 l31lnx813 systemd[1]: aziot-identityd.service: Service hold-off time over, scheduling restart.
Jan 17 14:01:36 l31lnx813 systemd[1]: aziot-identityd.service: Scheduled restart job, restart counter is at 783.
Jan 17 14:01:36 l31lnx813 systemd[1]: Stopped Azure IoT Identity Service.
Jan 17 14:01:36 l31lnx813 aziot-edged[22058]: 2022-01-17T12:01:36Z [WARN] - The daemon could not start up successfully: Could not retrieve device information
Jan 17 14:01:36 l31lnx813 aziot-edged[22058]: 2022-01-17T12:01:36Z [WARN] -         caused by: HTTP request error
Jan 17 14:01:36 l31lnx813 aziot-edged[22058]: 2022-01-17T12:01:36Z [WARN] -         caused by: connection error: Connection reset by peer (os error 104)
Jan 17 14:01:36 l31lnx813 aziot-edged[22058]: 2022-01-17T12:01:36Z [WARN] - Requesting device reprovision.
Jan 17 14:01:36 l31lnx813 aziot-edged[22058]: 2022-01-17T12:01:36Z [WARN] - The reprovisioning operation failed
Jan 17 14:01:36 l31lnx813 systemd[1]: Started Azure IoT Identity Service.
Jan 17 14:01:36 l31lnx813 aziot-identityd[26978]: 2022-01-17T12:01:36Z [INFO] - Starting service...
Jan 17 14:01:36 l31lnx813 aziot-identityd[26978]: 2022-01-17T12:01:36Z [INFO] - Version - 1.2.4
Jan 17 14:01:36 l31lnx813 aziot-identityd[26978]: 2022-01-17T12:01:36Z [INFO] - Provisioning starting. Reason: Startup
Jan 17 14:01:36 l31lnx813 aziot-identityd[26978]: 2022-01-17T12:01:36Z [WARN] - Failed to clear old module backup state: Permission denied (os error 13)
Jan 17 14:01:36 l31lnx813 aziot-identityd[26978]: 2022-01-17T12:01:36Z [WARN] - Failed to create module information backup state folder: internal error
Jan 17 14:01:36 l31lnx813 aziot-identityd[26978]: 2022-01-17T12:01:36Z [INFO] - Updated device info for cdp-apps-dev-nordland-iot-edge.
Jan 17 14:01:36 l31lnx813 aziot-identityd[26978]: 2022-01-17T12:01:36Z [INFO] - Provisioning complete.
Jan 17 14:01:36 l31lnx813 aziot-identityd[26978]: 2022-01-17T12:01:36Z [INFO] - Identity reconciliation started. Reason: Startup
Jan 17 14:01:36 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:01:36Z [INFO] - <-- GET /key/device-id?api-version=2020-09-01 {"host": "keyd.sock"}
Jan 17 14:01:36 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:01:36Z [ERR!] - Permission denied (os error 13)
Jan 17 14:01:36 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:01:36Z [ERR!] - !!! internal error
Jan 17 14:01:36 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:01:36Z [ERR!] - !!! caused by: could not generate key
Jan 17 14:01:36 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:01:36Z [ERR!] - !!! caused by: could not create key: AZIOT_KEYS_RC_ERR_EXTERNAL
Jan 17 14:01:36 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:01:36Z [INFO] - --> 500 {"content-type": "application/json"}
Jan 17 14:01:36 l31lnx813 aziot-identityd[26978]: 2022-01-17T12:01:36Z [INFO] - Could not reconcile Identities with current device data. Reprovisioning.
Jan 17 14:01:36 l31lnx813 aziot-identityd[26978]: 2022-01-17T12:01:36Z [WARN] - Failed to clear old module backup state: Permission denied (os error 13)
Jan 17 14:01:36 l31lnx813 aziot-identityd[26978]: 2022-01-17T12:01:36Z [WARN] - Failed to create module information backup state folder: internal error
Jan 17 14:01:36 l31lnx813 aziot-identityd[26978]: 2022-01-17T12:01:36Z [INFO] - Updated device info for cdp-apps-dev-nordland-iot-edge.
Jan 17 14:01:36 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:01:36Z [INFO] - <-- GET /key/device-id?api-version=2020-09-01 {"host": "keyd.sock"}
Jan 17 14:01:36 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:01:36Z [ERR!] - Permission denied (os error 13)
Jan 17 14:01:36 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:01:36Z [ERR!] - !!! internal error
Jan 17 14:01:36 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:01:36Z [ERR!] - !!! caused by: could not generate key
Jan 17 14:01:36 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:01:36Z [ERR!] - !!! caused by: could not create key: AZIOT_KEYS_RC_ERR_EXTERNAL
Jan 17 14:01:36 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:01:36Z [INFO] - --> 500 {"content-type": "application/json"}
Jan 17 14:01:36 l31lnx813 aziot-identityd[26978]: 2022-01-17T12:01:36Z [ERR!] - Failed to provision with IoT Hub, and no valid device backup was found: Hub client error
Jan 17 14:01:36 l31lnx813 aziot-identityd[26978]: 2022-01-17T12:01:36Z [ERR!] - service encountered an error
Jan 17 14:01:36 l31lnx813 aziot-identityd[26978]: 2022-01-17T12:01:36Z [ERR!] - caused by: Hub client error
Jan 17 14:01:36 l31lnx813 aziot-identityd[26978]: 2022-01-17T12:01:36Z [ERR!] - caused by: internal error
Jan 17 14:01:36 l31lnx813 aziot-identityd[26978]: 2022-01-17T12:01:36Z [ERR!] -    0: <unknown>
Jan 17 14:01:36 l31lnx813 aziot-identityd[26978]:    1: <unknown>
Jan 17 14:01:36 l31lnx813 systemd[1]: aziot-identityd.service: Main process exited, code=exited, status=1/FAILURE
Jan 17 14:01:36 l31lnx813 systemd[1]: aziot-identityd.service: Failed with result 'exit-code'.
Jan 17 14:01:41 l31lnx813 aziot-edged[22058]: 2022-01-17T12:01:41Z [INFO] - Obtaining edge device provisioning data...
Jan 17 14:01:42 l31lnx813 systemd[1]: aziot-identityd.service: Service hold-off time over, scheduling restart.
Jan 17 14:01:42 l31lnx813 systemd[1]: aziot-identityd.service: Scheduled restart job, restart counter is at 784.
Jan 17 14:01:42 l31lnx813 systemd[1]: Stopped Azure IoT Identity Service.
Jan 17 14:01:42 l31lnx813 aziot-edged[22058]: 2022-01-17T12:01:42Z [WARN] - The daemon could not start up successfully: Could not retrieve device information
Jan 17 14:01:42 l31lnx813 aziot-edged[22058]: 2022-01-17T12:01:42Z [WARN] -         caused by: HTTP request error
Jan 17 14:01:42 l31lnx813 aziot-edged[22058]: 2022-01-17T12:01:42Z [WARN] -         caused by: connection error: Connection reset by peer (os error 104)
Jan 17 14:01:42 l31lnx813 aziot-edged[22058]: 2022-01-17T12:01:42Z [WARN] - Requesting device reprovision.
Jan 17 14:01:42 l31lnx813 aziot-edged[22058]: 2022-01-17T12:01:42Z [WARN] - The reprovisioning operation failed
Jan 17 14:01:42 l31lnx813 systemd[1]: Started Azure IoT Identity Service.
Jan 17 14:01:42 l31lnx813 aziot-identityd[27007]: 2022-01-17T12:01:42Z [INFO] - Starting service...
Jan 17 14:01:42 l31lnx813 aziot-identityd[27007]: 2022-01-17T12:01:42Z [INFO] - Version - 1.2.4
Jan 17 14:01:42 l31lnx813 aziot-identityd[27007]: 2022-01-17T12:01:42Z [INFO] - Provisioning starting. Reason: Startup
Jan 17 14:01:42 l31lnx813 aziot-identityd[27007]: 2022-01-17T12:01:42Z [WARN] - Failed to clear old module backup state: Permission denied (os error 13)
Jan 17 14:01:42 l31lnx813 aziot-identityd[27007]: 2022-01-17T12:01:42Z [WARN] - Failed to create module information backup state folder: internal error
Jan 17 14:01:42 l31lnx813 aziot-identityd[27007]: 2022-01-17T12:01:42Z [INFO] - Updated device info for cdp-apps-dev-nordland-iot-edge.
Jan 17 14:01:42 l31lnx813 aziot-identityd[27007]: 2022-01-17T12:01:42Z [INFO] - Provisioning complete.
Jan 17 14:01:42 l31lnx813 aziot-identityd[27007]: 2022-01-17T12:01:42Z [INFO] - Identity reconciliation started. Reason: Startup
Jan 17 14:01:42 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:01:42Z [INFO] - <-- GET /key/device-id?api-version=2020-09-01 {"host": "keyd.sock"}
Jan 17 14:01:42 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:01:42Z [ERR!] - Permission denied (os error 13)
Jan 17 14:01:42 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:01:42Z [ERR!] - !!! internal error
Jan 17 14:01:42 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:01:42Z [ERR!] - !!! caused by: could not generate key
Jan 17 14:01:42 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:01:42Z [ERR!] - !!! caused by: could not create key: AZIOT_KEYS_RC_ERR_EXTERNAL
Jan 17 14:01:42 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:01:42Z [INFO] - --> 500 {"content-type": "application/json"}
Jan 17 14:01:42 l31lnx813 aziot-identityd[27007]: 2022-01-17T12:01:42Z [INFO] - Could not reconcile Identities with current device data. Reprovisioning.
Jan 17 14:01:42 l31lnx813 aziot-identityd[27007]: 2022-01-17T12:01:42Z [WARN] - Failed to clear old module backup state: Permission denied (os error 13)
Jan 17 14:01:42 l31lnx813 aziot-identityd[27007]: 2022-01-17T12:01:42Z [WARN] - Failed to create module information backup state folder: internal error
Jan 17 14:01:42 l31lnx813 aziot-identityd[27007]: 2022-01-17T12:01:42Z [INFO] - Updated device info for cdp-apps-dev-nordland-iot-edge.
Jan 17 14:01:42 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:01:42Z [INFO] - <-- GET /key/device-id?api-version=2020-09-01 {"host": "keyd.sock"}
Jan 17 14:01:42 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:01:42Z [ERR!] - Permission denied (os error 13)
Jan 17 14:01:42 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:01:42Z [ERR!] - !!! internal error
Jan 17 14:01:42 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:01:42Z [ERR!] - !!! caused by: could not generate key
Jan 17 14:01:42 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:01:42Z [ERR!] - !!! caused by: could not create key: AZIOT_KEYS_RC_ERR_EXTERNAL
Jan 17 14:01:42 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:01:42Z [INFO] - --> 500 {"content-type": "application/json"}
Jan 17 14:01:42 l31lnx813 aziot-identityd[27007]: 2022-01-17T12:01:42Z [ERR!] - Failed to provision with IoT Hub, and no valid device backup was found: Hub client error
Jan 17 14:01:42 l31lnx813 aziot-identityd[27007]: 2022-01-17T12:01:42Z [ERR!] - service encountered an error
Jan 17 14:01:42 l31lnx813 aziot-identityd[27007]: 2022-01-17T12:01:42Z [ERR!] - caused by: Hub client error
Jan 17 14:01:42 l31lnx813 aziot-identityd[27007]: 2022-01-17T12:01:42Z [ERR!] - caused by: internal error
Jan 17 14:01:42 l31lnx813 aziot-identityd[27007]: 2022-01-17T12:01:42Z [ERR!] -    0: <unknown>
Jan 17 14:01:42 l31lnx813 aziot-identityd[27007]:    1: <unknown>
Jan 17 14:01:42 l31lnx813 systemd[1]: aziot-identityd.service: Main process exited, code=exited, status=1/FAILURE
Jan 17 14:01:42 l31lnx813 systemd[1]: aziot-identityd.service: Failed with result 'exit-code'.
Jan 17 14:01:47 l31lnx813 aziot-edged[22058]: 2022-01-17T12:01:47Z [INFO] - Obtaining edge device provisioning data...
Jan 17 14:01:47 l31lnx813 systemd[1]: aziot-identityd.service: Service hold-off time over, scheduling restart.
Jan 17 14:01:47 l31lnx813 systemd[1]: aziot-identityd.service: Scheduled restart job, restart counter is at 785.
Jan 17 14:01:47 l31lnx813 systemd[1]: Stopped Azure IoT Identity Service.
Jan 17 14:01:47 l31lnx813 aziot-edged[22058]: 2022-01-17T12:01:47Z [WARN] - The daemon could not start up successfully: Could not retrieve device information
Jan 17 14:01:47 l31lnx813 aziot-edged[22058]: 2022-01-17T12:01:47Z [WARN] -         caused by: HTTP request error
Jan 17 14:01:47 l31lnx813 aziot-edged[22058]: 2022-01-17T12:01:47Z [WARN] -         caused by: connection error: Connection reset by peer (os error 104)
Jan 17 14:01:47 l31lnx813 aziot-edged[22058]: 2022-01-17T12:01:47Z [WARN] - Requesting device reprovision.
Jan 17 14:01:47 l31lnx813 aziot-edged[22058]: 2022-01-17T12:01:47Z [WARN] - The reprovisioning operation failed
Jan 17 14:01:47 l31lnx813 systemd[1]: Started Azure IoT Identity Service.
Jan 17 14:01:47 l31lnx813 aziot-identityd[27036]: 2022-01-17T12:01:47Z [INFO] - Starting service...
Jan 17 14:01:47 l31lnx813 aziot-identityd[27036]: 2022-01-17T12:01:47Z [INFO] - Version - 1.2.4
Jan 17 14:01:47 l31lnx813 aziot-identityd[27036]: 2022-01-17T12:01:47Z [INFO] - Provisioning starting. Reason: Startup
Jan 17 14:01:47 l31lnx813 aziot-identityd[27036]: 2022-01-17T12:01:47Z [WARN] - Failed to clear old module backup state: Permission denied (os error 13)
Jan 17 14:01:47 l31lnx813 aziot-identityd[27036]: 2022-01-17T12:01:47Z [WARN] - Failed to create module information backup state folder: internal error
Jan 17 14:01:47 l31lnx813 aziot-identityd[27036]: 2022-01-17T12:01:47Z [INFO] - Updated device info for cdp-apps-dev-nordland-iot-edge.
Jan 17 14:01:47 l31lnx813 aziot-identityd[27036]: 2022-01-17T12:01:47Z [INFO] - Provisioning complete.
Jan 17 14:01:47 l31lnx813 aziot-identityd[27036]: 2022-01-17T12:01:47Z [INFO] - Identity reconciliation started. Reason: Startup
Jan 17 14:01:47 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:01:47Z [INFO] - <-- GET /key/device-id?api-version=2020-09-01 {"host": "keyd.sock"}
Jan 17 14:01:47 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:01:47Z [ERR!] - Permission denied (os error 13)
Jan 17 14:01:47 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:01:47Z [ERR!] - !!! internal error
Jan 17 14:01:47 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:01:47Z [ERR!] - !!! caused by: could not generate key
Jan 17 14:01:47 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:01:47Z [ERR!] - !!! caused by: could not create key: AZIOT_KEYS_RC_ERR_EXTERNAL
Jan 17 14:01:47 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:01:47Z [INFO] - --> 500 {"content-type": "application/json"}
Jan 17 14:01:47 l31lnx813 aziot-identityd[27036]: 2022-01-17T12:01:47Z [INFO] - Could not reconcile Identities with current device data. Reprovisioning.
Jan 17 14:01:47 l31lnx813 aziot-identityd[27036]: 2022-01-17T12:01:47Z [WARN] - Failed to clear old module backup state: Permission denied (os error 13)
Jan 17 14:01:47 l31lnx813 aziot-identityd[27036]: 2022-01-17T12:01:47Z [WARN] - Failed to create module information backup state folder: internal error
Jan 17 14:01:47 l31lnx813 aziot-identityd[27036]: 2022-01-17T12:01:47Z [INFO] - Updated device info for cdp-apps-dev-nordland-iot-edge.
Jan 17 14:01:47 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:01:47Z [INFO] - <-- GET /key/device-id?api-version=2020-09-01 {"host": "keyd.sock"}
Jan 17 14:01:47 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:01:47Z [ERR!] - Permission denied (os error 13)
Jan 17 14:01:47 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:01:47Z [ERR!] - !!! internal error
Jan 17 14:01:47 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:01:47Z [ERR!] - !!! caused by: could not generate key
Jan 17 14:01:47 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:01:47Z [ERR!] - !!! caused by: could not create key: AZIOT_KEYS_RC_ERR_EXTERNAL
Jan 17 14:01:47 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:01:47Z [INFO] - --> 500 {"content-type": "application/json"}
Jan 17 14:01:47 l31lnx813 aziot-identityd[27036]: 2022-01-17T12:01:47Z [ERR!] - Failed to provision with IoT Hub, and no valid device backup was found: Hub client error
Jan 17 14:01:47 l31lnx813 aziot-identityd[27036]: 2022-01-17T12:01:47Z [ERR!] - service encountered an error
Jan 17 14:01:47 l31lnx813 aziot-identityd[27036]: 2022-01-17T12:01:47Z [ERR!] - caused by: Hub client error
Jan 17 14:01:47 l31lnx813 aziot-identityd[27036]: 2022-01-17T12:01:47Z [ERR!] - caused by: internal error
Jan 17 14:01:47 l31lnx813 aziot-identityd[27036]: 2022-01-17T12:01:47Z [ERR!] -    0: <unknown>
Jan 17 14:01:47 l31lnx813 aziot-identityd[27036]:    1: <unknown>
Jan 17 14:01:47 l31lnx813 systemd[1]: aziot-identityd.service: Main process exited, code=exited, status=1/FAILURE
Jan 17 14:01:47 l31lnx813 systemd[1]: aziot-identityd.service: Failed with result 'exit-code'.
Jan 17 14:01:52 l31lnx813 aziot-edged[22058]: 2022-01-17T12:01:52Z [INFO] - Obtaining edge device provisioning data...
Jan 17 14:01:52 l31lnx813 systemd[1]: aziot-identityd.service: Service hold-off time over, scheduling restart.
Jan 17 14:01:52 l31lnx813 systemd[1]: aziot-identityd.service: Scheduled restart job, restart counter is at 786.
Jan 17 14:01:52 l31lnx813 systemd[1]: Stopped Azure IoT Identity Service.
Jan 17 14:01:52 l31lnx813 aziot-edged[22058]: 2022-01-17T12:01:52Z [WARN] - The daemon could not start up successfully: Could not retrieve device information
Jan 17 14:01:52 l31lnx813 aziot-edged[22058]: 2022-01-17T12:01:52Z [WARN] -         caused by: HTTP request error
Jan 17 14:01:52 l31lnx813 aziot-edged[22058]: 2022-01-17T12:01:52Z [WARN] -         caused by: connection error: Connection reset by peer (os error 104)
Jan 17 14:01:52 l31lnx813 aziot-edged[22058]: 2022-01-17T12:01:52Z [WARN] - Requesting device reprovision.
Jan 17 14:01:52 l31lnx813 aziot-edged[22058]: 2022-01-17T12:01:52Z [WARN] - The reprovisioning operation failed
Jan 17 14:01:52 l31lnx813 systemd[1]: Started Azure IoT Identity Service.
Jan 17 14:01:52 l31lnx813 aziot-identityd[27072]: 2022-01-17T12:01:52Z [INFO] - Starting service...
Jan 17 14:01:52 l31lnx813 aziot-identityd[27072]: 2022-01-17T12:01:52Z [INFO] - Version - 1.2.4
Jan 17 14:01:52 l31lnx813 aziot-identityd[27072]: 2022-01-17T12:01:52Z [INFO] - Provisioning starting. Reason: Startup
Jan 17 14:01:52 l31lnx813 aziot-identityd[27072]: 2022-01-17T12:01:52Z [WARN] - Failed to clear old module backup state: Permission denied (os error 13)
Jan 17 14:01:52 l31lnx813 aziot-identityd[27072]: 2022-01-17T12:01:52Z [WARN] - Failed to create module information backup state folder: internal error
Jan 17 14:01:52 l31lnx813 aziot-identityd[27072]: 2022-01-17T12:01:52Z [INFO] - Updated device info for cdp-apps-dev-nordland-iot-edge.
Jan 17 14:01:52 l31lnx813 aziot-identityd[27072]: 2022-01-17T12:01:52Z [INFO] - Provisioning complete.
Jan 17 14:01:52 l31lnx813 aziot-identityd[27072]: 2022-01-17T12:01:52Z [INFO] - Identity reconciliation started. Reason: Startup
Jan 17 14:01:52 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:01:52Z [INFO] - <-- GET /key/device-id?api-version=2020-09-01 {"host": "keyd.sock"}
Jan 17 14:01:52 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:01:52Z [ERR!] - Permission denied (os error 13)
Jan 17 14:01:52 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:01:52Z [ERR!] - !!! internal error
Jan 17 14:01:52 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:01:52Z [ERR!] - !!! caused by: could not generate key
Jan 17 14:01:52 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:01:52Z [ERR!] - !!! caused by: could not create key: AZIOT_KEYS_RC_ERR_EXTERNAL
Jan 17 14:01:52 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:01:52Z [INFO] - --> 500 {"content-type": "application/json"}
Jan 17 14:01:52 l31lnx813 aziot-identityd[27072]: 2022-01-17T12:01:52Z [INFO] - Could not reconcile Identities with current device data. Reprovisioning.
Jan 17 14:01:52 l31lnx813 aziot-identityd[27072]: 2022-01-17T12:01:52Z [WARN] - Failed to clear old module backup state: Permission denied (os error 13)
Jan 17 14:01:52 l31lnx813 aziot-identityd[27072]: 2022-01-17T12:01:52Z [WARN] - Failed to create module information backup state folder: internal error
Jan 17 14:01:52 l31lnx813 aziot-identityd[27072]: 2022-01-17T12:01:52Z [INFO] - Updated device info for cdp-apps-dev-nordland-iot-edge.
Jan 17 14:01:52 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:01:52Z [INFO] - <-- GET /key/device-id?api-version=2020-09-01 {"host": "keyd.sock"}
Jan 17 14:01:52 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:01:52Z [ERR!] - Permission denied (os error 13)
Jan 17 14:01:52 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:01:52Z [ERR!] - !!! internal error
Jan 17 14:01:52 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:01:52Z [ERR!] - !!! caused by: could not generate key
Jan 17 14:01:52 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:01:52Z [ERR!] - !!! caused by: could not create key: AZIOT_KEYS_RC_ERR_EXTERNAL
Jan 17 14:01:52 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:01:52Z [INFO] - --> 500 {"content-type": "application/json"}
Jan 17 14:01:52 l31lnx813 aziot-identityd[27072]: 2022-01-17T12:01:52Z [ERR!] - Failed to provision with IoT Hub, and no valid device backup was found: Hub client error
Jan 17 14:01:52 l31lnx813 aziot-identityd[27072]: 2022-01-17T12:01:52Z [ERR!] - service encountered an error
Jan 17 14:01:52 l31lnx813 aziot-identityd[27072]: 2022-01-17T12:01:52Z [ERR!] - caused by: Hub client error
Jan 17 14:01:52 l31lnx813 aziot-identityd[27072]: 2022-01-17T12:01:52Z [ERR!] - caused by: internal error
Jan 17 14:01:52 l31lnx813 aziot-identityd[27072]: 2022-01-17T12:01:52Z [ERR!] -    0: <unknown>
Jan 17 14:01:52 l31lnx813 aziot-identityd[27072]:    1: <unknown>
Jan 17 14:01:52 l31lnx813 systemd[1]: aziot-identityd.service: Main process exited, code=exited, status=1/FAILURE
Jan 17 14:01:52 l31lnx813 systemd[1]: aziot-identityd.service: Failed with result 'exit-code'.
Jan 17 14:01:57 l31lnx813 aziot-edged[22058]: 2022-01-17T12:01:57Z [INFO] - Obtaining edge device provisioning data...
Jan 17 14:01:57 l31lnx813 systemd[1]: aziot-identityd.service: Service hold-off time over, scheduling restart.
Jan 17 14:01:57 l31lnx813 systemd[1]: aziot-identityd.service: Scheduled restart job, restart counter is at 787.
Jan 17 14:01:57 l31lnx813 systemd[1]: Stopped Azure IoT Identity Service.
Jan 17 14:01:57 l31lnx813 aziot-edged[22058]: 2022-01-17T12:01:57Z [WARN] - The daemon could not start up successfully: Could not retrieve device information
Jan 17 14:01:57 l31lnx813 aziot-edged[22058]: 2022-01-17T12:01:57Z [WARN] -         caused by: HTTP request error
Jan 17 14:01:57 l31lnx813 aziot-edged[22058]: 2022-01-17T12:01:57Z [WARN] -         caused by: connection error: Connection reset by peer (os error 104)
Jan 17 14:01:57 l31lnx813 aziot-edged[22058]: 2022-01-17T12:01:57Z [WARN] - Requesting device reprovision.
Jan 17 14:01:57 l31lnx813 aziot-edged[22058]: 2022-01-17T12:01:57Z [WARN] - The reprovisioning operation failed
Jan 17 14:01:57 l31lnx813 systemd[1]: Started Azure IoT Identity Service.
Jan 17 14:01:57 l31lnx813 aziot-identityd[27090]: 2022-01-17T12:01:57Z [INFO] - Starting service...
Jan 17 14:01:57 l31lnx813 aziot-identityd[27090]: 2022-01-17T12:01:57Z [INFO] - Version - 1.2.4
Jan 17 14:01:57 l31lnx813 aziot-identityd[27090]: 2022-01-17T12:01:57Z [INFO] - Provisioning starting. Reason: Startup
Jan 17 14:01:57 l31lnx813 aziot-identityd[27090]: 2022-01-17T12:01:57Z [WARN] - Failed to clear old module backup state: Permission denied (os error 13)
Jan 17 14:01:57 l31lnx813 aziot-identityd[27090]: 2022-01-17T12:01:57Z [WARN] - Failed to create module information backup state folder: internal error
Jan 17 14:01:57 l31lnx813 aziot-identityd[27090]: 2022-01-17T12:01:57Z [INFO] - Updated device info for cdp-apps-dev-nordland-iot-edge.
Jan 17 14:01:57 l31lnx813 aziot-identityd[27090]: 2022-01-17T12:01:57Z [INFO] - Provisioning complete.
Jan 17 14:01:57 l31lnx813 aziot-identityd[27090]: 2022-01-17T12:01:57Z [INFO] - Identity reconciliation started. Reason: Startup
Jan 17 14:01:57 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:01:57Z [INFO] - <-- GET /key/device-id?api-version=2020-09-01 {"host": "keyd.sock"}
Jan 17 14:01:57 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:01:57Z [ERR!] - Permission denied (os error 13)
Jan 17 14:01:57 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:01:57Z [ERR!] - !!! internal error
Jan 17 14:01:57 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:01:57Z [ERR!] - !!! caused by: could not generate key
Jan 17 14:01:57 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:01:57Z [ERR!] - !!! caused by: could not create key: AZIOT_KEYS_RC_ERR_EXTERNAL
Jan 17 14:01:57 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:01:57Z [INFO] - --> 500 {"content-type": "application/json"}
Jan 17 14:01:57 l31lnx813 aziot-identityd[27090]: 2022-01-17T12:01:57Z [INFO] - Could not reconcile Identities with current device data. Reprovisioning.
Jan 17 14:01:57 l31lnx813 aziot-identityd[27090]: 2022-01-17T12:01:57Z [WARN] - Failed to clear old module backup state: Permission denied (os error 13)
Jan 17 14:01:57 l31lnx813 aziot-identityd[27090]: 2022-01-17T12:01:57Z [WARN] - Failed to create module information backup state folder: internal error
Jan 17 14:01:57 l31lnx813 aziot-identityd[27090]: 2022-01-17T12:01:57Z [INFO] - Updated device info for cdp-apps-dev-nordland-iot-edge.
Jan 17 14:01:57 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:01:57Z [INFO] - <-- GET /key/device-id?api-version=2020-09-01 {"host": "keyd.sock"}
Jan 17 14:01:57 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:01:57Z [ERR!] - Permission denied (os error 13)
Jan 17 14:01:57 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:01:57Z [ERR!] - !!! internal error
Jan 17 14:01:57 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:01:57Z [ERR!] - !!! caused by: could not generate key
Jan 17 14:01:57 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:01:57Z [ERR!] - !!! caused by: could not create key: AZIOT_KEYS_RC_ERR_EXTERNAL
Jan 17 14:01:57 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:01:57Z [INFO] - --> 500 {"content-type": "application/json"}
Jan 17 14:01:57 l31lnx813 aziot-identityd[27090]: 2022-01-17T12:01:57Z [ERR!] - Failed to provision with IoT Hub, and no valid device backup was found: Hub client error
Jan 17 14:01:57 l31lnx813 aziot-identityd[27090]: 2022-01-17T12:01:57Z [ERR!] - service encountered an error
Jan 17 14:01:57 l31lnx813 aziot-identityd[27090]: 2022-01-17T12:01:57Z [ERR!] - caused by: Hub client error
Jan 17 14:01:57 l31lnx813 aziot-identityd[27090]: 2022-01-17T12:01:57Z [ERR!] - caused by: internal error
Jan 17 14:01:57 l31lnx813 aziot-identityd[27090]: 2022-01-17T12:01:57Z [ERR!] -    0: <unknown>
Jan 17 14:01:57 l31lnx813 aziot-identityd[27090]:    1: <unknown>
Jan 17 14:01:57 l31lnx813 systemd[1]: aziot-identityd.service: Main process exited, code=exited, status=1/FAILURE
Jan 17 14:01:57 l31lnx813 systemd[1]: aziot-identityd.service: Failed with result 'exit-code'.
Jan 17 14:02:02 l31lnx813 aziot-edged[22058]: 2022-01-17T12:02:02Z [INFO] - Obtaining edge device provisioning data...
Jan 17 14:02:03 l31lnx813 systemd[1]: aziot-identityd.service: Service hold-off time over, scheduling restart.
Jan 17 14:02:03 l31lnx813 systemd[1]: aziot-identityd.service: Scheduled restart job, restart counter is at 788.
Jan 17 14:02:03 l31lnx813 systemd[1]: Stopped Azure IoT Identity Service.
Jan 17 14:02:03 l31lnx813 aziot-edged[22058]: 2022-01-17T12:02:03Z [WARN] - The daemon could not start up successfully: Could not retrieve device information
Jan 17 14:02:03 l31lnx813 aziot-edged[22058]: 2022-01-17T12:02:03Z [WARN] -         caused by: HTTP request error
Jan 17 14:02:03 l31lnx813 aziot-edged[22058]: 2022-01-17T12:02:03Z [WARN] -         caused by: connection error: Connection reset by peer (os error 104)
Jan 17 14:02:03 l31lnx813 aziot-edged[22058]: 2022-01-17T12:02:03Z [WARN] - Requesting device reprovision.
Jan 17 14:02:03 l31lnx813 aziot-edged[22058]: 2022-01-17T12:02:03Z [WARN] - The reprovisioning operation failed
Jan 17 14:02:03 l31lnx813 systemd[1]: Started Azure IoT Identity Service.
Jan 17 14:02:03 l31lnx813 aziot-identityd[27127]: 2022-01-17T12:02:03Z [INFO] - Starting service...
Jan 17 14:02:03 l31lnx813 aziot-identityd[27127]: 2022-01-17T12:02:03Z [INFO] - Version - 1.2.4
Jan 17 14:02:03 l31lnx813 aziot-identityd[27127]: 2022-01-17T12:02:03Z [INFO] - Provisioning starting. Reason: Startup
Jan 17 14:02:03 l31lnx813 aziot-identityd[27127]: 2022-01-17T12:02:03Z [WARN] - Failed to clear old module backup state: Permission denied (os error 13)
Jan 17 14:02:03 l31lnx813 aziot-identityd[27127]: 2022-01-17T12:02:03Z [WARN] - Failed to create module information backup state folder: internal error
Jan 17 14:02:03 l31lnx813 aziot-identityd[27127]: 2022-01-17T12:02:03Z [INFO] - Updated device info for cdp-apps-dev-nordland-iot-edge.
Jan 17 14:02:03 l31lnx813 aziot-identityd[27127]: 2022-01-17T12:02:03Z [INFO] - Provisioning complete.
Jan 17 14:02:03 l31lnx813 aziot-identityd[27127]: 2022-01-17T12:02:03Z [INFO] - Identity reconciliation started. Reason: Startup
Jan 17 14:02:03 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:02:03Z [INFO] - <-- GET /key/device-id?api-version=2020-09-01 {"host": "keyd.sock"}
Jan 17 14:02:03 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:02:03Z [ERR!] - Permission denied (os error 13)
Jan 17 14:02:03 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:02:03Z [ERR!] - !!! internal error
Jan 17 14:02:03 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:02:03Z [ERR!] - !!! caused by: could not generate key
Jan 17 14:02:03 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:02:03Z [ERR!] - !!! caused by: could not create key: AZIOT_KEYS_RC_ERR_EXTERNAL
Jan 17 14:02:03 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:02:03Z [INFO] - --> 500 {"content-type": "application/json"}
Jan 17 14:02:03 l31lnx813 aziot-identityd[27127]: 2022-01-17T12:02:03Z [INFO] - Could not reconcile Identities with current device data. Reprovisioning.
Jan 17 14:02:03 l31lnx813 aziot-identityd[27127]: 2022-01-17T12:02:03Z [WARN] - Failed to clear old module backup state: Permission denied (os error 13)
Jan 17 14:02:03 l31lnx813 aziot-identityd[27127]: 2022-01-17T12:02:03Z [WARN] - Failed to create module information backup state folder: internal error
Jan 17 14:02:03 l31lnx813 aziot-identityd[27127]: 2022-01-17T12:02:03Z [INFO] - Updated device info for cdp-apps-dev-nordland-iot-edge.
Jan 17 14:02:03 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:02:03Z [INFO] - <-- GET /key/device-id?api-version=2020-09-01 {"host": "keyd.sock"}
Jan 17 14:02:03 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:02:03Z [ERR!] - Permission denied (os error 13)
Jan 17 14:02:03 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:02:03Z [ERR!] - !!! internal error
Jan 17 14:02:03 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:02:03Z [ERR!] - !!! caused by: could not generate key
Jan 17 14:02:03 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:02:03Z [ERR!] - !!! caused by: could not create key: AZIOT_KEYS_RC_ERR_EXTERNAL
Jan 17 14:02:03 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:02:03Z [INFO] - --> 500 {"content-type": "application/json"}
Jan 17 14:02:03 l31lnx813 aziot-identityd[27127]: 2022-01-17T12:02:03Z [ERR!] - Failed to provision with IoT Hub, and no valid device backup was found: Hub client error
Jan 17 14:02:03 l31lnx813 aziot-identityd[27127]: 2022-01-17T12:02:03Z [ERR!] - service encountered an error
Jan 17 14:02:03 l31lnx813 aziot-identityd[27127]: 2022-01-17T12:02:03Z [ERR!] - caused by: Hub client error
Jan 17 14:02:03 l31lnx813 aziot-identityd[27127]: 2022-01-17T12:02:03Z [ERR!] - caused by: internal error
Jan 17 14:02:03 l31lnx813 aziot-identityd[27127]: 2022-01-17T12:02:03Z [ERR!] -    0: <unknown>
Jan 17 14:02:03 l31lnx813 aziot-identityd[27127]:    1: <unknown>
Jan 17 14:02:03 l31lnx813 systemd[1]: aziot-identityd.service: Main process exited, code=exited, status=1/FAILURE
Jan 17 14:02:03 l31lnx813 systemd[1]: aziot-identityd.service: Failed with result 'exit-code'.
Jan 17 14:02:08 l31lnx813 aziot-edged[22058]: 2022-01-17T12:02:08Z [INFO] - Obtaining edge device provisioning data...
Jan 17 14:02:08 l31lnx813 systemd[1]: aziot-identityd.service: Service hold-off time over, scheduling restart.
Jan 17 14:02:08 l31lnx813 systemd[1]: aziot-identityd.service: Scheduled restart job, restart counter is at 789.
Jan 17 14:02:08 l31lnx813 systemd[1]: Stopped Azure IoT Identity Service.
Jan 17 14:02:08 l31lnx813 aziot-edged[22058]: 2022-01-17T12:02:08Z [WARN] - The daemon could not start up successfully: Could not retrieve device information
Jan 17 14:02:08 l31lnx813 aziot-edged[22058]: 2022-01-17T12:02:08Z [WARN] -         caused by: HTTP request error
Jan 17 14:02:08 l31lnx813 aziot-edged[22058]: 2022-01-17T12:02:08Z [WARN] -         caused by: connection error: Connection reset by peer (os error 104)
Jan 17 14:02:08 l31lnx813 aziot-edged[22058]: 2022-01-17T12:02:08Z [WARN] - Requesting device reprovision.
Jan 17 14:02:08 l31lnx813 aziot-edged[22058]: 2022-01-17T12:02:08Z [WARN] - The reprovisioning operation failed
Jan 17 14:02:08 l31lnx813 systemd[1]: Started Azure IoT Identity Service.
Jan 17 14:02:08 l31lnx813 aziot-identityd[27148]: 2022-01-17T12:02:08Z [INFO] - Starting service...
Jan 17 14:02:08 l31lnx813 aziot-identityd[27148]: 2022-01-17T12:02:08Z [INFO] - Version - 1.2.4
Jan 17 14:02:08 l31lnx813 aziot-identityd[27148]: 2022-01-17T12:02:08Z [INFO] - Provisioning starting. Reason: Startup
Jan 17 14:02:08 l31lnx813 aziot-identityd[27148]: 2022-01-17T12:02:08Z [WARN] - Failed to clear old module backup state: Permission denied (os error 13)
Jan 17 14:02:08 l31lnx813 aziot-identityd[27148]: 2022-01-17T12:02:08Z [WARN] - Failed to create module information backup state folder: internal error
Jan 17 14:02:08 l31lnx813 aziot-identityd[27148]: 2022-01-17T12:02:08Z [INFO] - Updated device info for cdp-apps-dev-nordland-iot-edge.
Jan 17 14:02:08 l31lnx813 aziot-identityd[27148]: 2022-01-17T12:02:08Z [INFO] - Provisioning complete.
Jan 17 14:02:08 l31lnx813 aziot-identityd[27148]: 2022-01-17T12:02:08Z [INFO] - Identity reconciliation started. Reason: Startup
Jan 17 14:02:08 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:02:08Z [INFO] - <-- GET /key/device-id?api-version=2020-09-01 {"host": "keyd.sock"}
Jan 17 14:02:08 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:02:08Z [ERR!] - Permission denied (os error 13)
Jan 17 14:02:08 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:02:08Z [ERR!] - !!! internal error
Jan 17 14:02:08 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:02:08Z [ERR!] - !!! caused by: could not generate key
Jan 17 14:02:08 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:02:08Z [ERR!] - !!! caused by: could not create key: AZIOT_KEYS_RC_ERR_EXTERNAL
Jan 17 14:02:08 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:02:08Z [INFO] - --> 500 {"content-type": "application/json"}
Jan 17 14:02:08 l31lnx813 aziot-identityd[27148]: 2022-01-17T12:02:08Z [INFO] - Could not reconcile Identities with current device data. Reprovisioning.
Jan 17 14:02:08 l31lnx813 aziot-identityd[27148]: 2022-01-17T12:02:08Z [WARN] - Failed to clear old module backup state: Permission denied (os error 13)
Jan 17 14:02:08 l31lnx813 aziot-identityd[27148]: 2022-01-17T12:02:08Z [WARN] - Failed to create module information backup state folder: internal error
Jan 17 14:02:08 l31lnx813 aziot-identityd[27148]: 2022-01-17T12:02:08Z [INFO] - Updated device info for cdp-apps-dev-nordland-iot-edge.
Jan 17 14:02:08 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:02:08Z [INFO] - <-- GET /key/device-id?api-version=2020-09-01 {"host": "keyd.sock"}
Jan 17 14:02:08 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:02:08Z [ERR!] - Permission denied (os error 13)
Jan 17 14:02:08 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:02:08Z [ERR!] - !!! internal error
Jan 17 14:02:08 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:02:08Z [ERR!] - !!! caused by: could not generate key
Jan 17 14:02:08 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:02:08Z [ERR!] - !!! caused by: could not create key: AZIOT_KEYS_RC_ERR_EXTERNAL
Jan 17 14:02:08 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:02:08Z [INFO] - --> 500 {"content-type": "application/json"}
Jan 17 14:02:08 l31lnx813 aziot-identityd[27148]: 2022-01-17T12:02:08Z [ERR!] - Failed to provision with IoT Hub, and no valid device backup was found: Hub client error
Jan 17 14:02:08 l31lnx813 aziot-identityd[27148]: 2022-01-17T12:02:08Z [ERR!] - service encountered an error
Jan 17 14:02:08 l31lnx813 aziot-identityd[27148]: 2022-01-17T12:02:08Z [ERR!] - caused by: Hub client error
Jan 17 14:02:08 l31lnx813 aziot-identityd[27148]: 2022-01-17T12:02:08Z [ERR!] - caused by: internal error
Jan 17 14:02:08 l31lnx813 aziot-identityd[27148]: 2022-01-17T12:02:08Z [ERR!] -    0: <unknown>
Jan 17 14:02:08 l31lnx813 aziot-identityd[27148]:    1: <unknown>
Jan 17 14:02:08 l31lnx813 systemd[1]: aziot-identityd.service: Main process exited, code=exited, status=1/FAILURE
Jan 17 14:02:08 l31lnx813 systemd[1]: aziot-identityd.service: Failed with result 'exit-code'.
Jan 17 14:02:13 l31lnx813 aziot-edged[22058]: 2022-01-17T12:02:13Z [INFO] - Obtaining edge device provisioning data...
Jan 17 14:02:13 l31lnx813 systemd[1]: aziot-identityd.service: Service hold-off time over, scheduling restart.
Jan 17 14:02:13 l31lnx813 systemd[1]: aziot-identityd.service: Scheduled restart job, restart counter is at 790.
Jan 17 14:02:13 l31lnx813 systemd[1]: Stopped Azure IoT Identity Service.
Jan 17 14:02:13 l31lnx813 aziot-edged[22058]: 2022-01-17T12:02:13Z [WARN] - The daemon could not start up successfully: Could not retrieve device information
Jan 17 14:02:13 l31lnx813 aziot-edged[22058]: 2022-01-17T12:02:13Z [WARN] -         caused by: HTTP request error
Jan 17 14:02:13 l31lnx813 aziot-edged[22058]: 2022-01-17T12:02:13Z [WARN] -         caused by: connection error: Connection reset by peer (os error 104)
Jan 17 14:02:13 l31lnx813 aziot-edged[22058]: 2022-01-17T12:02:13Z [WARN] - Requesting device reprovision.
Jan 17 14:02:13 l31lnx813 aziot-edged[22058]: 2022-01-17T12:02:13Z [WARN] - The reprovisioning operation failed
Jan 17 14:02:13 l31lnx813 systemd[1]: Started Azure IoT Identity Service.
Jan 17 14:02:13 l31lnx813 aziot-identityd[27199]: 2022-01-17T12:02:13Z [INFO] - Starting service...
Jan 17 14:02:13 l31lnx813 aziot-identityd[27199]: 2022-01-17T12:02:13Z [INFO] - Version - 1.2.4
Jan 17 14:02:13 l31lnx813 aziot-identityd[27199]: 2022-01-17T12:02:13Z [INFO] - Provisioning starting. Reason: Startup
Jan 17 14:02:13 l31lnx813 aziot-identityd[27199]: 2022-01-17T12:02:13Z [WARN] - Failed to clear old module backup state: Permission denied (os error 13)
Jan 17 14:02:13 l31lnx813 aziot-identityd[27199]: 2022-01-17T12:02:13Z [WARN] - Failed to create module information backup state folder: internal error
Jan 17 14:02:13 l31lnx813 aziot-identityd[27199]: 2022-01-17T12:02:13Z [INFO] - Updated device info for cdp-apps-dev-nordland-iot-edge.
Jan 17 14:02:13 l31lnx813 aziot-identityd[27199]: 2022-01-17T12:02:13Z [INFO] - Provisioning complete.
Jan 17 14:02:13 l31lnx813 aziot-identityd[27199]: 2022-01-17T12:02:13Z [INFO] - Identity reconciliation started. Reason: Startup
Jan 17 14:02:13 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:02:13Z [INFO] - <-- GET /key/device-id?api-version=2020-09-01 {"host": "keyd.sock"}
Jan 17 14:02:13 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:02:13Z [ERR!] - Permission denied (os error 13)
Jan 17 14:02:13 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:02:13Z [ERR!] - !!! internal error
Jan 17 14:02:13 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:02:13Z [ERR!] - !!! caused by: could not generate key
Jan 17 14:02:13 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:02:13Z [ERR!] - !!! caused by: could not create key: AZIOT_KEYS_RC_ERR_EXTERNAL
Jan 17 14:02:13 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:02:13Z [INFO] - --> 500 {"content-type": "application/json"}
Jan 17 14:02:13 l31lnx813 aziot-identityd[27199]: 2022-01-17T12:02:13Z [INFO] - Could not reconcile Identities with current device data. Reprovisioning.
Jan 17 14:02:13 l31lnx813 aziot-identityd[27199]: 2022-01-17T12:02:13Z [WARN] - Failed to clear old module backup state: Permission denied (os error 13)
Jan 17 14:02:13 l31lnx813 aziot-identityd[27199]: 2022-01-17T12:02:13Z [WARN] - Failed to create module information backup state folder: internal error
Jan 17 14:02:13 l31lnx813 aziot-identityd[27199]: 2022-01-17T12:02:13Z [INFO] - Updated device info for cdp-apps-dev-nordland-iot-edge.
Jan 17 14:02:13 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:02:13Z [INFO] - <-- GET /key/device-id?api-version=2020-09-01 {"host": "keyd.sock"}
Jan 17 14:02:13 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:02:13Z [ERR!] - Permission denied (os error 13)
Jan 17 14:02:13 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:02:13Z [ERR!] - !!! internal error
Jan 17 14:02:13 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:02:13Z [ERR!] - !!! caused by: could not generate key
Jan 17 14:02:13 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:02:13Z [ERR!] - !!! caused by: could not create key: AZIOT_KEYS_RC_ERR_EXTERNAL
Jan 17 14:02:13 l31lnx813 aziot-keyd[22078]: 2022-01-17T12:02:13Z [INFO] - --> 500 {"content-type": "application/json"}
Jan 17 14:02:13 l31lnx813 aziot-identityd[27199]: 2022-01-17T12:02:13Z [ERR!] - Failed to provision with IoT Hub, and no valid device backup was found: Hub client error
Jan 17 14:02:13 l31lnx813 aziot-identityd[27199]: 2022-01-17T12:02:13Z [ERR!] - service encountered an error
Jan 17 14:02:13 l31lnx813 aziot-identityd[27199]: 2022-01-17T12:02:13Z [ERR!] - caused by: Hub client error
Jan 17 14:02:13 l31lnx813 aziot-identityd[27199]: 2022-01-17T12:02:13Z [ERR!] - caused by: internal error
Jan 17 14:02:13 l31lnx813 aziot-identityd[27199]: 2022-01-17T12:02:13Z [ERR!] -    0: <unknown>
Jan 17 14:02:13 l31lnx813 aziot-identityd[27199]:    1: <unknown>
Jan 17 14:02:13 l31lnx813 systemd[1]: aziot-identityd.service: Main process exited, code=exited, status=1/FAILURE
Jan 17 14:02:13 l31lnx813 systemd[1]: aziot-identityd.service: Failed with result 'exit-code'.
Jan 17 14:02:15 l31lnx813 systemd[1]: aziot-edged.service: Current command vanished from the unit file, execution of the command list won't be resumed.
Jan 17 14:02:15 l31lnx813 systemd[1]: Stopped Azure IoT Identity Service.
Jan 17 14:02:15 l31lnx813 systemd[1]: Stopping Azure IoT Keys Service...
Jan 17 14:02:15 l31lnx813 systemd[1]: Stopped Azure IoT Keys Service.
Jan 17 14:02:16 l31lnx813 systemd[1]: aziot-edged.service: Main process exited, code=killed, status=9/KILL
Jan 17 14:02:16 l31lnx813 systemd[1]: aziot-edged.service: Failed with result 'signal'.
Jan 17 14:04:25 l31lnx813 systemd[1]: Started Azure IoT Edge daemon.
Jan 17 14:04:25 l31lnx813 aziot-edged[28814]: 2022-01-17T12:04:25Z [INFO] - Starting Azure IoT Edge Module Runtime
Jan 17 14:04:25 l31lnx813 aziot-edged[28814]: 2022-01-17T12:04:25Z [INFO] - Version - 1.2.5
Jan 17 14:04:25 l31lnx813 aziot-edged[28814]: 2022-01-17T12:04:25Z [INFO] - Initializing the module runtime...
Jan 17 14:04:25 l31lnx813 aziot-edged[28814]: 2022-01-17T12:04:25Z [INFO] - Initializing module runtime...
Jan 17 14:04:25 l31lnx813 aziot-edged[28814]: 2022-01-17T12:04:25Z [INFO] - Using runtime network id azure-iot-edge
Jan 17 14:04:25 l31lnx813 aziot-edged[28814]: 2022-01-17T12:04:25Z [INFO] - Successfully initialized module runtime
Jan 17 14:04:25 l31lnx813 aziot-edged[28814]: 2022-01-17T12:04:25Z [INFO] - Finished initializing the module runtime.
Jan 17 14:04:25 l31lnx813 aziot-edged[28814]: 2022-01-17T12:04:25Z [INFO] - Obtaining edge device provisioning data...
Jan 17 14:04:25 l31lnx813 systemd[1]: Started Azure IoT Identity Service.
Jan 17 14:04:25 l31lnx813 aziot-identityd[28819]: 2022-01-17T12:04:25Z [INFO] - Starting service...
Jan 17 14:04:25 l31lnx813 aziot-identityd[28819]: 2022-01-17T12:04:25Z [INFO] - Version - 1.2.4
Jan 17 14:04:25 l31lnx813 aziot-identityd[28819]: 2022-01-17T12:04:25Z [INFO] - Provisioning starting. Reason: Startup
Jan 17 14:04:25 l31lnx813 aziot-identityd[28819]: 2022-01-17T12:04:25Z [INFO] - Updated device info for cdp-apps-dev-nordland-iot-edge.
Jan 17 14:04:25 l31lnx813 aziot-identityd[28819]: 2022-01-17T12:04:25Z [INFO] - Provisioning complete.
Jan 17 14:04:25 l31lnx813 aziot-identityd[28819]: 2022-01-17T12:04:25Z [INFO] - Identity reconciliation started. Reason: Startup
Jan 17 14:04:25 l31lnx813 systemd[1]: Started Azure IoT Keys Service.
Jan 17 14:04:25 l31lnx813 aziot-keyd[28832]: 2022-01-17T12:04:25Z [INFO] - Starting service...
Jan 17 14:04:25 l31lnx813 aziot-keyd[28832]: 2022-01-17T12:04:25Z [INFO] - Version - 1.2.4
Jan 17 14:04:25 l31lnx813 aziot-keyd[28832]: 2022-01-17T12:04:25Z [INFO] - Loaded libaziot-keys with version 0x02000000
Jan 17 14:04:25 l31lnx813 aziot-keyd[28832]: 2022-01-17T12:04:25Z [INFO] - Starting server...
Jan 17 14:04:25 l31lnx813 aziot-keyd[28832]: 2022-01-17T12:04:25Z [INFO] - <-- GET /key/device-id?api-version=2020-09-01 {"host": "keyd.sock"}
Jan 17 14:04:25 l31lnx813 aziot-keyd[28832]: 2022-01-17T12:04:25Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:04:25 l31lnx813 aziot-keyd[28832]: 2022-01-17T12:04:25Z [INFO] - <-- POST /sign?api-version=2020-09-01 {"content-type": "application/json", "host": "keyd.sock", "content-length": "414"}
Jan 17 14:04:25 l31lnx813 aziot-keyd[28832]: 2022-01-17T12:04:25Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:04:26 l31lnx813 aziot-identityd[28819]: 2022-01-17T12:04:26Z [INFO] - Identity reconciliation complete.
Jan 17 14:04:26 l31lnx813 aziot-identityd[28819]: 2022-01-17T12:04:26Z [INFO] - Starting server...
Jan 17 14:04:26 l31lnx813 aziot-identityd[28819]: 2022-01-17T12:04:26Z [INFO] - <-- POST /identities/device?api-version=2020-09-01 {"content-type": "application/json", "host": "2f72756e2f617a696f742f6964656e74697479642e736f636b:0", "content-length": "16"}
Jan 17 14:04:26 l31lnx813 aziot-keyd[28832]: 2022-01-17T12:04:26Z [INFO] - <-- GET /key/device-id?api-version=2020-09-01 {"host": "keyd.sock"}
Jan 17 14:04:26 l31lnx813 aziot-keyd[28832]: 2022-01-17T12:04:26Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:04:26 l31lnx813 aziot-identityd[28819]: 2022-01-17T12:04:26Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:04:26 l31lnx813 aziot-edged[28814]: 2022-01-17T12:04:26Z [INFO] - Finished provisioning edge device.
Jan 17 14:04:26 l31lnx813 aziot-edged[28814]: 2022-01-17T12:04:26Z [INFO] - Stopping all modules...
Jan 17 14:04:26 l31lnx813 aziot-edged[28814]: 2022-01-17T12:04:26Z [INFO] - Finished stopping modules.
Jan 17 14:04:26 l31lnx813 aziot-edged[28814]: 2022-01-17T12:04:26Z [INFO] - Detecting if device information has changed...
Jan 17 14:04:26 l31lnx813 aziot-edged[28814]: 2022-01-17T12:04:26Z [INFO] - Change to provisioning state detected.
Jan 17 14:04:26 l31lnx813 aziot-edged[28814]: 2022-01-17T12:04:26Z [INFO] - Removing all modules...
Jan 17 14:04:26 l31lnx813 aziot-edged[28814]: 2022-01-17T12:04:26Z [INFO] - Finished removing modules.
Jan 17 14:04:26 l31lnx813 aziot-edged[28814]: 2022-01-17T12:04:26Z [INFO] - Starting management API...
Jan 17 14:04:26 l31lnx813 aziot-edged[28814]: 2022-01-17T12:04:26Z [INFO] - Starting watchdog with 60 second frequency...
Jan 17 14:04:26 l31lnx813 aziot-edged[28814]: 2022-01-17T12:04:26Z [INFO] - Listening on fd://aziot-edged.mgmt.socket with 1 thread for management API.
Jan 17 14:04:26 l31lnx813 aziot-edged[28814]: 2022-01-17T12:04:26Z [INFO] - Checking edge runtime status
Jan 17 14:04:26 l31lnx813 aziot-edged[28814]: 2022-01-17T12:04:26Z [INFO] - Creating and starting edge runtime module edgeAgent
Jan 17 14:04:26 l31lnx813 systemd[1]: Started Azure IoT Certificates Service.
Jan 17 14:04:26 l31lnx813 aziot-certd[28845]: 2022-01-17T12:04:26Z [INFO] - Starting service...
Jan 17 14:04:26 l31lnx813 aziot-certd[28845]: 2022-01-17T12:04:26Z [INFO] - Version - 1.2.4
Jan 17 14:04:26 l31lnx813 aziot-certd[28845]: 2022-01-17T12:04:26Z [INFO] - Starting server...
Jan 17 14:04:26 l31lnx813 aziot-identityd[28819]: 2022-01-17T12:04:26Z [INFO] - <-- PUT /identities/modules/$edgeAgent?api-version=2020-09-01&type=aziot {"content-type": "application/json", "host": "2f72756e2f617a696f742f6964656e74697479642e736f636b:0", "content-length": "40"}
Jan 17 14:04:26 l31lnx813 aziot-keyd[28832]: 2022-01-17T12:04:26Z [INFO] - <-- GET /key/device-id?api-version=2020-09-01 {"host": "keyd.sock"}
Jan 17 14:04:26 l31lnx813 aziot-keyd[28832]: 2022-01-17T12:04:26Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:04:26 l31lnx813 aziot-keyd[28832]: 2022-01-17T12:04:26Z [INFO] - <-- POST /sign?api-version=2020-09-01 {"content-type": "application/json", "host": "keyd.sock", "content-length": "414"}
Jan 17 14:04:26 l31lnx813 aziot-keyd[28832]: 2022-01-17T12:04:26Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:04:26 l31lnx813 aziot-certd[28845]: 2022-01-17T12:04:26Z [INFO] - <-- GET /certificates/aziot-edged-ca?api-version=2020-09-01 {"host": "2f72756e2f617a696f742f63657274642e736f636b:0"}
Jan 17 14:04:26 l31lnx813 aziot-certd[28845]: 2022-01-17T12:04:26Z [INFO] - !!! parameter "id" has an invalid value
Jan 17 14:04:26 l31lnx813 aziot-certd[28845]: 2022-01-17T12:04:26Z [INFO] - !!! caused by: not found
Jan 17 14:04:26 l31lnx813 aziot-certd[28845]: 2022-01-17T12:04:26Z [INFO] - --> 400 {"content-type": "application/json"}
Jan 17 14:04:26 l31lnx813 aziot-keyd[28832]: 2022-01-17T12:04:26Z [INFO] - <-- POST /keypair?api-version=2020-09-01 {"content-length": "61", "content-type": "application/json"}
Jan 17 14:04:26 l31lnx813 aziot-keyd[28832]: 2022-01-17T12:04:26Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:04:26 l31lnx813 aziot-keyd[28832]: 2022-01-17T12:04:26Z [INFO] - <-- POST /parameters/algorithm?api-version=2020-09-01 {"content-length": "252", "content-type": "application/json"}
Jan 17 14:04:26 l31lnx813 aziot-keyd[28832]: 2022-01-17T12:04:26Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:04:26 l31lnx813 aziot-keyd[28832]: 2022-01-17T12:04:26Z [INFO] - <-- POST /parameters/rsa-modulus?api-version=2020-09-01 {"content-length": "252", "content-type": "application/json"}
Jan 17 14:04:26 l31lnx813 aziot-keyd[28832]: 2022-01-17T12:04:26Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:04:26 l31lnx813 aziot-keyd[28832]: 2022-01-17T12:04:26Z [INFO] - <-- POST /parameters/rsa-exponent?api-version=2020-09-01 {"content-length": "252", "content-type": "application/json"}
Jan 17 14:04:26 l31lnx813 aziot-keyd[28832]: 2022-01-17T12:04:26Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:04:26 l31lnx813 aziot-keyd[28832]: 2022-01-17T12:04:26Z [INFO] - <-- POST /parameters/algorithm?api-version=2020-09-01 {"content-length": "252", "content-type": "application/json"}
Jan 17 14:04:26 l31lnx813 aziot-keyd[28832]: 2022-01-17T12:04:26Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:04:26 l31lnx813 aziot-keyd[28832]: 2022-01-17T12:04:26Z [INFO] - <-- POST /parameters/rsa-modulus?api-version=2020-09-01 {"content-length": "252", "content-type": "application/json"}
Jan 17 14:04:26 l31lnx813 aziot-keyd[28832]: 2022-01-17T12:04:26Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:04:26 l31lnx813 aziot-keyd[28832]: 2022-01-17T12:04:26Z [INFO] - <-- POST /parameters/rsa-exponent?api-version=2020-09-01 {"content-length": "252", "content-type": "application/json"}
Jan 17 14:04:26 l31lnx813 aziot-keyd[28832]: 2022-01-17T12:04:26Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:04:26 l31lnx813 aziot-keyd[28832]: 2022-01-17T12:04:26Z [INFO] - <-- POST /encrypt?api-version=2020-09-01 {"content-length": "359", "content-type": "application/json"}
Jan 17 14:04:26 l31lnx813 aziot-keyd[28832]: 2022-01-17T12:04:26Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:04:26 l31lnx813 aziot-certd[28845]: 2022-01-17T12:04:26Z [INFO] - <-- POST /certificates?api-version=2020-09-01 {"content-type": "application/json", "host": "2f72756e2f617a696f742f63657274642e736f636b:0", "content-length": "1042"}
Jan 17 14:04:26 l31lnx813 aziot-keyd[28832]: 2022-01-17T12:04:26Z [INFO] - <-- GET /keypair/aziot-edged-ca?api-version=2020-09-01 {}
Jan 17 14:04:26 l31lnx813 aziot-keyd[28832]: 2022-01-17T12:04:26Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:04:26 l31lnx813 aziot-keyd[28832]: 2022-01-17T12:04:26Z [INFO] - <-- POST /parameters/algorithm?api-version=2020-09-01 {"content-length": "252", "content-type": "application/json"}
Jan 17 14:04:26 l31lnx813 aziot-keyd[28832]: 2022-01-17T12:04:26Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:04:26 l31lnx813 aziot-keyd[28832]: 2022-01-17T12:04:26Z [INFO] - <-- POST /parameters/rsa-modulus?api-version=2020-09-01 {"content-length": "252", "content-type": "application/json"}
Jan 17 14:04:26 l31lnx813 aziot-keyd[28832]: 2022-01-17T12:04:26Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:04:26 l31lnx813 aziot-keyd[28832]: 2022-01-17T12:04:26Z [INFO] - <-- POST /parameters/rsa-exponent?api-version=2020-09-01 {"content-length": "252", "content-type": "application/json"}
Jan 17 14:04:26 l31lnx813 aziot-keyd[28832]: 2022-01-17T12:04:26Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:04:26 l31lnx813 aziot-keyd[28832]: 2022-01-17T12:04:26Z [INFO] - <-- POST /encrypt?api-version=2020-09-01 {"content-length": "359", "content-type": "application/json"}
Jan 17 14:04:26 l31lnx813 aziot-keyd[28832]: 2022-01-17T12:04:26Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:04:26 l31lnx813 aziot-certd[28845]: 2022-01-17T12:04:26Z [INFO] - --> 201 {"content-type": "application/json"}
Jan 17 14:04:26 l31lnx813 aziot-edged[28814]: 2022-01-17T12:04:26Z [INFO] - Listening on fd://aziot-edged.workload.socket with 1 thread for workload API.
Jan 17 14:04:26 l31lnx813 aziot-keyd[28832]: 2022-01-17T12:04:26Z [INFO] - <-- GET /key/aziot_identityd_master_id?api-version=2020-09-01 {"host": "keyd.sock"}
Jan 17 14:04:26 l31lnx813 aziot-keyd[28832]: 2022-01-17T12:04:26Z [ERR!] - invalid parameter "id": not found
Jan 17 14:04:26 l31lnx813 aziot-keyd[28832]: 2022-01-17T12:04:26Z [INFO] - !!! a parameter has an invalid value
Jan 17 14:04:26 l31lnx813 aziot-keyd[28832]: 2022-01-17T12:04:26Z [INFO] - --> 400 {"content-type": "application/json"}
Jan 17 14:04:26 l31lnx813 aziot-keyd[28832]: 2022-01-17T12:04:26Z [INFO] - <-- POST /key?api-version=2020-09-01 {"content-type": "application/json", "host": "keyd.sock", "content-length": "75"}
Jan 17 14:04:26 l31lnx813 aziot-keyd[28832]: 2022-01-17T12:04:26Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:04:26 l31lnx813 aziot-keyd[28832]: 2022-01-17T12:04:26Z [INFO] - <-- POST /derivedkey?api-version=2020-09-01 {"content-type": "application/json", "host": "keyd.sock", "content-length": "340"}
Jan 17 14:04:26 l31lnx813 aziot-keyd[28832]: 2022-01-17T12:04:26Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:04:26 l31lnx813 aziot-keyd[28832]: 2022-01-17T12:04:26Z [INFO] - <-- POST /derivedkey/export?api-version=2020-09-01 {"content-type": "application/json", "host": "keyd.sock", "content-length": "736"}
Jan 17 14:04:26 l31lnx813 aziot-keyd[28832]: 2022-01-17T12:04:26Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:04:26 l31lnx813 aziot-keyd[28832]: 2022-01-17T12:04:26Z [INFO] - <-- POST /derivedkey?api-version=2020-09-01 {"content-type": "application/json", "host": "keyd.sock", "content-length": "340"}
Jan 17 14:04:26 l31lnx813 aziot-keyd[28832]: 2022-01-17T12:04:26Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:04:26 l31lnx813 aziot-keyd[28832]: 2022-01-17T12:04:26Z [INFO] - <-- POST /derivedkey/export?api-version=2020-09-01 {"content-type": "application/json", "host": "keyd.sock", "content-length": "748"}
Jan 17 14:04:26 l31lnx813 aziot-keyd[28832]: 2022-01-17T12:04:26Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:04:26 l31lnx813 aziot-keyd[28832]: 2022-01-17T12:04:26Z [INFO] - <-- GET /key/device-id?api-version=2020-09-01 {"host": "keyd.sock"}
Jan 17 14:04:26 l31lnx813 aziot-keyd[28832]: 2022-01-17T12:04:26Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:04:26 l31lnx813 aziot-keyd[28832]: 2022-01-17T12:04:26Z [INFO] - <-- POST /sign?api-version=2020-09-01 {"content-type": "application/json", "host": "keyd.sock", "content-length": "414"}
Jan 17 14:04:26 l31lnx813 aziot-keyd[28832]: 2022-01-17T12:04:26Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:04:27 l31lnx813 aziot-identityd[28819]: 2022-01-17T12:04:27Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:04:27 l31lnx813 aziot-edged[28814]: 2022-01-17T12:04:27Z [INFO] - Pulling image via tag mcr.microsoft.com/azureiotedge-agent:1.2...
Jan 17 14:04:28 l31lnx813 aziot-edged[28814]: 2022-01-17T12:04:28Z [INFO] - Successfully pulled image mcr.microsoft.com/azureiotedge-agent:1.2
Jan 17 14:04:28 l31lnx813 aziot-edged[28814]: 2022-01-17T12:04:28Z [INFO] - Creating module edgeAgent...
Jan 17 14:04:28 l31lnx813 aziot-edged[28814]: 2022-01-17T12:04:28Z [INFO] - Creating image via tag mcr.microsoft.com/azureiotedge-agent:1.2...
Jan 17 14:04:28 l31lnx813 aziot-edged[28814]: 2022-01-17T12:04:28Z [INFO] - Successfully created module edgeAgent
Jan 17 14:04:28 l31lnx813 aziot-edged[28814]: 2022-01-17T12:04:28Z [INFO] - Starting module edgeAgent...
Jan 17 14:04:28 l31lnx813 aziot-edged[28814]: 2022-01-17T12:04:28Z [INFO] - Starting new listener for module edgeAgent
Jan 17 14:04:28 l31lnx813 aziot-certd[28845]: 2022-01-17T12:04:28Z [INFO] - <-- GET /certificates/aziot-edged-ca?api-version=2020-09-01 {"host": "2f72756e2f617a696f742f63657274642e736f636b:0"}
Jan 17 14:04:28 l31lnx813 aziot-certd[28845]: 2022-01-17T12:04:28Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:04:28 l31lnx813 aziot-edged[28814]: 2022-01-17T12:04:28Z [INFO] - Listening on unix:///var/lib/aziot/edged/mnt/edgeAgent.sock with 1 thread for workload API.
Jan 17 14:04:29 l31lnx813 aziot-edged[28814]: 2022-01-17T12:04:29Z [INFO] - Successfully started module edgeAgent
Jan 17 14:04:29 l31lnx813 aziot-certd[28845]: 2022-01-17T12:04:29Z [INFO] - <-- GET /certificates/aziot-edged-trust-bundle?api-version=2020-09-01 {"host": "2f72756e2f617a696f742f63657274642e736f636b:0"}
Jan 17 14:04:29 l31lnx813 aziot-certd[28845]: 2022-01-17T12:04:29Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:04:29 l31lnx813 aziot-edged[28814]: 2022-01-17T12:04:29Z [INFO] - [work] - - - [2022-01-17 12:04:29.818693940 UTC] "GET /trust-bundle?api-version=2019-01-30 HTTP/1.1" 200 OK 1118 "-" "-" auth_id(-)
Jan 17 14:05:19 l31lnx813 aziot-edged[28814]: 2022-01-17T12:05:19Z [INFO] - [mgmt] - - - [2022-01-17 12:05:19.078328290 UTC] "GET /modules?api-version=2020-07-07 HTTP/1.1" 200 OK 690 "-" "iotedge/0.1.0" auth_id(-)
Jan 17 14:05:26 l31lnx813 aziot-edged[28814]: 2022-01-17T12:05:26Z [INFO] - Checking edge runtime status
Jan 17 14:05:26 l31lnx813 aziot-edged[28814]: 2022-01-17T12:05:26Z [INFO] - Edge runtime status is failed, starting module now...
Jan 17 14:05:26 l31lnx813 aziot-edged[28814]: 2022-01-17T12:05:26Z [INFO] - Starting module edgeAgent...
Jan 17 14:05:26 l31lnx813 aziot-edged[28814]: 2022-01-17T12:05:26Z [INFO] - Starting new listener for module edgeAgent
Jan 17 14:05:26 l31lnx813 aziot-edged[28814]: 2022-01-17T12:05:26Z [INFO] - Listener  edgeAgent already started, removing old listener
Jan 17 14:05:26 l31lnx813 aziot-certd[28845]: 2022-01-17T12:05:26Z [INFO] - <-- GET /certificates/aziot-edged-ca?api-version=2020-09-01 {"host": "2f72756e2f617a696f742f63657274642e736f636b:0"}
Jan 17 14:05:26 l31lnx813 aziot-certd[28845]: 2022-01-17T12:05:26Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:05:26 l31lnx813 aziot-edged[28814]: 2022-01-17T12:05:26Z [INFO] - Listening on unix:///var/lib/aziot/edged/mnt/edgeAgent.sock with 1 thread for workload API.
Jan 17 14:05:26 l31lnx813 aziot-edged[28814]: 2022-01-17T12:05:26Z [INFO] - Successfully started module edgeAgent
Jan 17 14:05:27 l31lnx813 aziot-certd[28845]: 2022-01-17T12:05:27Z [INFO] - <-- GET /certificates/aziot-edged-trust-bundle?api-version=2020-09-01 {"host": "2f72756e2f617a696f742f63657274642e736f636b:0"}
Jan 17 14:05:27 l31lnx813 aziot-certd[28845]: 2022-01-17T12:05:27Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:05:27 l31lnx813 aziot-edged[28814]: 2022-01-17T12:05:27Z [INFO] - [work] - - - [2022-01-17 12:05:27.127803836 UTC] "GET /trust-bundle?api-version=2019-01-30 HTTP/1.1" 200 OK 1118 "-" "-" auth_id(-)
Jan 17 14:06:26 l31lnx813 aziot-edged[28814]: 2022-01-17T12:06:26Z [INFO] - Checking edge runtime status
Jan 17 14:06:26 l31lnx813 aziot-edged[28814]: 2022-01-17T12:06:26Z [INFO] - Edge runtime status is failed, starting module now...
Jan 17 14:06:26 l31lnx813 aziot-edged[28814]: 2022-01-17T12:06:26Z [INFO] - Starting module edgeAgent...
Jan 17 14:06:26 l31lnx813 aziot-edged[28814]: 2022-01-17T12:06:26Z [INFO] - Starting new listener for module edgeAgent
Jan 17 14:06:26 l31lnx813 aziot-edged[28814]: 2022-01-17T12:06:26Z [INFO] - Listener  edgeAgent already started, removing old listener
Jan 17 14:06:26 l31lnx813 aziot-certd[28845]: 2022-01-17T12:06:26Z [INFO] - <-- GET /certificates/aziot-edged-ca?api-version=2020-09-01 {"host": "2f72756e2f617a696f742f63657274642e736f636b:0"}
Jan 17 14:06:26 l31lnx813 aziot-certd[28845]: 2022-01-17T12:06:26Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:06:26 l31lnx813 aziot-edged[28814]: 2022-01-17T12:06:26Z [INFO] - Listening on unix:///var/lib/aziot/edged/mnt/edgeAgent.sock with 1 thread for workload API.
Jan 17 14:06:26 l31lnx813 aziot-edged[28814]: 2022-01-17T12:06:26Z [INFO] - Successfully started module edgeAgent
Jan 17 14:06:27 l31lnx813 aziot-certd[28845]: 2022-01-17T12:06:27Z [INFO] - <-- GET /certificates/aziot-edged-trust-bundle?api-version=2020-09-01 {"host": "2f72756e2f617a696f742f63657274642e736f636b:0"}
Jan 17 14:06:27 l31lnx813 aziot-certd[28845]: 2022-01-17T12:06:27Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:06:27 l31lnx813 aziot-edged[28814]: 2022-01-17T12:06:27Z [INFO] - [work] - - - [2022-01-17 12:06:27.189442751 UTC] "GET /trust-bundle?api-version=2019-01-30 HTTP/1.1" 200 OK 1118 "-" "-" auth_id(-)
Jan 17 14:07:26 l31lnx813 aziot-edged[28814]: 2022-01-17T12:07:26Z [INFO] - Checking edge runtime status
Jan 17 14:07:26 l31lnx813 aziot-edged[28814]: 2022-01-17T12:07:26Z [INFO] - Edge runtime status is failed, starting module now...
Jan 17 14:07:26 l31lnx813 aziot-edged[28814]: 2022-01-17T12:07:26Z [INFO] - Starting module edgeAgent...
Jan 17 14:07:26 l31lnx813 aziot-edged[28814]: 2022-01-17T12:07:26Z [INFO] - Starting new listener for module edgeAgent
Jan 17 14:07:26 l31lnx813 aziot-edged[28814]: 2022-01-17T12:07:26Z [INFO] - Listener  edgeAgent already started, removing old listener
Jan 17 14:07:26 l31lnx813 aziot-certd[28845]: 2022-01-17T12:07:26Z [INFO] - <-- GET /certificates/aziot-edged-ca?api-version=2020-09-01 {"host": "2f72756e2f617a696f742f63657274642e736f636b:0"}
Jan 17 14:07:26 l31lnx813 aziot-certd[28845]: 2022-01-17T12:07:26Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:07:26 l31lnx813 aziot-edged[28814]: 2022-01-17T12:07:26Z [INFO] - Listening on unix:///var/lib/aziot/edged/mnt/edgeAgent.sock with 1 thread for workload API.
Jan 17 14:07:26 l31lnx813 aziot-edged[28814]: 2022-01-17T12:07:26Z [INFO] - Successfully started module edgeAgent
Jan 17 14:07:27 l31lnx813 aziot-certd[28845]: 2022-01-17T12:07:27Z [INFO] - <-- GET /certificates/aziot-edged-trust-bundle?api-version=2020-09-01 {"host": "2f72756e2f617a696f742f63657274642e736f636b:0"}
Jan 17 14:07:27 l31lnx813 aziot-certd[28845]: 2022-01-17T12:07:27Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:07:27 l31lnx813 aziot-edged[28814]: 2022-01-17T12:07:27Z [INFO] - [work] - - - [2022-01-17 12:07:27.169261269 UTC] "GET /trust-bundle?api-version=2019-01-30 HTTP/1.1" 200 OK 1118 "-" "-" auth_id(-)
Jan 17 14:08:26 l31lnx813 aziot-edged[28814]: 2022-01-17T12:08:26Z [INFO] - Checking edge runtime status
Jan 17 14:08:26 l31lnx813 aziot-edged[28814]: 2022-01-17T12:08:26Z [INFO] - Edge runtime status is failed, starting module now...
Jan 17 14:08:26 l31lnx813 aziot-edged[28814]: 2022-01-17T12:08:26Z [INFO] - Starting module edgeAgent...
Jan 17 14:08:26 l31lnx813 aziot-edged[28814]: 2022-01-17T12:08:26Z [INFO] - Starting new listener for module edgeAgent
Jan 17 14:08:26 l31lnx813 aziot-edged[28814]: 2022-01-17T12:08:26Z [INFO] - Listener  edgeAgent already started, removing old listener
Jan 17 14:08:26 l31lnx813 aziot-certd[28845]: 2022-01-17T12:08:26Z [INFO] - <-- GET /certificates/aziot-edged-ca?api-version=2020-09-01 {"host": "2f72756e2f617a696f742f63657274642e736f636b:0"}
Jan 17 14:08:26 l31lnx813 aziot-certd[28845]: 2022-01-17T12:08:26Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:08:26 l31lnx813 aziot-edged[28814]: 2022-01-17T12:08:26Z [INFO] - Listening on unix:///var/lib/aziot/edged/mnt/edgeAgent.sock with 1 thread for workload API.
Jan 17 14:08:26 l31lnx813 aziot-edged[28814]: 2022-01-17T12:08:26Z [INFO] - Successfully started module edgeAgent
Jan 17 14:08:27 l31lnx813 aziot-certd[28845]: 2022-01-17T12:08:27Z [INFO] - <-- GET /certificates/aziot-edged-trust-bundle?api-version=2020-09-01 {"host": "2f72756e2f617a696f742f63657274642e736f636b:0"}
Jan 17 14:08:27 l31lnx813 aziot-certd[28845]: 2022-01-17T12:08:27Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:08:27 l31lnx813 aziot-edged[28814]: 2022-01-17T12:08:27Z [INFO] - [work] - - - [2022-01-17 12:08:27.181405325 UTC] "GET /trust-bundle?api-version=2019-01-30 HTTP/1.1" 200 OK 1118 "-" "-" auth_id(-)
Jan 17 14:09:26 l31lnx813 aziot-edged[28814]: 2022-01-17T12:09:26Z [INFO] - Checking edge runtime status
Jan 17 14:09:26 l31lnx813 aziot-edged[28814]: 2022-01-17T12:09:26Z [INFO] - Edge runtime status is failed, starting module now...
Jan 17 14:09:26 l31lnx813 aziot-edged[28814]: 2022-01-17T12:09:26Z [INFO] - Starting module edgeAgent...
Jan 17 14:09:26 l31lnx813 aziot-edged[28814]: 2022-01-17T12:09:26Z [INFO] - Starting new listener for module edgeAgent
Jan 17 14:09:26 l31lnx813 aziot-edged[28814]: 2022-01-17T12:09:26Z [INFO] - Listener  edgeAgent already started, removing old listener
Jan 17 14:09:26 l31lnx813 aziot-certd[28845]: 2022-01-17T12:09:26Z [INFO] - <-- GET /certificates/aziot-edged-ca?api-version=2020-09-01 {"host": "2f72756e2f617a696f742f63657274642e736f636b:0"}
Jan 17 14:09:26 l31lnx813 aziot-certd[28845]: 2022-01-17T12:09:26Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:09:26 l31lnx813 aziot-edged[28814]: 2022-01-17T12:09:26Z [INFO] - Listening on unix:///var/lib/aziot/edged/mnt/edgeAgent.sock with 1 thread for workload API.
Jan 17 14:09:26 l31lnx813 aziot-edged[28814]: 2022-01-17T12:09:26Z [INFO] - Successfully started module edgeAgent
Jan 17 14:09:27 l31lnx813 aziot-certd[28845]: 2022-01-17T12:09:27Z [INFO] - <-- GET /certificates/aziot-edged-trust-bundle?api-version=2020-09-01 {"host": "2f72756e2f617a696f742f63657274642e736f636b:0"}
Jan 17 14:09:27 l31lnx813 aziot-certd[28845]: 2022-01-17T12:09:27Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:09:27 l31lnx813 aziot-edged[28814]: 2022-01-17T12:09:27Z [INFO] - [work] - - - [2022-01-17 12:09:27.209230019 UTC] "GET /trust-bundle?api-version=2019-01-30 HTTP/1.1" 200 OK 1118 "-" "-" auth_id(-)
Jan 17 14:09:57 l31lnx813 systemd[1]: Stopping Azure IoT Edge daemon...
Jan 17 14:09:57 l31lnx813 aziot-edged[28814]: 2022-01-17T12:09:57Z [INFO] - Received SIGTERM, starting shutdown
Jan 17 14:09:57 l31lnx813 aziot-edged[28814]: 2022-01-17T12:09:57Z [INFO] - Stopping edge runtime module edgeAgent
Jan 17 14:09:57 l31lnx813 aziot-edged[28814]: 2022-01-17T12:09:57Z [INFO] - Stopping module edgeAgent...
Jan 17 14:09:57 l31lnx813 aziot-edged[28814]: 2022-01-17T12:09:57Z [WARN] - Could not stop module edgeAgent
Jan 17 14:09:57 l31lnx813 aziot-edged[28814]: 2022-01-17T12:09:57Z [WARN] -         caused by: Target of operation already in this state
Jan 17 14:09:57 l31lnx813 aziot-edged[28814]: 2022-01-17T12:09:57Z [ERR!] - Edge runtime will stop because the shutdown signal raised an error.
Jan 17 14:09:57 l31lnx813 aziot-edged[28814]: 2022-01-17T12:09:57Z [ERR!] - A module runtime error occurred.
Jan 17 14:09:57 l31lnx813 aziot-edged[28814]: 2022-01-17T12:09:57Z [ERR!] -         caused by: Could not stop module edgeAgent
Jan 17 14:09:57 l31lnx813 aziot-edged[28814]: 2022-01-17T12:09:57Z [ERR!] -         caused by: Target of operation already in this state
Jan 17 14:09:57 l31lnx813 systemd[1]: aziot-edged.service: Main process exited, code=exited, status=1/FAILURE
Jan 17 14:09:57 l31lnx813 systemd[1]: aziot-edged.service: Failed with result 'exit-code'.
Jan 17 14:09:57 l31lnx813 systemd[1]: Stopped Azure IoT Edge daemon.
Jan 17 14:09:57 l31lnx813 systemd[1]: Stopping Azure IoT Identity Service...
Jan 17 14:09:57 l31lnx813 systemd[1]: Stopped Azure IoT Identity Service.
Jan 17 14:09:57 l31lnx813 systemd[1]: Stopping Azure IoT Keys Service...
Jan 17 14:09:57 l31lnx813 systemd[1]: Stopped Azure IoT Keys Service.
Jan 17 14:09:57 l31lnx813 systemd[1]: Stopping Azure IoT Certificates Service...
Jan 17 14:09:57 l31lnx813 systemd[1]: Stopped Azure IoT Certificates Service.
Jan 17 14:09:57 l31lnx813 systemd[1]: Started Azure IoT Edge daemon.
Jan 17 14:09:57 l31lnx813 aziot-edged[30103]: 2022-01-17T12:09:57Z [INFO] - Starting Azure IoT Edge Module Runtime
Jan 17 14:09:57 l31lnx813 aziot-edged[30103]: 2022-01-17T12:09:57Z [INFO] - Version - 1.2.5
Jan 17 14:09:57 l31lnx813 aziot-edged[30103]: 2022-01-17T12:09:57Z [INFO] - Initializing the module runtime...
Jan 17 14:09:57 l31lnx813 aziot-edged[30103]: 2022-01-17T12:09:57Z [INFO] - Initializing module runtime...
Jan 17 14:09:57 l31lnx813 aziot-edged[30103]: 2022-01-17T12:09:57Z [INFO] - Using runtime network id azure-iot-edge
Jan 17 14:09:57 l31lnx813 aziot-edged[30103]: 2022-01-17T12:09:57Z [INFO] - Successfully initialized module runtime
Jan 17 14:09:57 l31lnx813 aziot-edged[30103]: 2022-01-17T12:09:57Z [INFO] - Finished initializing the module runtime.
Jan 17 14:09:57 l31lnx813 aziot-edged[30103]: 2022-01-17T12:09:57Z [INFO] - Obtaining edge device provisioning data...
Jan 17 14:09:57 l31lnx813 systemd[1]: Started Azure IoT Identity Service.
Jan 17 14:09:57 l31lnx813 aziot-identityd[30108]: 2022-01-17T12:09:57Z [INFO] - Starting service...
Jan 17 14:09:57 l31lnx813 aziot-identityd[30108]: 2022-01-17T12:09:57Z [INFO] - Version - 1.2.4
Jan 17 14:09:57 l31lnx813 aziot-identityd[30108]: 2022-01-17T12:09:57Z [INFO] - Provisioning starting. Reason: Startup
Jan 17 14:09:57 l31lnx813 aziot-identityd[30108]: 2022-01-17T12:09:57Z [INFO] - Updated device info for cdp-apps-dev-nordland-iot-edge.
Jan 17 14:09:57 l31lnx813 aziot-identityd[30108]: 2022-01-17T12:09:57Z [INFO] - Provisioning complete.
Jan 17 14:09:57 l31lnx813 aziot-identityd[30108]: 2022-01-17T12:09:57Z [INFO] - Identity reconciliation started. Reason: Startup
Jan 17 14:09:57 l31lnx813 systemd[1]: Started Azure IoT Keys Service.
Jan 17 14:09:57 l31lnx813 aziot-keyd[30123]: 2022-01-17T12:09:57Z [INFO] - Starting service...
Jan 17 14:09:57 l31lnx813 aziot-keyd[30123]: 2022-01-17T12:09:57Z [INFO] - Version - 1.2.4
Jan 17 14:09:57 l31lnx813 aziot-keyd[30123]: 2022-01-17T12:09:57Z [INFO] - Loaded libaziot-keys with version 0x02000000
Jan 17 14:09:57 l31lnx813 aziot-keyd[30123]: 2022-01-17T12:09:57Z [INFO] - Starting server...
Jan 17 14:09:57 l31lnx813 aziot-keyd[30123]: 2022-01-17T12:09:57Z [INFO] - <-- GET /key/device-id?api-version=2020-09-01 {"host": "keyd.sock"}
Jan 17 14:09:57 l31lnx813 aziot-keyd[30123]: 2022-01-17T12:09:57Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:09:57 l31lnx813 aziot-keyd[30123]: 2022-01-17T12:09:57Z [INFO] - <-- POST /sign?api-version=2020-09-01 {"content-type": "application/json", "host": "keyd.sock", "content-length": "414"}
Jan 17 14:09:57 l31lnx813 aziot-keyd[30123]: 2022-01-17T12:09:57Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:09:58 l31lnx813 aziot-identityd[30108]: 2022-01-17T12:09:58Z [INFO] - Identity reconciliation complete.
Jan 17 14:09:58 l31lnx813 aziot-identityd[30108]: 2022-01-17T12:09:58Z [INFO] - Starting server...
Jan 17 14:09:58 l31lnx813 aziot-identityd[30108]: 2022-01-17T12:09:58Z [INFO] - <-- POST /identities/device?api-version=2020-09-01 {"content-type": "application/json", "host": "2f72756e2f617a696f742f6964656e74697479642e736f636b:0", "content-length": "16"}
Jan 17 14:09:58 l31lnx813 aziot-keyd[30123]: 2022-01-17T12:09:58Z [INFO] - <-- GET /key/device-id?api-version=2020-09-01 {"host": "keyd.sock"}
Jan 17 14:09:58 l31lnx813 aziot-keyd[30123]: 2022-01-17T12:09:58Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:09:58 l31lnx813 aziot-identityd[30108]: 2022-01-17T12:09:58Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:09:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:09:58Z [INFO] - Finished provisioning edge device.
Jan 17 14:09:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:09:58Z [INFO] - Stopping all modules...
Jan 17 14:09:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:09:58Z [INFO] - Stopping module edgeAgent...
Jan 17 14:09:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:09:58Z [WARN] - Could not stop module edgeAgent
Jan 17 14:09:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:09:58Z [WARN] -         caused by: Target of operation already in this state
Jan 17 14:09:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:09:58Z [INFO] - Finished stopping modules.
Jan 17 14:09:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:09:58Z [INFO] - Detecting if device information has changed...
Jan 17 14:09:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:09:58Z [INFO] - Starting management API...
Jan 17 14:09:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:09:58Z [INFO] - Starting new listener for module edgeAgent
Jan 17 14:09:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:09:58Z [INFO] - Starting watchdog with 60 second frequency...
Jan 17 14:09:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:09:58Z [INFO] - Listening on fd://aziot-edged.mgmt.socket with 1 thread for management API.
Jan 17 14:09:58 l31lnx813 systemd[1]: Started Azure IoT Certificates Service.
Jan 17 14:09:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:09:58Z [INFO] - Checking edge runtime status
Jan 17 14:09:58 l31lnx813 aziot-certd[30134]: 2022-01-17T12:09:58Z [INFO] - Starting service...
Jan 17 14:09:58 l31lnx813 aziot-certd[30134]: 2022-01-17T12:09:58Z [INFO] - Version - 1.2.4
Jan 17 14:09:58 l31lnx813 aziot-certd[30134]: 2022-01-17T12:09:58Z [INFO] - Starting server...
Jan 17 14:09:58 l31lnx813 aziot-certd[30134]: 2022-01-17T12:09:58Z [INFO] - <-- GET /certificates/aziot-edged-ca?api-version=2020-09-01 {"host": "2f72756e2f617a696f742f63657274642e736f636b:0"}
Jan 17 14:09:58 l31lnx813 aziot-certd[30134]: 2022-01-17T12:09:58Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:09:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:09:58Z [INFO] - Listening on fd://aziot-edged.workload.socket with 1 thread for workload API.
Jan 17 14:09:58 l31lnx813 aziot-certd[30134]: 2022-01-17T12:09:58Z [INFO] - <-- GET /certificates/aziot-edged-ca?api-version=2020-09-01 {"host": "2f72756e2f617a696f742f63657274642e736f636b:0"}
Jan 17 14:09:58 l31lnx813 aziot-certd[30134]: 2022-01-17T12:09:58Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:09:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:09:58Z [INFO] - Listening on unix:///var/lib/aziot/edged/mnt/edgeAgent.sock with 1 thread for workload API.
Jan 17 14:09:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:09:58Z [INFO] - Edge runtime status is failed, starting module now...
Jan 17 14:09:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:09:58Z [INFO] - Starting module edgeAgent...
Jan 17 14:09:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:09:58Z [INFO] - Starting new listener for module edgeAgent
Jan 17 14:09:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:09:58Z [INFO] - Listener  edgeAgent already started, removing old listener
Jan 17 14:09:58 l31lnx813 aziot-certd[30134]: 2022-01-17T12:09:58Z [INFO] - <-- GET /certificates/aziot-edged-ca?api-version=2020-09-01 {"host": "2f72756e2f617a696f742f63657274642e736f636b:0"}
Jan 17 14:09:58 l31lnx813 aziot-certd[30134]: 2022-01-17T12:09:58Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:09:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:09:58Z [INFO] - Listening on unix:///var/lib/aziot/edged/mnt/edgeAgent.sock with 1 thread for workload API.
Jan 17 14:09:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:09:58Z [INFO] - Successfully started module edgeAgent
Jan 17 14:09:59 l31lnx813 aziot-certd[30134]: 2022-01-17T12:09:59Z [INFO] - <-- GET /certificates/aziot-edged-trust-bundle?api-version=2020-09-01 {"host": "2f72756e2f617a696f742f63657274642e736f636b:0"}
Jan 17 14:09:59 l31lnx813 aziot-certd[30134]: 2022-01-17T12:09:59Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:09:59 l31lnx813 aziot-edged[30103]: 2022-01-17T12:09:59Z [INFO] - [work] - - - [2022-01-17 12:09:59.209216990 UTC] "GET /trust-bundle?api-version=2019-01-30 HTTP/1.1" 200 OK 1118 "-" "-" auth_id(-)
Jan 17 14:10:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:10:58Z [INFO] - Checking edge runtime status
Jan 17 14:10:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:10:58Z [INFO] - Edge runtime status is failed, starting module now...
Jan 17 14:10:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:10:58Z [INFO] - Starting module edgeAgent...
Jan 17 14:10:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:10:58Z [INFO] - Starting new listener for module edgeAgent
Jan 17 14:10:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:10:58Z [INFO] - Listener  edgeAgent already started, removing old listener
Jan 17 14:10:58 l31lnx813 aziot-certd[30134]: 2022-01-17T12:10:58Z [INFO] - <-- GET /certificates/aziot-edged-ca?api-version=2020-09-01 {"host": "2f72756e2f617a696f742f63657274642e736f636b:0"}
Jan 17 14:10:58 l31lnx813 aziot-certd[30134]: 2022-01-17T12:10:58Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:10:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:10:58Z [INFO] - Listening on unix:///var/lib/aziot/edged/mnt/edgeAgent.sock with 1 thread for workload API.
Jan 17 14:10:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:10:58Z [INFO] - Successfully started module edgeAgent
Jan 17 14:10:59 l31lnx813 aziot-certd[30134]: 2022-01-17T12:10:59Z [INFO] - <-- GET /certificates/aziot-edged-trust-bundle?api-version=2020-09-01 {"host": "2f72756e2f617a696f742f63657274642e736f636b:0"}
Jan 17 14:10:59 l31lnx813 aziot-certd[30134]: 2022-01-17T12:10:59Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:10:59 l31lnx813 aziot-edged[30103]: 2022-01-17T12:10:59Z [INFO] - [work] - - - [2022-01-17 12:10:59.359046118 UTC] "GET /trust-bundle?api-version=2019-01-30 HTTP/1.1" 200 OK 1118 "-" "-" auth_id(-)
Jan 17 14:11:31 l31lnx813 aziot-edged[30103]: 2022-01-17T12:11:31Z [INFO] - [mgmt] - - - [2022-01-17 12:11:31.433335737 UTC] "GET /modules?api-version=2020-07-07 HTTP/1.1" 200 OK 690 "-" "iotedge/0.1.0" auth_id(-)
Jan 17 14:11:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:11:58Z [INFO] - Checking edge runtime status
Jan 17 14:11:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:11:58Z [INFO] - Edge runtime status is failed, starting module now...
Jan 17 14:11:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:11:58Z [INFO] - Starting module edgeAgent...
Jan 17 14:11:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:11:58Z [INFO] - Starting new listener for module edgeAgent
Jan 17 14:11:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:11:58Z [INFO] - Listener  edgeAgent already started, removing old listener
Jan 17 14:11:58 l31lnx813 aziot-certd[30134]: 2022-01-17T12:11:58Z [INFO] - <-- GET /certificates/aziot-edged-ca?api-version=2020-09-01 {"host": "2f72756e2f617a696f742f63657274642e736f636b:0"}
Jan 17 14:11:58 l31lnx813 aziot-certd[30134]: 2022-01-17T12:11:58Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:11:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:11:58Z [INFO] - Listening on unix:///var/lib/aziot/edged/mnt/edgeAgent.sock with 1 thread for workload API.
Jan 17 14:11:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:11:58Z [INFO] - Successfully started module edgeAgent
Jan 17 14:11:59 l31lnx813 aziot-certd[30134]: 2022-01-17T12:11:59Z [INFO] - <-- GET /certificates/aziot-edged-trust-bundle?api-version=2020-09-01 {"host": "2f72756e2f617a696f742f63657274642e736f636b:0"}
Jan 17 14:11:59 l31lnx813 aziot-certd[30134]: 2022-01-17T12:11:59Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:11:59 l31lnx813 aziot-edged[30103]: 2022-01-17T12:11:59Z [INFO] - [work] - - - [2022-01-17 12:11:59.210809904 UTC] "GET /trust-bundle?api-version=2019-01-30 HTTP/1.1" 200 OK 1118 "-" "-" auth_id(-)
Jan 17 14:12:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:12:58Z [INFO] - Checking edge runtime status
Jan 17 14:12:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:12:58Z [INFO] - Edge runtime status is failed, starting module now...
Jan 17 14:12:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:12:58Z [INFO] - Starting module edgeAgent...
Jan 17 14:12:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:12:58Z [INFO] - Starting new listener for module edgeAgent
Jan 17 14:12:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:12:58Z [INFO] - Listener  edgeAgent already started, removing old listener
Jan 17 14:12:58 l31lnx813 aziot-certd[30134]: 2022-01-17T12:12:58Z [INFO] - <-- GET /certificates/aziot-edged-ca?api-version=2020-09-01 {"host": "2f72756e2f617a696f742f63657274642e736f636b:0"}
Jan 17 14:12:58 l31lnx813 aziot-certd[30134]: 2022-01-17T12:12:58Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:12:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:12:58Z [INFO] - Listening on unix:///var/lib/aziot/edged/mnt/edgeAgent.sock with 1 thread for workload API.
Jan 17 14:12:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:12:58Z [INFO] - Successfully started module edgeAgent
Jan 17 14:12:59 l31lnx813 aziot-certd[30134]: 2022-01-17T12:12:59Z [INFO] - <-- GET /certificates/aziot-edged-trust-bundle?api-version=2020-09-01 {"host": "2f72756e2f617a696f742f63657274642e736f636b:0"}
Jan 17 14:12:59 l31lnx813 aziot-certd[30134]: 2022-01-17T12:12:59Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:12:59 l31lnx813 aziot-edged[30103]: 2022-01-17T12:12:59Z [INFO] - [work] - - - [2022-01-17 12:12:59.200196882 UTC] "GET /trust-bundle?api-version=2019-01-30 HTTP/1.1" 200 OK 1118 "-" "-" auth_id(-)
Jan 17 14:13:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:13:58Z [INFO] - Checking edge runtime status
Jan 17 14:13:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:13:58Z [INFO] - Edge runtime status is failed, starting module now...
Jan 17 14:13:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:13:58Z [INFO] - Starting module edgeAgent...
Jan 17 14:13:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:13:58Z [INFO] - Starting new listener for module edgeAgent
Jan 17 14:13:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:13:58Z [INFO] - Listener  edgeAgent already started, removing old listener
Jan 17 14:13:58 l31lnx813 aziot-certd[30134]: 2022-01-17T12:13:58Z [INFO] - <-- GET /certificates/aziot-edged-ca?api-version=2020-09-01 {"host": "2f72756e2f617a696f742f63657274642e736f636b:0"}
Jan 17 14:13:58 l31lnx813 aziot-certd[30134]: 2022-01-17T12:13:58Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:13:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:13:58Z [INFO] - Listening on unix:///var/lib/aziot/edged/mnt/edgeAgent.sock with 1 thread for workload API.
Jan 17 14:13:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:13:58Z [INFO] - Successfully started module edgeAgent
Jan 17 14:13:59 l31lnx813 aziot-certd[30134]: 2022-01-17T12:13:59Z [INFO] - <-- GET /certificates/aziot-edged-trust-bundle?api-version=2020-09-01 {"host": "2f72756e2f617a696f742f63657274642e736f636b:0"}
Jan 17 14:13:59 l31lnx813 aziot-certd[30134]: 2022-01-17T12:13:59Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:13:59 l31lnx813 aziot-edged[30103]: 2022-01-17T12:13:59Z [INFO] - [work] - - - [2022-01-17 12:13:59.275026158 UTC] "GET /trust-bundle?api-version=2019-01-30 HTTP/1.1" 200 OK 1118 "-" "-" auth_id(-)
Jan 17 14:14:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:14:58Z [INFO] - Checking edge runtime status
Jan 17 14:14:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:14:58Z [INFO] - Edge runtime status is failed, starting module now...
Jan 17 14:14:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:14:58Z [INFO] - Starting module edgeAgent...
Jan 17 14:14:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:14:58Z [INFO] - Starting new listener for module edgeAgent
Jan 17 14:14:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:14:58Z [INFO] - Listener  edgeAgent already started, removing old listener
Jan 17 14:14:58 l31lnx813 aziot-certd[30134]: 2022-01-17T12:14:58Z [INFO] - <-- GET /certificates/aziot-edged-ca?api-version=2020-09-01 {"host": "2f72756e2f617a696f742f63657274642e736f636b:0"}
Jan 17 14:14:58 l31lnx813 aziot-certd[30134]: 2022-01-17T12:14:58Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:14:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:14:58Z [INFO] - Listening on unix:///var/lib/aziot/edged/mnt/edgeAgent.sock with 1 thread for workload API.
Jan 17 14:14:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:14:58Z [INFO] - Successfully started module edgeAgent
Jan 17 14:14:59 l31lnx813 aziot-certd[30134]: 2022-01-17T12:14:59Z [INFO] - <-- GET /certificates/aziot-edged-trust-bundle?api-version=2020-09-01 {"host": "2f72756e2f617a696f742f63657274642e736f636b:0"}
Jan 17 14:14:59 l31lnx813 aziot-certd[30134]: 2022-01-17T12:14:59Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:14:59 l31lnx813 aziot-edged[30103]: 2022-01-17T12:14:59Z [INFO] - [work] - - - [2022-01-17 12:14:59.288869943 UTC] "GET /trust-bundle?api-version=2019-01-30 HTTP/1.1" 200 OK 1118 "-" "-" auth_id(-)
Jan 17 14:15:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:15:58Z [INFO] - Checking edge runtime status
Jan 17 14:15:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:15:58Z [INFO] - Edge runtime status is failed, starting module now...
Jan 17 14:15:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:15:58Z [INFO] - Starting module edgeAgent...
Jan 17 14:15:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:15:58Z [INFO] - Starting new listener for module edgeAgent
Jan 17 14:15:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:15:58Z [INFO] - Listener  edgeAgent already started, removing old listener
Jan 17 14:15:58 l31lnx813 aziot-certd[30134]: 2022-01-17T12:15:58Z [INFO] - <-- GET /certificates/aziot-edged-ca?api-version=2020-09-01 {"host": "2f72756e2f617a696f742f63657274642e736f636b:0"}
Jan 17 14:15:58 l31lnx813 aziot-certd[30134]: 2022-01-17T12:15:58Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:15:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:15:58Z [INFO] - Listening on unix:///var/lib/aziot/edged/mnt/edgeAgent.sock with 1 thread for workload API.
Jan 17 14:15:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:15:58Z [INFO] - Successfully started module edgeAgent
Jan 17 14:15:59 l31lnx813 aziot-certd[30134]: 2022-01-17T12:15:59Z [INFO] - <-- GET /certificates/aziot-edged-trust-bundle?api-version=2020-09-01 {"host": "2f72756e2f617a696f742f63657274642e736f636b:0"}
Jan 17 14:15:59 l31lnx813 aziot-certd[30134]: 2022-01-17T12:15:59Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:15:59 l31lnx813 aziot-edged[30103]: 2022-01-17T12:15:59Z [INFO] - [work] - - - [2022-01-17 12:15:59.214990821 UTC] "GET /trust-bundle?api-version=2019-01-30 HTTP/1.1" 200 OK 1118 "-" "-" auth_id(-)
Jan 17 14:16:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:16:58Z [INFO] - Checking edge runtime status
Jan 17 14:16:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:16:58Z [INFO] - Edge runtime status is failed, starting module now...
Jan 17 14:16:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:16:58Z [INFO] - Starting module edgeAgent...
Jan 17 14:16:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:16:58Z [INFO] - Starting new listener for module edgeAgent
Jan 17 14:16:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:16:58Z [INFO] - Listener  edgeAgent already started, removing old listener
Jan 17 14:16:58 l31lnx813 aziot-certd[30134]: 2022-01-17T12:16:58Z [INFO] - <-- GET /certificates/aziot-edged-ca?api-version=2020-09-01 {"host": "2f72756e2f617a696f742f63657274642e736f636b:0"}
Jan 17 14:16:58 l31lnx813 aziot-certd[30134]: 2022-01-17T12:16:58Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:16:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:16:58Z [INFO] - Listening on unix:///var/lib/aziot/edged/mnt/edgeAgent.sock with 1 thread for workload API.
Jan 17 14:16:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:16:58Z [INFO] - Successfully started module edgeAgent
Jan 17 14:16:59 l31lnx813 aziot-certd[30134]: 2022-01-17T12:16:59Z [INFO] - <-- GET /certificates/aziot-edged-trust-bundle?api-version=2020-09-01 {"host": "2f72756e2f617a696f742f63657274642e736f636b:0"}
Jan 17 14:16:59 l31lnx813 aziot-certd[30134]: 2022-01-17T12:16:59Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:16:59 l31lnx813 aziot-edged[30103]: 2022-01-17T12:16:59Z [INFO] - [work] - - - [2022-01-17 12:16:59.257556278 UTC] "GET /trust-bundle?api-version=2019-01-30 HTTP/1.1" 200 OK 1118 "-" "-" auth_id(-)
Jan 17 14:17:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:17:58Z [INFO] - Checking edge runtime status
Jan 17 14:17:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:17:58Z [INFO] - Edge runtime status is failed, starting module now...
Jan 17 14:17:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:17:58Z [INFO] - Starting module edgeAgent...
Jan 17 14:17:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:17:58Z [INFO] - Starting new listener for module edgeAgent
Jan 17 14:17:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:17:58Z [INFO] - Listener  edgeAgent already started, removing old listener
Jan 17 14:17:58 l31lnx813 aziot-certd[30134]: 2022-01-17T12:17:58Z [INFO] - <-- GET /certificates/aziot-edged-ca?api-version=2020-09-01 {"host": "2f72756e2f617a696f742f63657274642e736f636b:0"}
Jan 17 14:17:58 l31lnx813 aziot-certd[30134]: 2022-01-17T12:17:58Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:17:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:17:58Z [INFO] - Listening on unix:///var/lib/aziot/edged/mnt/edgeAgent.sock with 1 thread for workload API.
Jan 17 14:17:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:17:58Z [INFO] - Successfully started module edgeAgent
Jan 17 14:17:59 l31lnx813 aziot-certd[30134]: 2022-01-17T12:17:59Z [INFO] - <-- GET /certificates/aziot-edged-trust-bundle?api-version=2020-09-01 {"host": "2f72756e2f617a696f742f63657274642e736f636b:0"}
Jan 17 14:17:59 l31lnx813 aziot-certd[30134]: 2022-01-17T12:17:59Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:17:59 l31lnx813 aziot-edged[30103]: 2022-01-17T12:17:59Z [INFO] - [work] - - - [2022-01-17 12:17:59.220690949 UTC] "GET /trust-bundle?api-version=2019-01-30 HTTP/1.1" 200 OK 1118 "-" "-" auth_id(-)
Jan 17 14:18:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:18:58Z [INFO] - Checking edge runtime status
Jan 17 14:18:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:18:58Z [INFO] - Edge runtime status is failed, starting module now...
Jan 17 14:18:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:18:58Z [INFO] - Starting module edgeAgent...
Jan 17 14:18:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:18:58Z [INFO] - Starting new listener for module edgeAgent
Jan 17 14:18:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:18:58Z [INFO] - Listener  edgeAgent already started, removing old listener
Jan 17 14:18:58 l31lnx813 aziot-certd[30134]: 2022-01-17T12:18:58Z [INFO] - <-- GET /certificates/aziot-edged-ca?api-version=2020-09-01 {"host": "2f72756e2f617a696f742f63657274642e736f636b:0"}
Jan 17 14:18:58 l31lnx813 aziot-certd[30134]: 2022-01-17T12:18:58Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:18:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:18:58Z [INFO] - Listening on unix:///var/lib/aziot/edged/mnt/edgeAgent.sock with 1 thread for workload API.
Jan 17 14:18:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:18:58Z [INFO] - Successfully started module edgeAgent
Jan 17 14:18:59 l31lnx813 aziot-certd[30134]: 2022-01-17T12:18:59Z [INFO] - <-- GET /certificates/aziot-edged-trust-bundle?api-version=2020-09-01 {"host": "2f72756e2f617a696f742f63657274642e736f636b:0"}
Jan 17 14:18:59 l31lnx813 aziot-certd[30134]: 2022-01-17T12:18:59Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:18:59 l31lnx813 aziot-edged[30103]: 2022-01-17T12:18:59Z [INFO] - [work] - - - [2022-01-17 12:18:59.202954320 UTC] "GET /trust-bundle?api-version=2019-01-30 HTTP/1.1" 200 OK 1118 "-" "-" auth_id(-)
Jan 17 14:19:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:19:58Z [INFO] - Checking edge runtime status
Jan 17 14:19:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:19:58Z [INFO] - Edge runtime status is failed, starting module now...
Jan 17 14:19:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:19:58Z [INFO] - Starting module edgeAgent...
Jan 17 14:19:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:19:58Z [INFO] - Starting new listener for module edgeAgent
Jan 17 14:19:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:19:58Z [INFO] - Listener  edgeAgent already started, removing old listener
Jan 17 14:19:58 l31lnx813 aziot-certd[30134]: 2022-01-17T12:19:58Z [INFO] - <-- GET /certificates/aziot-edged-ca?api-version=2020-09-01 {"host": "2f72756e2f617a696f742f63657274642e736f636b:0"}
Jan 17 14:19:58 l31lnx813 aziot-certd[30134]: 2022-01-17T12:19:58Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:19:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:19:58Z [INFO] - Listening on unix:///var/lib/aziot/edged/mnt/edgeAgent.sock with 1 thread for workload API.
Jan 17 14:19:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:19:58Z [INFO] - Successfully started module edgeAgent
Jan 17 14:19:59 l31lnx813 aziot-certd[30134]: 2022-01-17T12:19:59Z [INFO] - <-- GET /certificates/aziot-edged-trust-bundle?api-version=2020-09-01 {"host": "2f72756e2f617a696f742f63657274642e736f636b:0"}
Jan 17 14:19:59 l31lnx813 aziot-certd[30134]: 2022-01-17T12:19:59Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:19:59 l31lnx813 aziot-edged[30103]: 2022-01-17T12:19:59Z [INFO] - [work] - - - [2022-01-17 12:19:59.265812743 UTC] "GET /trust-bundle?api-version=2019-01-30 HTTP/1.1" 200 OK 1118 "-" "-" auth_id(-)
Jan 17 14:20:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:20:58Z [INFO] - Checking edge runtime status
Jan 17 14:20:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:20:58Z [INFO] - Edge runtime status is failed, starting module now...
Jan 17 14:20:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:20:58Z [INFO] - Starting module edgeAgent...
Jan 17 14:20:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:20:58Z [INFO] - Starting new listener for module edgeAgent
Jan 17 14:20:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:20:58Z [INFO] - Listener  edgeAgent already started, removing old listener
Jan 17 14:20:58 l31lnx813 aziot-certd[30134]: 2022-01-17T12:20:58Z [INFO] - <-- GET /certificates/aziot-edged-ca?api-version=2020-09-01 {"host": "2f72756e2f617a696f742f63657274642e736f636b:0"}
Jan 17 14:20:58 l31lnx813 aziot-certd[30134]: 2022-01-17T12:20:58Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:20:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:20:58Z [INFO] - Listening on unix:///var/lib/aziot/edged/mnt/edgeAgent.sock with 1 thread for workload API.
Jan 17 14:20:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:20:58Z [INFO] - Successfully started module edgeAgent
Jan 17 14:20:59 l31lnx813 aziot-certd[30134]: 2022-01-17T12:20:59Z [INFO] - <-- GET /certificates/aziot-edged-trust-bundle?api-version=2020-09-01 {"host": "2f72756e2f617a696f742f63657274642e736f636b:0"}
Jan 17 14:20:59 l31lnx813 aziot-certd[30134]: 2022-01-17T12:20:59Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:20:59 l31lnx813 aziot-edged[30103]: 2022-01-17T12:20:59Z [INFO] - [work] - - - [2022-01-17 12:20:59.262564022 UTC] "GET /trust-bundle?api-version=2019-01-30 HTTP/1.1" 200 OK 1118 "-" "-" auth_id(-)
Jan 17 14:21:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:21:58Z [INFO] - Checking edge runtime status
Jan 17 14:21:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:21:58Z [INFO] - Edge runtime status is failed, starting module now...
Jan 17 14:21:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:21:58Z [INFO] - Starting module edgeAgent...
Jan 17 14:21:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:21:58Z [INFO] - Starting new listener for module edgeAgent
Jan 17 14:21:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:21:58Z [INFO] - Listener  edgeAgent already started, removing old listener
Jan 17 14:21:58 l31lnx813 aziot-certd[30134]: 2022-01-17T12:21:58Z [INFO] - <-- GET /certificates/aziot-edged-ca?api-version=2020-09-01 {"host": "2f72756e2f617a696f742f63657274642e736f636b:0"}
Jan 17 14:21:58 l31lnx813 aziot-certd[30134]: 2022-01-17T12:21:58Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:21:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:21:58Z [INFO] - Listening on unix:///var/lib/aziot/edged/mnt/edgeAgent.sock with 1 thread for workload API.
Jan 17 14:21:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:21:58Z [INFO] - Successfully started module edgeAgent
Jan 17 14:21:59 l31lnx813 aziot-certd[30134]: 2022-01-17T12:21:59Z [INFO] - <-- GET /certificates/aziot-edged-trust-bundle?api-version=2020-09-01 {"host": "2f72756e2f617a696f742f63657274642e736f636b:0"}
Jan 17 14:21:59 l31lnx813 aziot-certd[30134]: 2022-01-17T12:21:59Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:21:59 l31lnx813 aziot-edged[30103]: 2022-01-17T12:21:59Z [INFO] - [work] - - - [2022-01-17 12:21:59.224695462 UTC] "GET /trust-bundle?api-version=2019-01-30 HTTP/1.1" 200 OK 1118 "-" "-" auth_id(-)
Jan 17 14:22:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:22:58Z [INFO] - Checking edge runtime status
Jan 17 14:22:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:22:58Z [INFO] - Edge runtime status is failed, starting module now...
Jan 17 14:22:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:22:58Z [INFO] - Starting module edgeAgent...
Jan 17 14:22:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:22:58Z [INFO] - Starting new listener for module edgeAgent
Jan 17 14:22:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:22:58Z [INFO] - Listener  edgeAgent already started, removing old listener
Jan 17 14:22:58 l31lnx813 aziot-certd[30134]: 2022-01-17T12:22:58Z [INFO] - <-- GET /certificates/aziot-edged-ca?api-version=2020-09-01 {"host": "2f72756e2f617a696f742f63657274642e736f636b:0"}
Jan 17 14:22:58 l31lnx813 aziot-certd[30134]: 2022-01-17T12:22:58Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:22:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:22:58Z [INFO] - Listening on unix:///var/lib/aziot/edged/mnt/edgeAgent.sock with 1 thread for workload API.
Jan 17 14:22:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:22:58Z [INFO] - Successfully started module edgeAgent
Jan 17 14:22:59 l31lnx813 aziot-certd[30134]: 2022-01-17T12:22:59Z [INFO] - <-- GET /certificates/aziot-edged-trust-bundle?api-version=2020-09-01 {"host": "2f72756e2f617a696f742f63657274642e736f636b:0"}
Jan 17 14:22:59 l31lnx813 aziot-certd[30134]: 2022-01-17T12:22:59Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:22:59 l31lnx813 aziot-edged[30103]: 2022-01-17T12:22:59Z [INFO] - [work] - - - [2022-01-17 12:22:59.232483203 UTC] "GET /trust-bundle?api-version=2019-01-30 HTTP/1.1" 200 OK 1118 "-" "-" auth_id(-)
Jan 17 14:23:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:23:58Z [INFO] - Checking edge runtime status
Jan 17 14:23:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:23:58Z [INFO] - Edge runtime status is failed, starting module now...
Jan 17 14:23:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:23:58Z [INFO] - Starting module edgeAgent...
Jan 17 14:23:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:23:58Z [INFO] - Starting new listener for module edgeAgent
Jan 17 14:23:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:23:58Z [INFO] - Listener  edgeAgent already started, removing old listener
Jan 17 14:23:58 l31lnx813 aziot-certd[30134]: 2022-01-17T12:23:58Z [INFO] - <-- GET /certificates/aziot-edged-ca?api-version=2020-09-01 {"host": "2f72756e2f617a696f742f63657274642e736f636b:0"}
Jan 17 14:23:58 l31lnx813 aziot-certd[30134]: 2022-01-17T12:23:58Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:23:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:23:58Z [INFO] - Listening on unix:///var/lib/aziot/edged/mnt/edgeAgent.sock with 1 thread for workload API.
Jan 17 14:23:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:23:58Z [INFO] - Successfully started module edgeAgent
Jan 17 14:23:59 l31lnx813 aziot-certd[30134]: 2022-01-17T12:23:59Z [INFO] - <-- GET /certificates/aziot-edged-trust-bundle?api-version=2020-09-01 {"host": "2f72756e2f617a696f742f63657274642e736f636b:0"}
Jan 17 14:23:59 l31lnx813 aziot-certd[30134]: 2022-01-17T12:23:59Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:23:59 l31lnx813 aziot-edged[30103]: 2022-01-17T12:23:59Z [INFO] - [work] - - - [2022-01-17 12:23:59.277085772 UTC] "GET /trust-bundle?api-version=2019-01-30 HTTP/1.1" 200 OK 1118 "-" "-" auth_id(-)
Jan 17 14:24:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:24:58Z [INFO] - Checking edge runtime status
Jan 17 14:24:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:24:58Z [INFO] - Edge runtime status is failed, starting module now...
Jan 17 14:24:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:24:58Z [INFO] - Starting module edgeAgent...
Jan 17 14:24:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:24:58Z [INFO] - Starting new listener for module edgeAgent
Jan 17 14:24:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:24:58Z [INFO] - Listener  edgeAgent already started, removing old listener
Jan 17 14:24:58 l31lnx813 aziot-certd[30134]: 2022-01-17T12:24:58Z [INFO] - <-- GET /certificates/aziot-edged-ca?api-version=2020-09-01 {"host": "2f72756e2f617a696f742f63657274642e736f636b:0"}
Jan 17 14:24:58 l31lnx813 aziot-certd[30134]: 2022-01-17T12:24:58Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:24:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:24:58Z [INFO] - Listening on unix:///var/lib/aziot/edged/mnt/edgeAgent.sock with 1 thread for workload API.
Jan 17 14:24:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:24:58Z [INFO] - Successfully started module edgeAgent
Jan 17 14:24:59 l31lnx813 aziot-certd[30134]: 2022-01-17T12:24:59Z [INFO] - <-- GET /certificates/aziot-edged-trust-bundle?api-version=2020-09-01 {"host": "2f72756e2f617a696f742f63657274642e736f636b:0"}
Jan 17 14:24:59 l31lnx813 aziot-certd[30134]: 2022-01-17T12:24:59Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:24:59 l31lnx813 aziot-edged[30103]: 2022-01-17T12:24:59Z [INFO] - [work] - - - [2022-01-17 12:24:59.286120883 UTC] "GET /trust-bundle?api-version=2019-01-30 HTTP/1.1" 200 OK 1118 "-" "-" auth_id(-)
Jan 17 14:24:59 l31lnx813 aziot-edged[30103]: 2022-01-17T12:24:59Z [INFO] - [mgmt] - - - [2022-01-17 12:24:59.567549579 UTC] "GET /modules?api-version=2020-07-07 HTTP/1.1" 200 OK 690 "-" "iotedge/0.1.0" auth_id(-)
Jan 17 14:25:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:25:58Z [INFO] - Checking edge runtime status
Jan 17 14:25:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:25:58Z [INFO] - Edge runtime status is failed, starting module now...
Jan 17 14:25:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:25:58Z [INFO] - Starting module edgeAgent...
Jan 17 14:25:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:25:58Z [INFO] - Starting new listener for module edgeAgent
Jan 17 14:25:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:25:58Z [INFO] - Listener  edgeAgent already started, removing old listener
Jan 17 14:25:58 l31lnx813 aziot-certd[30134]: 2022-01-17T12:25:58Z [INFO] - <-- GET /certificates/aziot-edged-ca?api-version=2020-09-01 {"host": "2f72756e2f617a696f742f63657274642e736f636b:0"}
Jan 17 14:25:58 l31lnx813 aziot-certd[30134]: 2022-01-17T12:25:58Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:25:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:25:58Z [INFO] - Listening on unix:///var/lib/aziot/edged/mnt/edgeAgent.sock with 1 thread for workload API.
Jan 17 14:25:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:25:58Z [INFO] - Successfully started module edgeAgent
Jan 17 14:25:59 l31lnx813 aziot-certd[30134]: 2022-01-17T12:25:59Z [INFO] - <-- GET /certificates/aziot-edged-trust-bundle?api-version=2020-09-01 {"host": "2f72756e2f617a696f742f63657274642e736f636b:0"}
Jan 17 14:25:59 l31lnx813 aziot-certd[30134]: 2022-01-17T12:25:59Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:25:59 l31lnx813 aziot-edged[30103]: 2022-01-17T12:25:59Z [INFO] - [work] - - - [2022-01-17 12:25:59.276345244 UTC] "GET /trust-bundle?api-version=2019-01-30 HTTP/1.1" 200 OK 1118 "-" "-" auth_id(-)
Jan 17 14:26:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:26:58Z [INFO] - Checking edge runtime status
Jan 17 14:26:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:26:58Z [INFO] - Edge runtime status is failed, starting module now...
Jan 17 14:26:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:26:58Z [INFO] - Starting module edgeAgent...
Jan 17 14:26:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:26:58Z [INFO] - Starting new listener for module edgeAgent
Jan 17 14:26:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:26:58Z [INFO] - Listener  edgeAgent already started, removing old listener
Jan 17 14:26:58 l31lnx813 aziot-certd[30134]: 2022-01-17T12:26:58Z [INFO] - <-- GET /certificates/aziot-edged-ca?api-version=2020-09-01 {"host": "2f72756e2f617a696f742f63657274642e736f636b:0"}
Jan 17 14:26:58 l31lnx813 aziot-certd[30134]: 2022-01-17T12:26:58Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:26:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:26:58Z [INFO] - Listening on unix:///var/lib/aziot/edged/mnt/edgeAgent.sock with 1 thread for workload API.
Jan 17 14:26:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:26:58Z [INFO] - Successfully started module edgeAgent
Jan 17 14:26:59 l31lnx813 aziot-certd[30134]: 2022-01-17T12:26:59Z [INFO] - <-- GET /certificates/aziot-edged-trust-bundle?api-version=2020-09-01 {"host": "2f72756e2f617a696f742f63657274642e736f636b:0"}
Jan 17 14:26:59 l31lnx813 aziot-certd[30134]: 2022-01-17T12:26:59Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:26:59 l31lnx813 aziot-edged[30103]: 2022-01-17T12:26:59Z [INFO] - [work] - - - [2022-01-17 12:26:59.226087961 UTC] "GET /trust-bundle?api-version=2019-01-30 HTTP/1.1" 200 OK 1118 "-" "-" auth_id(-)
Jan 17 14:27:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:27:58Z [INFO] - Checking edge runtime status
Jan 17 14:27:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:27:58Z [INFO] - Edge runtime status is failed, starting module now...
Jan 17 14:27:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:27:58Z [INFO] - Starting module edgeAgent...
Jan 17 14:27:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:27:58Z [INFO] - Starting new listener for module edgeAgent
Jan 17 14:27:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:27:58Z [INFO] - Listener  edgeAgent already started, removing old listener
Jan 17 14:27:58 l31lnx813 aziot-certd[30134]: 2022-01-17T12:27:58Z [INFO] - <-- GET /certificates/aziot-edged-ca?api-version=2020-09-01 {"host": "2f72756e2f617a696f742f63657274642e736f636b:0"}
Jan 17 14:27:58 l31lnx813 aziot-certd[30134]: 2022-01-17T12:27:58Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:27:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:27:58Z [INFO] - Listening on unix:///var/lib/aziot/edged/mnt/edgeAgent.sock with 1 thread for workload API.
Jan 17 14:27:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:27:58Z [INFO] - Successfully started module edgeAgent
Jan 17 14:27:59 l31lnx813 aziot-certd[30134]: 2022-01-17T12:27:59Z [INFO] - <-- GET /certificates/aziot-edged-trust-bundle?api-version=2020-09-01 {"host": "2f72756e2f617a696f742f63657274642e736f636b:0"}
Jan 17 14:27:59 l31lnx813 aziot-certd[30134]: 2022-01-17T12:27:59Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:27:59 l31lnx813 aziot-edged[30103]: 2022-01-17T12:27:59Z [INFO] - [work] - - - [2022-01-17 12:27:59.221208426 UTC] "GET /trust-bundle?api-version=2019-01-30 HTTP/1.1" 200 OK 1118 "-" "-" auth_id(-)
Jan 17 14:28:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:28:58Z [INFO] - Checking edge runtime status
Jan 17 14:28:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:28:58Z [INFO] - Edge runtime status is failed, starting module now...
Jan 17 14:28:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:28:58Z [INFO] - Starting module edgeAgent...
Jan 17 14:28:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:28:58Z [INFO] - Starting new listener for module edgeAgent
Jan 17 14:28:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:28:58Z [INFO] - Listener  edgeAgent already started, removing old listener
Jan 17 14:28:58 l31lnx813 aziot-certd[30134]: 2022-01-17T12:28:58Z [INFO] - <-- GET /certificates/aziot-edged-ca?api-version=2020-09-01 {"host": "2f72756e2f617a696f742f63657274642e736f636b:0"}
Jan 17 14:28:58 l31lnx813 aziot-certd[30134]: 2022-01-17T12:28:58Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:28:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:28:58Z [INFO] - Listening on unix:///var/lib/aziot/edged/mnt/edgeAgent.sock with 1 thread for workload API.
Jan 17 14:28:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:28:58Z [INFO] - Successfully started module edgeAgent
Jan 17 14:28:59 l31lnx813 aziot-certd[30134]: 2022-01-17T12:28:59Z [INFO] - <-- GET /certificates/aziot-edged-trust-bundle?api-version=2020-09-01 {"host": "2f72756e2f617a696f742f63657274642e736f636b:0"}
Jan 17 14:28:59 l31lnx813 aziot-certd[30134]: 2022-01-17T12:28:59Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:28:59 l31lnx813 aziot-edged[30103]: 2022-01-17T12:28:59Z [INFO] - [work] - - - [2022-01-17 12:28:59.288325678 UTC] "GET /trust-bundle?api-version=2019-01-30 HTTP/1.1" 200 OK 1118 "-" "-" auth_id(-)
Jan 17 14:29:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:29:58Z [INFO] - Checking edge runtime status
Jan 17 14:29:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:29:58Z [INFO] - Edge runtime status is failed, starting module now...
Jan 17 14:29:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:29:58Z [INFO] - Starting module edgeAgent...
Jan 17 14:29:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:29:58Z [INFO] - Starting new listener for module edgeAgent
Jan 17 14:29:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:29:58Z [INFO] - Listener  edgeAgent already started, removing old listener
Jan 17 14:29:58 l31lnx813 aziot-certd[30134]: 2022-01-17T12:29:58Z [INFO] - <-- GET /certificates/aziot-edged-ca?api-version=2020-09-01 {"host": "2f72756e2f617a696f742f63657274642e736f636b:0"}
Jan 17 14:29:58 l31lnx813 aziot-certd[30134]: 2022-01-17T12:29:58Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:29:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:29:58Z [INFO] - Listening on unix:///var/lib/aziot/edged/mnt/edgeAgent.sock with 1 thread for workload API.
Jan 17 14:29:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:29:58Z [INFO] - Successfully started module edgeAgent
Jan 17 14:29:59 l31lnx813 aziot-certd[30134]: 2022-01-17T12:29:59Z [INFO] - <-- GET /certificates/aziot-edged-trust-bundle?api-version=2020-09-01 {"host": "2f72756e2f617a696f742f63657274642e736f636b:0"}
Jan 17 14:29:59 l31lnx813 aziot-certd[30134]: 2022-01-17T12:29:59Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:29:59 l31lnx813 aziot-edged[30103]: 2022-01-17T12:29:59Z [INFO] - [work] - - - [2022-01-17 12:29:59.279480759 UTC] "GET /trust-bundle?api-version=2019-01-30 HTTP/1.1" 200 OK 1118 "-" "-" auth_id(-)
Jan 17 14:30:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:30:58Z [INFO] - Checking edge runtime status
Jan 17 14:30:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:30:58Z [INFO] - Edge runtime status is failed, starting module now...
Jan 17 14:30:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:30:58Z [INFO] - Starting module edgeAgent...
Jan 17 14:30:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:30:58Z [INFO] - Starting new listener for module edgeAgent
Jan 17 14:30:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:30:58Z [INFO] - Listener  edgeAgent already started, removing old listener
Jan 17 14:30:58 l31lnx813 aziot-certd[30134]: 2022-01-17T12:30:58Z [INFO] - <-- GET /certificates/aziot-edged-ca?api-version=2020-09-01 {"host": "2f72756e2f617a696f742f63657274642e736f636b:0"}
Jan 17 14:30:58 l31lnx813 aziot-certd[30134]: 2022-01-17T12:30:58Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:30:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:30:58Z [INFO] - Listening on unix:///var/lib/aziot/edged/mnt/edgeAgent.sock with 1 thread for workload API.
Jan 17 14:30:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:30:58Z [INFO] - Successfully started module edgeAgent
Jan 17 14:30:59 l31lnx813 aziot-certd[30134]: 2022-01-17T12:30:59Z [INFO] - <-- GET /certificates/aziot-edged-trust-bundle?api-version=2020-09-01 {"host": "2f72756e2f617a696f742f63657274642e736f636b:0"}
Jan 17 14:30:59 l31lnx813 aziot-certd[30134]: 2022-01-17T12:30:59Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:30:59 l31lnx813 aziot-edged[30103]: 2022-01-17T12:30:59Z [INFO] - [work] - - - [2022-01-17 12:30:59.240842281 UTC] "GET /trust-bundle?api-version=2019-01-30 HTTP/1.1" 200 OK 1118 "-" "-" auth_id(-)
Jan 17 14:31:35 l31lnx813 aziot-edged[30103]: 2022-01-17T12:31:35Z [INFO] - [mgmt] - - - [2022-01-17 12:31:35.380276937 UTC] "GET /modules?api-version=2020-07-07 HTTP/1.1" 200 OK 690 "-" "iotedge/0.1.0" auth_id(-)
Jan 17 14:31:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:31:58Z [INFO] - Checking edge runtime status
Jan 17 14:31:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:31:58Z [INFO] - Edge runtime status is failed, starting module now...
Jan 17 14:31:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:31:58Z [INFO] - Starting module edgeAgent...
Jan 17 14:31:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:31:58Z [INFO] - Starting new listener for module edgeAgent
Jan 17 14:31:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:31:58Z [INFO] - Listener  edgeAgent already started, removing old listener
Jan 17 14:31:58 l31lnx813 aziot-certd[30134]: 2022-01-17T12:31:58Z [INFO] - <-- GET /certificates/aziot-edged-ca?api-version=2020-09-01 {"host": "2f72756e2f617a696f742f63657274642e736f636b:0"}
Jan 17 14:31:58 l31lnx813 aziot-certd[30134]: 2022-01-17T12:31:58Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:31:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:31:58Z [INFO] - Listening on unix:///var/lib/aziot/edged/mnt/edgeAgent.sock with 1 thread for workload API.
Jan 17 14:31:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:31:58Z [INFO] - Successfully started module edgeAgent
Jan 17 14:31:59 l31lnx813 aziot-certd[30134]: 2022-01-17T12:31:59Z [INFO] - <-- GET /certificates/aziot-edged-trust-bundle?api-version=2020-09-01 {"host": "2f72756e2f617a696f742f63657274642e736f636b:0"}
Jan 17 14:31:59 l31lnx813 aziot-certd[30134]: 2022-01-17T12:31:59Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:31:59 l31lnx813 aziot-edged[30103]: 2022-01-17T12:31:59Z [INFO] - [work] - - - [2022-01-17 12:31:59.208518441 UTC] "GET /trust-bundle?api-version=2019-01-30 HTTP/1.1" 200 OK 1118 "-" "-" auth_id(-)
Jan 17 14:32:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:32:58Z [INFO] - Checking edge runtime status
Jan 17 14:32:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:32:58Z [INFO] - Edge runtime status is failed, starting module now...
Jan 17 14:32:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:32:58Z [INFO] - Starting module edgeAgent...
Jan 17 14:32:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:32:58Z [INFO] - Starting new listener for module edgeAgent
Jan 17 14:32:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:32:58Z [INFO] - Listener  edgeAgent already started, removing old listener
Jan 17 14:32:58 l31lnx813 aziot-certd[30134]: 2022-01-17T12:32:58Z [INFO] - <-- GET /certificates/aziot-edged-ca?api-version=2020-09-01 {"host": "2f72756e2f617a696f742f63657274642e736f636b:0"}
Jan 17 14:32:58 l31lnx813 aziot-certd[30134]: 2022-01-17T12:32:58Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:32:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:32:58Z [INFO] - Listening on unix:///var/lib/aziot/edged/mnt/edgeAgent.sock with 1 thread for workload API.
Jan 17 14:32:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:32:58Z [INFO] - Successfully started module edgeAgent
Jan 17 14:32:59 l31lnx813 aziot-certd[30134]: 2022-01-17T12:32:59Z [INFO] - <-- GET /certificates/aziot-edged-trust-bundle?api-version=2020-09-01 {"host": "2f72756e2f617a696f742f63657274642e736f636b:0"}
Jan 17 14:32:59 l31lnx813 aziot-certd[30134]: 2022-01-17T12:32:59Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:32:59 l31lnx813 aziot-edged[30103]: 2022-01-17T12:32:59Z [INFO] - [work] - - - [2022-01-17 12:32:59.297545468 UTC] "GET /trust-bundle?api-version=2019-01-30 HTTP/1.1" 200 OK 1118 "-" "-" auth_id(-)
Jan 17 14:33:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:33:58Z [INFO] - Checking edge runtime status
Jan 17 14:33:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:33:58Z [INFO] - Edge runtime status is failed, starting module now...
Jan 17 14:33:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:33:58Z [INFO] - Starting module edgeAgent...
Jan 17 14:33:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:33:58Z [INFO] - Starting new listener for module edgeAgent
Jan 17 14:33:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:33:58Z [INFO] - Listener  edgeAgent already started, removing old listener
Jan 17 14:33:58 l31lnx813 aziot-certd[30134]: 2022-01-17T12:33:58Z [INFO] - <-- GET /certificates/aziot-edged-ca?api-version=2020-09-01 {"host": "2f72756e2f617a696f742f63657274642e736f636b:0"}
Jan 17 14:33:58 l31lnx813 aziot-certd[30134]: 2022-01-17T12:33:58Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:33:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:33:58Z [INFO] - Listening on unix:///var/lib/aziot/edged/mnt/edgeAgent.sock with 1 thread for workload API.
Jan 17 14:33:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:33:58Z [INFO] - Successfully started module edgeAgent
Jan 17 14:33:59 l31lnx813 aziot-certd[30134]: 2022-01-17T12:33:59Z [INFO] - <-- GET /certificates/aziot-edged-trust-bundle?api-version=2020-09-01 {"host": "2f72756e2f617a696f742f63657274642e736f636b:0"}
Jan 17 14:33:59 l31lnx813 aziot-certd[30134]: 2022-01-17T12:33:59Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:33:59 l31lnx813 aziot-edged[30103]: 2022-01-17T12:33:59Z [INFO] - [work] - - - [2022-01-17 12:33:59.318245995 UTC] "GET /trust-bundle?api-version=2019-01-30 HTTP/1.1" 200 OK 1118 "-" "-" auth_id(-)
Jan 17 14:34:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:34:58Z [INFO] - Checking edge runtime status
Jan 17 14:34:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:34:58Z [INFO] - Edge runtime status is failed, starting module now...
Jan 17 14:34:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:34:58Z [INFO] - Starting module edgeAgent...
Jan 17 14:34:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:34:58Z [INFO] - Starting new listener for module edgeAgent
Jan 17 14:34:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:34:58Z [INFO] - Listener  edgeAgent already started, removing old listener
Jan 17 14:34:58 l31lnx813 aziot-certd[30134]: 2022-01-17T12:34:58Z [INFO] - <-- GET /certificates/aziot-edged-ca?api-version=2020-09-01 {"host": "2f72756e2f617a696f742f63657274642e736f636b:0"}
Jan 17 14:34:58 l31lnx813 aziot-certd[30134]: 2022-01-17T12:34:58Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:34:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:34:58Z [INFO] - Listening on unix:///var/lib/aziot/edged/mnt/edgeAgent.sock with 1 thread for workload API.
Jan 17 14:34:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:34:58Z [INFO] - Successfully started module edgeAgent
Jan 17 14:34:59 l31lnx813 aziot-certd[30134]: 2022-01-17T12:34:59Z [INFO] - <-- GET /certificates/aziot-edged-trust-bundle?api-version=2020-09-01 {"host": "2f72756e2f617a696f742f63657274642e736f636b:0"}
Jan 17 14:34:59 l31lnx813 aziot-certd[30134]: 2022-01-17T12:34:59Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:34:59 l31lnx813 aziot-edged[30103]: 2022-01-17T12:34:59Z [INFO] - [work] - - - [2022-01-17 12:34:59.232333170 UTC] "GET /trust-bundle?api-version=2019-01-30 HTTP/1.1" 200 OK 1118 "-" "-" auth_id(-)
Jan 17 14:35:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:35:58Z [INFO] - Checking edge runtime status
Jan 17 14:35:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:35:58Z [INFO] - Edge runtime status is failed, starting module now...
Jan 17 14:35:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:35:58Z [INFO] - Starting module edgeAgent...
Jan 17 14:35:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:35:58Z [INFO] - Starting new listener for module edgeAgent
Jan 17 14:35:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:35:58Z [INFO] - Listener  edgeAgent already started, removing old listener
Jan 17 14:35:58 l31lnx813 aziot-certd[30134]: 2022-01-17T12:35:58Z [INFO] - <-- GET /certificates/aziot-edged-ca?api-version=2020-09-01 {"host": "2f72756e2f617a696f742f63657274642e736f636b:0"}
Jan 17 14:35:58 l31lnx813 aziot-certd[30134]: 2022-01-17T12:35:58Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:35:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:35:58Z [INFO] - Listening on unix:///var/lib/aziot/edged/mnt/edgeAgent.sock with 1 thread for workload API.
Jan 17 14:35:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:35:58Z [INFO] - Successfully started module edgeAgent
Jan 17 14:35:59 l31lnx813 aziot-certd[30134]: 2022-01-17T12:35:59Z [INFO] - <-- GET /certificates/aziot-edged-trust-bundle?api-version=2020-09-01 {"host": "2f72756e2f617a696f742f63657274642e736f636b:0"}
Jan 17 14:35:59 l31lnx813 aziot-certd[30134]: 2022-01-17T12:35:59Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:35:59 l31lnx813 aziot-edged[30103]: 2022-01-17T12:35:59Z [INFO] - [work] - - - [2022-01-17 12:35:59.233628930 UTC] "GET /trust-bundle?api-version=2019-01-30 HTTP/1.1" 200 OK 1118 "-" "-" auth_id(-)
Jan 17 14:36:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:36:58Z [INFO] - Checking edge runtime status
Jan 17 14:36:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:36:58Z [INFO] - Edge runtime status is failed, starting module now...
Jan 17 14:36:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:36:58Z [INFO] - Starting module edgeAgent...
Jan 17 14:36:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:36:58Z [INFO] - Starting new listener for module edgeAgent
Jan 17 14:36:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:36:58Z [INFO] - Listener  edgeAgent already started, removing old listener
Jan 17 14:36:58 l31lnx813 aziot-certd[30134]: 2022-01-17T12:36:58Z [INFO] - <-- GET /certificates/aziot-edged-ca?api-version=2020-09-01 {"host": "2f72756e2f617a696f742f63657274642e736f636b:0"}
Jan 17 14:36:58 l31lnx813 aziot-certd[30134]: 2022-01-17T12:36:58Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:36:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:36:58Z [INFO] - Listening on unix:///var/lib/aziot/edged/mnt/edgeAgent.sock with 1 thread for workload API.
Jan 17 14:36:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:36:58Z [INFO] - Successfully started module edgeAgent
Jan 17 14:36:59 l31lnx813 aziot-certd[30134]: 2022-01-17T12:36:59Z [INFO] - <-- GET /certificates/aziot-edged-trust-bundle?api-version=2020-09-01 {"host": "2f72756e2f617a696f742f63657274642e736f636b:0"}
Jan 17 14:36:59 l31lnx813 aziot-certd[30134]: 2022-01-17T12:36:59Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:36:59 l31lnx813 aziot-edged[30103]: 2022-01-17T12:36:59Z [INFO] - [work] - - - [2022-01-17 12:36:59.231575325 UTC] "GET /trust-bundle?api-version=2019-01-30 HTTP/1.1" 200 OK 1118 "-" "-" auth_id(-)
Jan 17 14:37:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:37:58Z [INFO] - Checking edge runtime status
Jan 17 14:37:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:37:58Z [INFO] - Edge runtime status is failed, starting module now...
Jan 17 14:37:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:37:58Z [INFO] - Starting module edgeAgent...
Jan 17 14:37:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:37:58Z [INFO] - Starting new listener for module edgeAgent
Jan 17 14:37:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:37:58Z [INFO] - Listener  edgeAgent already started, removing old listener
Jan 17 14:37:58 l31lnx813 aziot-certd[30134]: 2022-01-17T12:37:58Z [INFO] - <-- GET /certificates/aziot-edged-ca?api-version=2020-09-01 {"host": "2f72756e2f617a696f742f63657274642e736f636b:0"}
Jan 17 14:37:58 l31lnx813 aziot-certd[30134]: 2022-01-17T12:37:58Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:37:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:37:58Z [INFO] - Listening on unix:///var/lib/aziot/edged/mnt/edgeAgent.sock with 1 thread for workload API.
Jan 17 14:37:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:37:58Z [INFO] - Successfully started module edgeAgent
Jan 17 14:37:59 l31lnx813 aziot-certd[30134]: 2022-01-17T12:37:59Z [INFO] - <-- GET /certificates/aziot-edged-trust-bundle?api-version=2020-09-01 {"host": "2f72756e2f617a696f742f63657274642e736f636b:0"}
Jan 17 14:37:59 l31lnx813 aziot-certd[30134]: 2022-01-17T12:37:59Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:37:59 l31lnx813 aziot-edged[30103]: 2022-01-17T12:37:59Z [INFO] - [work] - - - [2022-01-17 12:37:59.291417443 UTC] "GET /trust-bundle?api-version=2019-01-30 HTTP/1.1" 200 OK 1118 "-" "-" auth_id(-)
Jan 17 14:38:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:38:58Z [INFO] - Checking edge runtime status
Jan 17 14:38:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:38:58Z [INFO] - Edge runtime status is failed, starting module now...
Jan 17 14:38:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:38:58Z [INFO] - Starting module edgeAgent...
Jan 17 14:38:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:38:58Z [INFO] - Starting new listener for module edgeAgent
Jan 17 14:38:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:38:58Z [INFO] - Listener  edgeAgent already started, removing old listener
Jan 17 14:38:58 l31lnx813 aziot-certd[30134]: 2022-01-17T12:38:58Z [INFO] - <-- GET /certificates/aziot-edged-ca?api-version=2020-09-01 {"host": "2f72756e2f617a696f742f63657274642e736f636b:0"}
Jan 17 14:38:58 l31lnx813 aziot-certd[30134]: 2022-01-17T12:38:58Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:38:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:38:58Z [INFO] - Listening on unix:///var/lib/aziot/edged/mnt/edgeAgent.sock with 1 thread for workload API.
Jan 17 14:38:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:38:58Z [INFO] - Successfully started module edgeAgent
Jan 17 14:38:59 l31lnx813 aziot-certd[30134]: 2022-01-17T12:38:59Z [INFO] - <-- GET /certificates/aziot-edged-trust-bundle?api-version=2020-09-01 {"host": "2f72756e2f617a696f742f63657274642e736f636b:0"}
Jan 17 14:38:59 l31lnx813 aziot-certd[30134]: 2022-01-17T12:38:59Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:38:59 l31lnx813 aziot-edged[30103]: 2022-01-17T12:38:59Z [INFO] - [work] - - - [2022-01-17 12:38:59.200166812 UTC] "GET /trust-bundle?api-version=2019-01-30 HTTP/1.1" 200 OK 1118 "-" "-" auth_id(-)
Jan 17 14:39:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:39:58Z [INFO] - Checking edge runtime status
Jan 17 14:39:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:39:58Z [INFO] - Edge runtime status is failed, starting module now...
Jan 17 14:39:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:39:58Z [INFO] - Starting module edgeAgent...
Jan 17 14:39:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:39:58Z [INFO] - Starting new listener for module edgeAgent
Jan 17 14:39:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:39:58Z [INFO] - Listener  edgeAgent already started, removing old listener
Jan 17 14:39:58 l31lnx813 aziot-certd[30134]: 2022-01-17T12:39:58Z [INFO] - <-- GET /certificates/aziot-edged-ca?api-version=2020-09-01 {"host": "2f72756e2f617a696f742f63657274642e736f636b:0"}
Jan 17 14:39:58 l31lnx813 aziot-certd[30134]: 2022-01-17T12:39:58Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:39:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:39:58Z [INFO] - Listening on unix:///var/lib/aziot/edged/mnt/edgeAgent.sock with 1 thread for workload API.
Jan 17 14:39:58 l31lnx813 aziot-edged[30103]: 2022-01-17T12:39:58Z [INFO] - Successfully started module edgeAgent
Jan 17 14:39:59 l31lnx813 aziot-certd[30134]: 2022-01-17T12:39:59Z [INFO] - <-- GET /certificates/aziot-edged-trust-bundle?api-version=2020-09-01 {"host": "2f72756e2f617a696f742f63657274642e736f636b:0"}
Jan 17 14:39:59 l31lnx813 aziot-certd[30134]: 2022-01-17T12:39:59Z [INFO] - --> 200 {"content-type": "application/json"}
Jan 17 14:39:59 l31lnx813 aziot-edged[30103]: 2022-01-17T12:39:59Z [INFO] - [work] - - - [2022-01-17 12:39:59.222078098 UTC] "GET /trust-bundle?api-version=2019-01-30 HTTP/1.1" 200 OK 1118 "-" "-" auth_id(-)
edge-agent logs
<6> 2022-01-17 12:40:59.258 +00:00 [INF] - Experimental features configuration: {"Enabled":false,"DisableCloudSubscriptions":false}
<6> 2022-01-17 12:40:59.345 +00:00 [INF] - Installing certificates [CN=iotedged workload ca l31lnx813:04/17/2022 12:04:26] to Root
<0> 2022-01-17 12:40:59.359 +00:00 [FTL] - Fatal error building application.
System.Security.Cryptography.CryptographicException: The X509 certificate could not be added to the store.
 ---> System.UnauthorizedAccessException: Access to the path '/home/edgeagentuser/.dotnet/corefx/cryptography/x509stores/root' is denied.
 ---> System.IO.IOException: Permission denied
   --- End of inner exception stack trace ---
   at System.IO.FileSystem.CreateDirectory(String fullPath)
   at System.IO.Directory.CreateDirectory(String path)
   at Internal.Cryptography.Pal.DirectoryBasedStoreProvider.AddCertToStore(ICertificatePal certPal)
   at Internal.Cryptography.Pal.DirectoryBasedStoreProvider.Add(ICertificatePal certPal)
   --- End of inner exception stack trace ---
   at Internal.Cryptography.Pal.DirectoryBasedStoreProvider.Add(ICertificatePal certPal)
   at System.Security.Cryptography.X509Certificates.X509Store.Add(X509Certificate2 certificate)
   at Microsoft.Azure.Devices.Edge.Util.CertificateHelper.InstallCertificates(IEnumerable`1 certificateChain, ILogger logger) in /mnt/vss/_work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/CertificateHelper.cs:line 228
   at Microsoft.Azure.Devices.Edge.Agent.Service.Program.MainAsync(IConfiguration configuration) in /mnt/vss/_work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.Service/Program.cs:line 187

Additional Information

Please provide any additional information that may be helpful in understanding the issue.

@ancaantochi
Copy link
Contributor

Hi @harounshehata
Did you configure the module path to bind to host? This is recommended for storage folder and then you have to set up permissions on host, details here:
https://docs.microsoft.com/en-us/azure/iot-edge/how-to-access-host-storage-from-module?view=iotedge-2020-11#link-module-storage-to-device-storage

@ancaantochi ancaantochi self-assigned this Jan 18, 2022
@ancaantochi ancaantochi added the needinfo Needs more info from issue reporter / updates from PR submitter label Jan 18, 2022
@harounshehata
Copy link
Author

Yes for both the edgeHub and edgeAgent the Hostbindings are created. The permissions are also set according to the documentation.

@ancaantochi
Copy link
Contributor

can you please share the binding you set for edgeagent?

@harounshehata
Copy link
Author

Yes sure these are the container create options:
{ "HostConfig": { "Binds": [ "/etc/iotedge/storage/edgeAgent:/tmp/edgeAgent" ] } }

@harounshehata
Copy link
Author

These are the file permissions:
image

@asergaz
Copy link

asergaz commented Jan 19, 2022

@harounshehata
Copy link
Author

@asergaz yes I posted it, I think we have introduced a circular dependency now, the SO post links to this Github issue and vice versa :D

@ancaantochi
Copy link
Contributor

Can you please run these commands and share the results:

sudo docker exec -it edgeAgent id edgeagentuser

sudo id azureuser

@harounshehata
Copy link
Author

harounshehata commented Jan 20, 2022

image

@harounshehata
Copy link
Author

I only have these users:
image

@ancaantochi
Copy link
Contributor

ancaantochi commented Jan 21, 2022

  • Was this an upgrade from 1.0.x to 1.2?
  • Do you set the user id for edgeAgent in createOptions?

Can you please run 2 more commands:

sudo docker inspect edgeAgent

ps auxww | grep Edge.Agent

@harounshehata
Copy link
Author

harounshehata commented Jan 21, 2022

Result of the first command:
IotEdge-issue.zip

Result of the second command:
v798635 19062 0.0 0.0 13144 1044 pts/0 S+ 12:36 0:00 grep --color=auto Edge.Agent

@harounshehata
Copy link
Author

harounshehata commented Jan 21, 2022

No I am not doing anything special in the createOptions, this is an almost empty deployment.
edge-deployment.zip

@ancaantochi
Copy link
Contributor

In aziot-edged logs there a few errors when it starts, was this an update from a previous version?

Sorry, the ps command didn't help because edgeAgent was not running, can you please run it in a loop to catch the result while edgeAgent is running. Following command restarts iotedge (to trigger an edgeAgent start) and runs ps in a loop:

sudo iotedge system restart && for i in {1..5}; do ps augxww | fgrep Edge.Agent; sleep 10; done

please change the upper limit if necessary until the result contains 2 lines, second one should look like:

<your_user> 31718 0.3 0.6 2360044 103960 ? Ssl 01:22 3:55 /usr/bin/dotnet Microsoft.Azure.Devices.Edge.Agent.Service.dll

then please check what is the user id: sudo id <your_user>

@harounshehata
Copy link
Author

No I did not have a previous version, this is a fresh installation of the edge-runtime.
I don't thing looping through this helps, because the edgeAgent fails on startup immediatly and is never in the running state.
image

@ancaantochi
Copy link
Contributor

It should start at some point and quickly fail, if you can please change the loop {1..100} and sleep 1 instead of 10

@harounshehata
Copy link
Author

image

@harounshehata
Copy link
Author

The error is really misleading but I found the root cause. For some reason the runtime was creating old version of edgeAgent and edgeHub (1.1.0), I have manually specified 1.2.7 now in the deployment manifest and restarted the device and then it was almost working.
Then I fixed the issue with the edgeHub permissions, using this stack overflow post
(https://stackoverflow.com/questions/63904695/ubuntu-iot-edge-edgehub-module-fails-with-error-139-access-to-the-path-tmp-e)
Unfortunately, my command sudo iotedge check still doesn't work and is stuck after the first 6 lines.
The IoTHub has public access disabled and we are using a private endpoint, is this the reason why the command might not work?
Sync with cloud seems to work and telemetry will be tested soon, so I think I can assume the network setup is correct :D
image

@jlian
Copy link
Member

jlian commented Jan 27, 2022

@harounshehata Can you confirm that the initial issue (EdgeAgent failing) is now resolved? There's connectivity with IoT Hub?

@ancaantochi do you have any idea on the iotedge check issue?

@harounshehata
Copy link
Author

Yes the issue is resolved and I can successfully connect to the IoT Hub and telemetry messages are received in the cloud.
The iotedge check issue still persists

@ancaantochi
Copy link
Contributor

If you leave the command iotedge check is it still blocked after a few minutes? I tried to reproduce the issue with blocking public network access on 443 and I can see that after the first 6 lines it takes a while to move on, that's because it tries to connect to https://aka.ms/latest-aziot-identity-service to get the latest released version, It should timeout and continue the check, like this:

√ keyd configuration is well-formed - OK
√ certd configuration is well-formed - OK
√ tpmd configuration is well-formed - OK
√ identityd configuration is well-formed - OK
√ daemon configurations up-to-date with config.toml - OK
√ identityd config toml file specifies a valid hostname - OK
× aziot-identity-service package is up-to-date - Error
could not query https://aka.ms/latest-aziot-identity-service for latest available version
...

@harounshehata
Copy link
Author

Yes I can confirm that this was the case, thanks for your great help!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
customer-reported iotedge needinfo Needs more info from issue reporter / updates from PR submitter
Projects
None yet
Development

No branches or pull requests

4 participants