device update with x509 certificate

[PradeepKiruvale]

Hi All,

We were playing around with the device update agent, connecting to Azure, update the device software packages. We could connect the agent to Azure using the Shared Acess keys. But is there any way to do it using the x509 certs?. Please guide us if there any documents or links to some blogs.

[ValOlson]

Hello,

Thank you for your question. You can connect the Device Update agent as a Module (see how to) only using the SAS key because we have a current limitation on the support from IoT Identity Service . This should be expanded in the future.

Alternatively if you are ok with using the Device identity, you can connect the Device Update agent using SAS keys, TPM or x509 certs (see how to).

[PradeepKiruvale]

Thanks for your answer. But the second part of your answer about using the device identity to connect the device agent to azure is not so clear to me from the link. Could you please help me, how can we use certificates?

[nihemstr]

Hey Pradeep,

Sorry this took so long I had to follow up with some of our internal partners to make sure I understood what the process would be for you and if we natively supported the scenario you're discussing.

From what I gather you're trying to run the DU agent which will connect to an IotHub device (not a module) using an x509 certificate without using IotEdge. We don't currently support connecting using self-signed x509 certificates out of the box without IotEdge but I outline a method for doing so if you want to take our reference agent and make the code changes yourself.

To make this work you'll need to follow the next three steps. Please note that only the third step requires you to crack open the agent and add some code.

1. Create the primary and secondary certificates you want to use for the connection. You can read generally about how the IotHub uses x509 certificates and how they apply to the configuration of your devices here
   1. For the development environment you can generate self-signed certificates using the steps found here
   2. For production environments you should be using CA signed certificates obtained from your Certified Authority.Learn more here
2. Once you've generated your certificates and gotten your x509 thumbprints you simply need to create the device
   1. For both Iot Devices and Iot Edge Devices you simply need to navigate to the IotHub on which you wish to create these devices, select "New" under the Iot Devices or IotEdge Devices leaf (Microsoft for the sidebar tab)
   2. In the information panel you must select either X.509 Self-Signed or X.509 CA Signed depending on what kind of certificate you generated in (1). Please note that for any and all production environments you should be using CA signed certificates. Self-Signed certificates should only ever be used in development phases.
   3. Enter the thumbprints for the primary and secondary certificates obtained above into their respective fields.
   4. Press "Save"
   5. You should now be looking at your fancy new x509 authenticating device!
3. Setup DU to use the certificates in your device
   1. First please take a look at the sample that accomplishes your scenario here
   2. You'll notice that you need to include the certificates and private keys that you generated and then took the thumbprint of to create your IotHub device in either plaintext or add a method for ADU to read them into the main.c so they can be added to your device handle like below (from the sample)

(IoTHubDeviceClient_LL_SetOption(device_ll_handle, OPTION_X509_CERT, x509certificate) != IOTHUB_CLIENT_OK) || (IoTHubDeviceClient_LL_SetOption(device_ll_handle, OPTION_X509_PRIVATE_KEY, x509privatekey) != IOTHUB_CLIENT_OK)

3. Within the DU agent we add these kind of additions within this fucntion
4. If you were trying to be fancy I would suggest adding a parameter to the adu-conf.txt configuration file that has the absolute path to your certificate and private key within our GetConnectionInfoFromADUConfigFile() function located here. Now you can add some functionality to read in the certificate and private key into the agent.
5. Then you can just add your connection string, x509CertificatePath, and privateKeyPath to the configuration file to get those values into the agent.
6. Once you are reading those values in you can add new variables to our ADUC_LaunchArguments structure or reuse old ones so you can pass your certificate and private key that you added above to the ADUC_DeviceClientCreate function
7. The final step would to be add the options above to the clientHandle being created in ADUC_DeviceCLientCreate. I wrote the code that should work if you want to add it within the if-else-if structure we have within ADUC_DeviceClientCreate. For the code I assumed you added the new member values of selfSignedCertificateString and selfSignedPrivateKey to the ADUC_LaunchArguments structure for your certificate and private key strings read into the agent above

    else if (
        connInfo->selfSignedCertificateString != NULL 
        && (iothubResult = IoTHubDeviceClient_LL_SetOption(g_iotHubClientHandle,OPTION_X509_CERT,connInfo->selfSignedCertificateString) != IOTHUB_CLIENT_OK))
    {
        Log_Error("Unable to set certificate string for validation");
        result = false;
    }
    else if (
        connInfo->selfSignedPrivateKey != NULL 
        && (iothubResult = IoTHubDeviceClient_LL_SetOption(g_iotHubClientHandle,OPTION_X509_PRIVATE_KEY,connInfo->selfSignedPrivateKey) != IOTHUB_CLIENT_OK))
    {
        Log_Error("Unable to set private key string for validation");
        result = false;
    }

8. You should then be able to build the agent for use on linux platforms and ready to deploy APT updates using ./scripts/build.sh -p linux --content-handlers microsoft/apt --build-packages. Then install the agent using debian, add your variables to the adu-conf.txt and then get right on going.

I want to let you know we're looking into support this out of the box but I don't have a timeline for you. that's why I figured I'd get you the code and steps that will at least get you going right now until we have a more robust solution.

Let me know if you have any questions.

-Nic Hemstreet
Software Engineer, Adu Client

[PradeepKiruvale]

@nihemstr Thanks for the detailed explanation. You are right I am trying to connect my iot device directly to the IoThub without using the IotEdge. My question here is, Is it common practice to connect IoT devices with IotEdge or directly?. Which is better?

Now I have an environment wherein I connect the IoT Device using the x509 self-signed certificate to IoTHub.

[ValOlson]

Hello,

It is great to hear that this is what you were looking for!

Connecting the IoT device with the IoT Identity Service (works with IoTEdge and non-Edge devices) or not depends on your use case. For development and when using a few devices, it’s more convenient to use config file. But for production, it is recommended to use the IoT Identity service for scalability and security.

[GauravChoube]

Hi nihemstr,
I am working same concept where adding DPS feature to adu agent code same way as you explained with little minor changed.
I used x509 certificate type instead of connection string and used the IoTHubDeviceClient_LL_CreateFromDeviceAuth

I got successfully registered with DPS to IOT hub and get connect to hub, update twins properties.
But code get stuck after subscription to topic $iothub/twin/PATCH/properties/desired/#.
I have build the all on pi 4 running on Raspbian os.

Below are low please look and help me out finding root clause.

Nov 29 04:50:03 raspberrypi systemd[1]: Stopped Azure Device Update Agent daemon..
Nov 29 04:50:03 raspberrypi systemd[1]: Started Azure Device Update Agent daemon..
Nov 29 04:50:17 raspberrypi AducIotAgent[4218]: Hi This is Gaurav from agentEnter into test function sucessfully2021-11-29T04:50:03.1160Z [I] Agent (linux; 0.6.0-public-preview) starting. [main]
Nov 29 04:50:17 raspberrypi AducIotAgent[4218]: 2021-11-29T04:50:03.1160Z [I] Git Info: main:743bb72 [main]
Nov 29 04:50:17 raspberrypi AducIotAgent[4218]: 2021-11-29T04:50:03.1160Z [I] Agent built with handlers: microsoft/swupdate. [main]
Nov 29 04:50:17 raspberrypi AducIotAgent[4218]: inside AllocateDeviceClientHandle
Nov 29 04:50:17 raspberrypi AducIotAgent[4218]: Provisioning API Version: 1.7.0
Nov 29 04:50:17 raspberrypi AducIotAgent[4218]: Info: Initiating DPS client to retrieve IoT Hub connection information
Nov 29 04:50:17 raspberrypi AducIotAgent[4218]: Provisioning Status: 0
Nov 29 04:50:17 raspberrypi AducIotAgent[4218]: Provisioning Status: 2
Nov 29 04:50:17 raspberrypi AducIotAgent[4218]: Registration Information received from service: devmgmtiothub.azure-devices.net!
Nov 29 04:50:17 raspberrypi AducIotAgent[4218]: Info: Provisioning callback indicates success.  iothubUri=devmgmtiothub.azure-devices.net, deviceId=dev_iot_001
Nov 29 04:50:17 raspberrypi AducIotAgent[4218]: Info: DPS successfully registered.  Continuing on to creation of IoTHub device client handle.
Nov 29 04:50:17 raspberrypi AducIotAgent[4218]: 2021-11-29T04:50:14.7812Z [I] Initalizing PnP components. [ADUC_PnP_Components_Create]
Nov 29 04:50:17 raspberrypi AducIotAgent[4218]: 2021-11-29T04:50:14.7813Z [I] ADUC agent started. Using IoT Hub Client SDK 1.7.0 [AzureDeviceUpdateCoreInterface_Create]
Nov 29 04:50:17 raspberrypi AducIotAgent[4218]: 2021-11-29T04:50:14.7813Z [I] Calling ADUC_Register [ADUC_MethodCall_Register]
Nov 29 04:50:17 raspberrypi AducIotAgent[4218]: 2021-11-29T04:50:14.7814Z [I] IoTHub Device Twin callback registered. [ADUC_DeviceClient_Create]
Nov 29 04:50:17 raspberrypi AducIotAgent[4218]: 2021-11-29T04:50:14.7819Z [W] Failed to pass connection string to DO, error: -1 [StartupAgent]
Nov 29 04:50:17 raspberrypi AducIotAgent[4218]: 2021-11-29T04:50:14.7819Z [I] Agent running. [main]
Nov 29 04:50:17 raspberrypi AducIotAgent[4218]: -> 04:50:15 CONNECT | VER: 4 | KEEPALIVE: 240 | FLAGS: 128 | USERNAME: devmgmtiothub.azure-devices.net/dev_iot_001/?api-version=2020-09-30&DeviceClientType=iothubclient%2f1.7.0%20(native%3b%20Linux%3b%20armv7l)&model-id=dtmi%3aAzureDeviceUpdate%3b1 | CLEAN: 0
Nov 29 04:50:17 raspberrypi AducIotAgent[4218]: <- 04:50:16 CONNACK | SESSION_PRESENT: true | RETURN_CODE: 0x0
Nov 29 04:50:17 raspberrypi AducIotAgent[4218]: 2021-11-29T04:50:16.2355Z [D] IotHub connection status: 0, reason:6 [ADUC_ConnectionStatus_Callback]
Nov 29 04:50:17 raspberrypi AducIotAgent[4218]: -> 04:50:16 SUBSCRIBE | PACKET_ID: 2 | TOPIC_NAME: $iothub/twin/res/# | QOS: 0
Nov 29 04:50:17 raspberrypi AducIotAgent[4218]: <- 04:50:16 SUBACK | PACKET_ID: 2 | RETURN_CODE: 0
Nov 29 04:50:17 raspberrypi AducIotAgent[4218]: -> 04:50:16 PUBLISH | IS_DUP: false | RETAIN: 0 | QOS: DELIVER_AT_MOST_ONCE | TOPIC_NAME: $iothub/twin/GET/?$rid=3
Nov 29 04:50:17 raspberrypi AducIotAgent[4218]: <- 04:50:17 PUBLISH | IS_DUP: false | RETAIN: 0 | QOS: DELIVER_AT_MOST_ONCE | TOPIC_NAME: $iothub/twin/res/200/?$rid=3 | PAYLOAD_LEN: 537
Nov 29 04:50:17 raspberrypi AducIotAgent[4218]: 2021-11-29T04:50:17.0376Z [I] Processing existing Device Twin data after agent started. [ADUC_PnPDeviceTwin_Callback]
Nov 29 04:50:17 raspberrypi AducIotAgent[4218]: 2021-11-29T04:50:17.0376Z [D] Notifies components that all callback are subscribed. [ADUC_PnPDeviceTwin_Callback]
Nov 29 04:50:17 raspberrypi AducIotAgent[4218]: 2021-11-29T04:50:17.0377Z [I] DeviceInformation component is ready - reporting properties [DeviceInfoInterface_Connected]
Nov 29 04:50:17 raspberrypi AducIotAgent[4218]: 2021-11-29T04:50:17.0379Z [I] Property manufacturer changed to Contoso [RefreshDeviceInfoInterfaceData]
Nov 29 04:50:17 raspberrypi AducIotAgent[4218]: 2021-11-29T04:50:17.0380Z [I] Property model changed to Virtual-Machine [RefreshDeviceInfoInterfaceData]
Nov 29 04:50:17 raspberrypi AducIotAgent[4218]: 2021-11-29T04:50:17.0380Z [I] Property osName changed to Linux [RefreshDeviceInfoInterfaceData]
Nov 29 04:50:17 raspberrypi AducIotAgent[4218]: 2021-11-29T04:50:17.0381Z [I] Property swVersion changed to 5.10.17-v7l- [RefreshDeviceInfoInterfaceData]
Nov 29 04:50:17 raspberrypi AducIotAgent[4218]: 2021-11-29T04:50:17.0382Z [I] Property processorArchitecture changed to armv7l [RefreshDeviceInfoInterfaceData]
Nov 29 04:50:17 raspberrypi AducIotAgent[4218]: 2021-11-29T04:50:17.0628Z [I] Property processorManufacturer changed to ARM [RefreshDeviceInfoInterfaceData]
Nov 29 04:50:17 raspberrypi AducIotAgent[4218]: 2021-11-29T04:50:17.0630Z [I] Property totalMemory changed to 3919128 [RefreshDeviceInfoInterfaceData]
Nov 29 04:50:17 raspberrypi AducIotAgent[4218]: 2021-11-29T04:50:17.0631Z [I] Property totalStorage changed to 2405632 [RefreshDeviceInfoInterfaceData]
Nov 29 04:50:17 raspberrypi AducIotAgent[4218]: 2021-11-29T04:50:17.0633Z [I] No update content. Reporting Idle state. [ADUC_Workflow_HandleStartupWorkflowData]
Nov 29 04:50:17 raspberrypi AducIotAgent[4218]: 2021-11-29T04:50:17.0634Z [I] Setting UpdateState to Idle [ADUC_SetUpdateStateHelper]
Nov 29 04:50:17 raspberrypi AducIotAgent[4218]: 2021-11-29T04:50:17.0634Z [I] Reporting state: 0, Idle (0); HTTP 200; result 1, 0 [AzureDeviceUpdateCoreInterface_ReportStateAndResultAsync]
Nov 29 04:50:17 raspberrypi AducIotAgent[4218]: 2021-11-29T04:50:17.0635Z [I] UpdateAction: Idle. WorkflowId: 211129045017 [ADUC_MethodCall_Idle]
Nov 29 04:50:17 raspberrypi AducIotAgent[4218]: 2021-11-29T04:50:17.0636Z [I] Calling IdleCallback [ADUC_MethodCall_Idle]
Nov 29 04:50:17 raspberrypi AducIotAgent[4218]: 2021-11-29T04:50:17.0636Z [I] Now idle. workflowId: 211129045017 [Idle]
Nov 29 04:50:17 raspberrypi AducIotAgent[4218]: -> 04:50:17 SUBSCRIBE | PACKET_ID: 4 | TOPIC_NAME: $iothub/twin/PATCH/properties/desired/# | QOS: 0

[nihemstr]

Hey @GauravChoube I'm not sure how to debug the issue with the topic. I can ping the service, but it looks like we resolved the initial issue. If you're still working on this implementation, please feel free to create a new issue and I can pass it along to my service side counterparts to take a deeper dive.

Thanks!