[Bug Report] ProvisioningDeviceClient.RegisterAsync: CancellationToken not effective when offline

[its-phil]

Context

• OS, version, SKU and CPU architecture used: Ubuntu 18.04 x86
• Application's .NET Target Framework : net5.0
• SDK version used: Microsoft.Azure.Devices.Provisioning.Transport.Amqp 1.14.1

Description of the issue

Calling ProvisioningDeviceClient.RegisterAsync with a cancellation token ignores the token timeout if the device is offline (tested on an Azure VM where all outbound connections are blocked). The ProvisioningDeviceClient instance is created with a ProvisioningTransportHandlerAmqp instance.

I would expect RegisterAsync to cancel if I set the cancellation token source timeout to e.g. 15 seconds. Instead, the timeout is always 1 minute.

This seems related to this issue and this issue but the workaround mentioned here doesn't seem work with the ProvisioningDeviceClient.

Is there another way to work around this issue? I'd like the RegisterAsync call to fail faster when the device is offline.

Code sample exhibiting the issue

using var transport = new ProvisioningTransportHandlerAmqp(TransportFallbackType.TcpOnly);
var provisioningDeviceClient = ProvisioningDeviceClient.Create(dpsUrl, scopeId, securityProvider, transport);

const int timeoutMs = 15000;
using var cts = new CancellationTokenSource(TimeSpan.FromMilliseconds(timeoutMs));
var sw = new Stopwatch();
try
{
    sw.Start();
    var result = await provisioningDeviceClient.RegisterAsync(cts.Token).ConfigureAwait(false);
    Console.WriteLine(result.Status);
}
catch (Exception ex)
{
    Console.WriteLine(ex.Message);
    Console.WriteLine(ex.InnerException?.Message);
}
finally
{
    sw.Stop();
    Console.WriteLine($"Registration finished or cancelled after {sw.ElapsedMilliseconds} ms.");
    // Here I'd expect ~15 seconds. Instead I always get ~60 seconds.
}

[timtay-microsoft]

As mentioned in #1921 the AMQP library that both the IoT Hub device client and the DPS device client use does not support cancellation tokens, so we have some hardcoded timeout values for scenarios like these. While we are pushing for the AMQP library to add support for cancellation tokens, it may be a while before that is finished.

That said, the real issue here seems to be that the registerAsync call doesn't immediately fail when a network disconnection is detected. That seems like an issue we could investigate now.

[timtay-microsoft]

And we can also look into making these hardcoded timeouts configurable as well

[its-phil]

Thanks! I'd appreciate any solution that allows for customizing the timeouts. What I'd like to achieve is to connect an IoT device to an Azure Edge gateway, but not after trying to connect to the DPS (to update the assigned IoT Hub) and IoT Hub directly (to update desired properties). Should that fail (e.g. because the device and Edge are offline), the device can fall back to a cached IoT Hub and Azure Edge gateway address. That's why - in my case - it doesn't make sense for the device to wait for one minute.

As a matter of fact, it seems that there is the same issue when connecting directly to the IoT Hub (also via AMQP). So, in effect, I have a device patiently waiting for 2 minutes while it could already be connected and running if my cancellation timeouts worked.

For the sake of completeness, here is my code sample of the direct IoT Hub connection: specifying the timeout as TimeSpan doesn't seem to do the trick either in

var authenticationMethod = new DeviceAuthenticationWithRegistrySymmetricKey(deviceId, primary);
using var cts = new CancellationTokenSource(TimeSpan.FromMilliseconds(timeoutMs));
dc = DeviceClient.Create(iotHubAddress, authenticationMethod, TransportType.Amqp_Tcp_Only);
await dc.OpenAsync(cts.Token);

(using Microsoft.Azure.Devices.Client 1.37.2)

[timtay-microsoft]

#2041 will bring the ability to configure the timespan that we pass into the AMQP library, so I'm tagging this as "fix checked in"

For the cancellation token half of this issue, #1921 will be where we track that support.

[timtay-microsoft]

Since we have released timespan support for this API, and since we will be tracking better cancellation token support in #1921, I'm going to close this issue

https://github.com/Azure/azure-iot-sdk-csharp/releases/tag/2021-8-11