Unsubscribe actor events when actorservice is deleted

[PepijnGramberg]

The UnsubscribeAsync for actor events is behaving in a way that is not directly logical when the actor service is deleted.
The following code is used to unsubscribe from an actorevent channel.

 Task DoDisconnect()
 {
    var actor = ActorProxy.Create<TActor>(new ActorId(Id), FabricApplicationName);
    return actor.UnsubscribeAsync(ActorEventsSubscriber);
 }

When this code is called and the ActorService does not exists any more the exception is thrown as shown below. This exception is logical, as it specifically tells us the service does not exist.

System.Fabric.FabricServiceNotFoundException: Service does not exist. ---> System.Runtime.InteropServices.COMException: Exception from HRESULT: 0x80071BCD
   at System.Fabric.Interop.NativeClient.IFabricServiceManagementClient6.EndResolveServicePartition(IFabricAsyncOperationContext context)
   at System.Fabric.FabricClient.ServiceManagementClient.ResolveServicePartitionEndWrapper(IFabricAsyncOperationContext context)
   at System.Fabric.Interop.AsyncCallOutAdapter2`1.Finish(IFabricAsyncOperationContext context, Boolean expectedCompletedSynchronously)
   --- End of inner exception stack trace ---
   at Microsoft.ServiceFabric.Services.Client.ServicePartitionResolver.ResolveHelperAsync(Func`5 resolveFunc, ResolvedServicePartition previousRsp, TimeSpan resolveTimeout, TimeSpan maxRetryInterval, CancellationToken cancellationToken, Uri serviceUri)
   at Microsoft.ServiceFabric.Services.Communication.Client.CommunicationClientFactoryBase`1.GetClientAsync(Uri serviceUri, ServicePartitionKey partitionKey, TargetReplicaSelector targetReplicaSelector, String listenerName, OperationRetrySettings retrySettings, CancellationToken cancellationToken)
   at Microsoft.ServiceFabric.Services.Remoting.V2.FabricTransport.Client.FabricTransportServiceRemotingClientFactory.GetClientAsync(Uri serviceUri, ServicePartitionKey partitionKey, TargetReplicaSelector targetReplicaSelector, String listenerName, OperationRetrySettings retrySettings, CancellationToken cancellationToken)
   at Microsoft.ServiceFabric.Services.Communication.Client.ServicePartitionClient`1.GetCommunicationClientAsync(CancellationToken cancellationToken)
   at Microsoft.ServiceFabric.Services.Communication.Client.ServicePartitionClient`1.InvokeWithRetryAsync[TResult](Func`2 func, CancellationToken cancellationToken, Type[] doNotRetryExceptionTypes)
   at Microsoft.ServiceFabric.Actors.Client.ActorProxy.UnsubscribeAsyncV2(Type eventType, Object subscriber)
   at Microsoft.ServiceFabric.Actors.Client.ActorProxy.UnsubscribeAsync(Type eventType, Object subscriber)

The Disconnect task is being retried on exceptions, as there are some exceptions that are worth retrying. The ActorService is still not existing when the retry is happening (as we have deleted it, it will not be coming back), but the second time there is no exception. It seems as if the SF framework does something to recognise the exception and alters its execution path when the UnsubscribeAsync is tried again. I would expect the same exception to be thrown. Why does this not happen?

Another strange thing is that when the next subscriber tries to unsubscribe the exception is thrown again. Even if it is in the same service/partition/replica, why does this again behave differently than a retry for 1 actorId?