The operation did not complete within the allocated time 00:01:00 for object session140

[grik001]

Hi,

I am encountering the below error randomly.

The operation did not complete within the allocated time 00:01:00 for object session140. For more information on exception types and proper exception handling, please refer to http://go.microsoft.com/fwlink/?LinkId=761101

EventProcessorHost error (Action=Close) : System.TimeoutException: The operation did not complete within the allocated time 00:01:00 for object session140. For more information on exception types and proper exception handling, please refer to http://go.microsoft.com/fwlink/?LinkId=761101
at Microsoft.ServiceBus.Common.AsyncResult.End[TAsyncResult](IAsyncResult result)
at Microsoft.ServiceBus.Messaging.Amqp.AmqpObject.EndClose(IAsyncResult result)
at Microsoft.ServiceBus.Messaging.Amqp.AmqpMessagingFactory.CloseEntityAsyncResult.<>c.b__2_1(CloseEntityAsyncResult thisPtr, IAsyncResult r)
at Microsoft.ServiceBus.Messaging.IteratorAsyncResult1.StepCallback(IAsyncResult result) --- End of stack trace from previous location where exception was thrown --- at Microsoft.ServiceBus.Common.AsyncResult.End[TAsyncResult](IAsyncResult result) at Microsoft.ServiceBus.Messaging.Amqp.AmqpMessageReceiver.OnEndClose(IAsyncResult result) at Microsoft.ServiceBus.Messaging.ClientEntity.EndClose(IAsyncResult result) at Microsoft.ServiceBus.Messaging.EventHubReceiver.OnEndClose(IAsyncResult result) at Microsoft.ServiceBus.Messaging.ClientEntity.EndClose(IAsyncResult result) at System.Threading.Tasks.TaskFactory1.FromAsyncCoreLogic(IAsyncResult iar, Func2 endFunction, Action1 endAction, Task1 promise, Boolean requiresSynchronization) --- End of stack trace from previous location where exception was thrown --- at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.ServiceBus.Common.TaskHelpers.EndAsyncResult(IAsyncResult asyncResult) at Microsoft.ServiceBus.Messaging.IteratorAsyncResult1.<>c.b__24_1(TIteratorAsyncResult thisPtr, IAsyncResult r)
at Microsoft.ServiceBus.Messaging.IteratorAsyncResult`1.StepCallback(IAsyncResult result)

System.TimeoutException:
at Microsoft.ServiceBus.Common.AsyncResult.End (Microsoft.ServiceBus, Version=3.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35)
at Microsoft.ServiceBus.Messaging.Amqp.AmqpObject.EndClose (Microsoft.ServiceBus, Version=3.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35)
at Microsoft.ServiceBus.Messaging.Amqp.AmqpMessagingFactory+CloseEntityAsyncResult+<>c.b__2_1 (Microsoft.ServiceBus, Version=3.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35)
at Microsoft.ServiceBus.Messaging.IteratorAsyncResult1.StepCallback (Microsoft.ServiceBus, Version=3.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35) at Microsoft.ServiceBus.Common.AsyncResult.End (Microsoft.ServiceBus, Version=3.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35) at Microsoft.ServiceBus.Messaging.Amqp.AmqpMessageReceiver.OnEndClose (Microsoft.ServiceBus, Version=3.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35) at Microsoft.ServiceBus.Messaging.ClientEntity.EndClose (Microsoft.ServiceBus, Version=3.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35) at Microsoft.ServiceBus.Messaging.EventHubReceiver.OnEndClose (Microsoft.ServiceBus, Version=3.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35) at Microsoft.ServiceBus.Messaging.ClientEntity.EndClose (Microsoft.ServiceBus, Version=3.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35) at System.Threading.Tasks.TaskFactory1.FromAsyncCoreLogic (mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089)
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089)
at Microsoft.ServiceBus.Common.TaskHelpers.EndAsyncResult (Microsoft.ServiceBus, Version=3.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35)
at Microsoft.ServiceBus.Messaging.IteratorAsyncResult1+<>c.<CallTask>b__24_1 (Microsoft.ServiceBus, Version=3.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35) at Microsoft.ServiceBus.Messaging.IteratorAsyncResult1.StepCallback (Microsoft.ServiceBus, Version=3.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35)

[tohling]

Referencing the link http://go.microsoft.com/fwlink/?LinkId=761101 provided in the error message, for TimeoutExceptions, the suggestion is to incorporate retries, which should already be happening. The log entries showing these exceptions should be transient and your Function should recover and continue execution.

Are you Function(s) not recovering from these errors?

[grik001]

Hi tohling,

Thanks for your response, I have not yet confirmed if the Function is failing to recover. My issue is that is is causing a lot of noise in the AppInsights, should this be conisdered as a warning rather than an error?

[tohling]

@grik001, thanks for the quick response. Yes, we will look into downgrading these errors as warnings. I have opened an issue to track this.

I will close this issue for now. Please feel free to re-open or create a new issue if these your Function is not recovering from these errors.

[grik001]

Great, thanks :)

[rohancme]

@tohling we're seeing the same messages and it looks like at least some of these messages are being dropped.
I'm happy to provide any additional logs/details you need!

[tohling]

@rchakra3, seeing these transient error messages may not be the root cause of why your messages are being dropped. Could you kindly open a new GitHub issue with more details so that we can investigate. Thanks!