Load balancing for event processor failed

[lindsve]

Query/Question
We keep seeing these errors in our logs in our subscriber modules: Load balancing for event processor failed - Did not observe any item or terminal signal within 60000ms in 'filter' (and no fallback has been configured) Did not observe any item or terminal signal within 60000ms in 'filter' (and no fallback has been configured).

It can be several hours between the occasions when these are logged.

This is the code that sets up our event processor clients:

eventProcessorClient = new EventProcessorClientBuilder()
                .connectionString(eventHubConnectionString, getChannelName())
                .processEvent(this::ProcessEvent)
                .processError(this::ProcessError)
                .consumerGroup(consumerGroup)
                .checkpointStore(new BlobCheckpointStore(blobContainerAsyncClient))
                .buildEventProcessorClient();

Any idea why these messages are logged?

Why is this not a Bug or a feature Request?
Because I am not definitely sure that it is a bug or if it is a missing setup on our end. I have not been successful in finding anything related to this otherwise online either.

Setup (please complete the following information if applicable):

• OS: MacOS 10.15 / Alpine Linux (openjdk:8-jdk-alpine docker image)
• IDE : IntelliJ IDEA Ultimate 2020.1
• Version of the Library used

<dependency>
  <groupId>com.azure</groupId>
  <artifactId>azure-messaging-eventhubs</artifactId>
  <version>5.1.1</version>
</dependency>
<dependency>
  <groupId>com.azure</groupId>
  <artifactId>azure-messaging-eventhubs-checkpointstore-blob</artifactId>
  <version>1.1.1</version>
</dependency>

Information Checklist
Kindly make sure that you have added all the following information above and checkoff the required fields otherwise we will treat the issuer as an incomplete report

• Query Added
• Setup information Added

[srnagar]

@lindsve Does this happen all the time or occasionally?

If it's happening occasionally, it might be due to a network call to blob storage taking longer than a minute to complete (60000 ms shown in the error message). This can be ignored as this is a transient error and the next cycle of load balancing will retry and resume normal operations. The error is still provided as a notification to the user for any bookkeeping purposes.

If this is happening all the time and the processor is not consuming any events then it's possible that there might be an issue connecting to the storage service and there might be several reasons for that - most commonly the proxy setup.

Please let us know what you observe and if possible provide more details with logs and sample to reproduce this issue.

[lindsve]

It seems to happen about once per hour at most, and there can be several hours between them. Seems like we can ignore these if they are transient and it seems it resumes as normal afterwards.

That being said, it could be useful if the ErrorContext could provide a bit more info if is an error or a warning that is thrown since we are picking up this via the processError handler.

[srnagar]

Thanks for the confirmation and for your feedback!

Adding info about error or warning in ErrorContext has been considered but given the wide variety of errors that could occur, accurately categorizing errors into warnings and errors may not always be feasible.

However, we will consider looking at the most common errors and try to surface them in ErrorContext as warnings. Thanks again for the feedback.

Resolving the issue.