ServiceBusReceiver receive_messages max_wait_time race condition locks the message but sends back empty list #35591

[cabal-daniel]

Description

Package Version: 7.12.2
Operating System: MacOS and Ubuntu
Python Version: 3.8.18

pip install azure-servicebus

We are using the Azure service bus SDK on Python. We have a simple piece of code to receive messages.

Receiving code:

with sb_client.get_queue_receiver(queue) as receiver:
    messages = receiver.receive_messages(
        max_message_count=max_message_count,
        max_wait_time=max_wait_time,
    )
    print(messages)
    for message in messages:
           receiver.complete_message(message)

If max_wait_time is set in ServiceBusReceiver.receive_messages, then a race condition can happen where a message gets locked by the receive_message action but receive_messages returns back an empty list, locking this message for the duration of the lockout period.

To Reproduce
Steps to reproduce the behavior:

We used max_wait_time to 2 and are receiving 1 message at a time
Spin up 10 workers, constantly calling the Receiving code above in a polling interval in a while loop
Have a sender send 100 or so messages to the queue 1 at a time with a decently large body

Actual Behavior

1. Some messages are locked but not received by any workers. i.e. no messages are returned with those values. From our experiments roughly 2-4 messages out of 100 get put into the locked state without them being complete. The receive_messages method shouldn't lock a message unless it can return that message in full.

Expected Behavior

1. All messages should be received by workers and should not be re-tried.

[github-actions]

Thank you for your feedback. Tagging and routing to the team member best able to assist.