Azure Go SDK Service Bus / Event Hubs idle connection bug possibly impacting Keda

[berndverst]

Report

This is a heads-up that a bug was identified in github.com/azure/go-amqp, the shared library for the Azure SDK for Go Service Bus and Event Hubs SDKs (Keda only uses the Service Bus SDK from here it seems).
I'm a Dapr maintainer and we have observed this issue. This bug occurs when a connection has been idle (server idle timeout is 10 minutes) and then an attempt is made by the SDK to close the connection. This closing operation can take 30 seconds. In the case of the azservicebus SDK an attempt to send a message (for example) after a connection has become idle and therefore is closed (and recreated) unnecessarily stalls for 30 seconds.

Depending on the frequency in peek operations of the Keda scaler this might impact Keda! For that reason I'm letting you know of this issue. If this impacts Keda most likely the behavior would be the following:
If a topic or queue has become idle (no peek within 10 minutes), the next peek operation likely will force the connection to close - and this closing operation will take 30 seconds before the connection is recreated and peek is performed. This could potentially mean that a scale from zero would occur 30 seconds later than expected.
However, if the peek frequency of the Keda scaler is a lot more frequent, I believe that would prevent the queue from becoming idle and as a result this problem would not be encountered.

On Monday https://github.com/Azure/go-amqp/releases/tag/v1.0.5 will be released to address this, and on Tuesday https://github.com/Azure/azure-sdk-for-go/tree/sdk/messaging/azservicebus/v1.6.1 (or maybe the version number will be different).

I would encourage Keda to upgrade the Service Bus scaler with these library versions.

Expected Behavior

N/A

Actual Behavior

N/A

Steps to Reproduce the Problem

Not tested / Observed.

Logs from KEDA operator

N/A

KEDA Version

2.13.1

Kubernetes Version

Other

Platform

Microsoft Azure

Scaler Details

Azure Service Bus

Anything else?

No response

[zroubalik]

@berndverst thanks a ton for the information!

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.

[stale]

This issue has been automatically closed due to inactivity.