[BUG] Paging through a considerable amount of queues in a namespace results in inconsistent page and total item count results

[sergevm]

Library name and version

Azure.Messaging.ServiceBus, Version=7.17.1.0

Describe the bug

I have a small service bus browser tool https://github.com/sergevm/azure-servicebus-browser that has some search features (contains on queue / topic name). To execute this search, I use ServicebusAdministrationClient.GetQueuesRuntimePropertiesAsync() with AsPages() to run through the the queue runtime informations and find matches.

However, I noticed that I don't always get the same number of search results. I added some logging to the method that processes the IAsyncEnumerable<Page>. With the page size set to 100, I see in the logging that all pages except for the last one contain 100 queues, except for the last page. In the logging, I see that the number of pages is not consistent, as are the number of queues on the last page received.

Expected behavior

Without queues being created or deleted, the number of pages is always the same, as well as the total number of items.

Actual behavior

There are no exceptions thrown, nor logging statements.

Reproduction Steps

Use a namespace that contains a considerable amount of queues. In my test context, I have +1900 queues.

Use the Azure.ServiceBus.Messaging ServicebusAdministration client to run over the queues in the namespace, using the ServicebusAdministrationClient.GetQueuesRuntimePropertiesAsync() API, and call AsPages() on the result, e.g. ` return client.GetQueuesRuntimePropertiesAsync(cancellationToken).AsPages(continuationToken, pageSize);

The implementation I use is as simple as

public IAsyncEnumerable<Azure.Page<QueueRuntimeProperties>> GetQueueRuntimeInfo(int pageSize, string? continuationToken, CancellationToken cancellationToken = default)
{
       return client.GetQueuesRuntimePropertiesAsync(cancellationToken).AsPages(continuationToken, pageSize);
}

This method is called from another method (on another class):

public async IAsyncEnumerable<IEnumerable<QueueViewModel>> GetQueues(string filter, int maxItems = 100, [EnumeratorCancellation] CancellationToken cancellationToken = default)
{
    var count = 0;
    var pageIndex = 0;
    var totalQueueCount = 0;

    var info = client.GetQueueRuntimeInfo(maxItems, null, cancellationToken);   
    await foreach (var page in info)
    {
        var itemCount = page.Values.Count;
        totalQueueCount += page.Values.Count;
        var matches = page.Values.Where(v => v.Name.Contains(filter, StringComparison.InvariantCultureIgnoreCase)).ToList();
        var matchesCount = matches.Count;
        count += matches.Count;

        logger.Information("Page {PageIndex} contains {ItemCount} items, {MatchesCount} matches, continuation token is {ContinuationToken}", pageIndex++, itemCount, matchesCount, page.ContinuationToken);

        yield return matches.Select(runtimeProperties => new QueueViewModel(namespaceContext, new QueueRuntimePropertiesWrapper(runtimeProperties)));

        if (count >= maxItems)
        {
            break;
        }
    }
    
    logger.Information("Total queue count that has been evaluated against filter: {TotalQueueCount}", totalQueueCount);
}

Run this multiple times, and see the total item count vary, as well as the page count. Note that all but the last page contain the page size worth of items

Environment

Installation Source: SDK 8.0.100
Manifest Version: 8.0.0-preview.1.23557.2/8.0.100
Manifest Path: /usr/local/share/dotnet/sdk-manifests/8.0.100/microsoft.net.sdk.aspire/8.0.0-preview.1.23557.2/WorkloadManifest.json
Install Type: FileBased

Host:
Version: 8.0.0
Architecture: arm64
Commit: 5535e31a71

.NET SDKs installed:
6.0.400 [/usr/local/share/dotnet/sdk]
6.0.412 [/usr/local/share/dotnet/sdk]
6.0.413 [/usr/local/share/dotnet/sdk]
7.0.304 [/usr/local/share/dotnet/sdk]
7.0.305 [/usr/local/share/dotnet/sdk]
7.0.306 [/usr/local/share/dotnet/sdk]
7.0.307 [/usr/local/share/dotnet/sdk]
8.0.100-rc.1.23463.5 [/usr/local/share/dotnet/sdk]
8.0.100-rc.2.23502.2 [/usr/local/share/dotnet/sdk]
8.0.100 [/usr/local/share/dotnet/sdk]

.NET runtimes installed:
Microsoft.AspNetCore.App 6.0.8 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 6.0.20 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 6.0.21 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 7.0.7 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 7.0.8 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 7.0.9 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 7.0.10 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 8.0.0-rc.2.23480.2 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 8.0.0 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
Microsoft.NETCore.App 6.0.8 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
Microsoft.NETCore.App 6.0.20 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
Microsoft.NETCore.App 6.0.21 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
Microsoft.NETCore.App 7.0.7 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
Microsoft.NETCore.App 7.0.8 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
Microsoft.NETCore.App 7.0.9 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
Microsoft.NETCore.App 7.0.10 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
Microsoft.NETCore.App 8.0.0-rc.2.23479.6 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
Microsoft.NETCore.App 8.0.0 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]

Other architectures found:
x64 [/usr/local/share/dotnet/x64]

Environment variables:
Not set

[jsquire]

Hi @sergevm. Thank you for reaching out and we regret that you're experiencing difficulties. The client library does not control what data is returned nor how it is broken into pages; that is done by the Service Bus administration service. Unfortunately, the Azure SDK maintainers are unable to assist with service issues.

Your best path forward would be to open an Azure support request. As an alternative, you could consider posting to the Service Bus forum of the Azure Feedback site, which is monitored by the Service Bus team.

I'm going to close this out; if I've misunderstood what you're describing, please let us know in a comment and we'd be happy to assist as we're able.

[sergevm]

In predecessors of this package, there were options to use a management client to filter on subscriptions, topics etc server-side, see e.g.
https://learn.microsoft.com/en-us/dotnet/api/microsoft.servicebus.namespacemanager.gettopicsasync?view=azure-dotnet-legacy#microsoft-servicebus-namespacemanager-gettopicsasync(system-string).

This feature seems to have disappeared in the supported package, therefore, I am filtering client-side.

Any chance this feature would be introduced again in the library?

Or should I fall back to the rest API in this case?