Feed iterator enters an infinite loop after a physical partition split has occurred

[alesk-kontent]

Describe the bug

When I run a targeted multi-partition query with a partition key that includes the first level of subpartitioning, the FeedIterator<T>.ReadNextAsync method never completes after a physical partition split has occurred. The iterator enters an infinite loop reading partition key ranges and the application eventually runs out of memory.

To Reproduce

I wrote a console application to reproduce the issue. Please download the solution, open the Program.cs file and specify a connection string to an Azure Cosmos DB for NoSQL account. The application will create a new database with one container and one physical partition. It will then upsert about 20 GB of data and attempt to read the first 10 documents from two logical partitions. At this point there should be two physical partitions. You can run the application multiple times. It will not upsert data if the container already exists.

The data consists of 40,000 documents (articles) with the following properties:

• Id : string
• CustomerId : Guid
• SectionId : Guid
• Content : string

There are two customers. Each customer has 20,000 articles organized in 2,000 sections (10 articles per section).

The container uses a hierarchical partition key /customerId, /sectionId, so there are 4,000 hierarchical logical partitions with 10 documents.

Finally, the application will execute a query SELECT * FROM c OFFSET 0 LIMIT 10, using the customer ID as the partition key in the request options. When a response is returned, it displays the number of documents and the total request charge.

Expected behavior

The FeedIterator<T>.ReadNextAsync method returns a response with the first 10 documents.

Actual behavior

The FeedIterator<T>.ReadNextAsync method never completes and the application eventually runs out of memory.

Environment summary

SDK Version: 3.38.1 (version 3.37.1 has the same issue)
OS Version: Windows 11 Enterprise (10.0.22631 Build 22631)

Additional context

You can examine the trace and the diagnostic string from another incident. It happens in both Direct and Gateway mode.

[NaluTripician]

This is a known issue with the hierarchical pk feature. There is a temporary fix that should solve the issue for now until a version of the SDK is released with the fix.

Rather than having the PK included in the query request options, filtering on top level hierarchical Pks should be done through where clauses.

CURRENT QUERY THAT HANGS:

var queryDefinition = new QueryDefinition(
    $"SELECT * FROM c ");
 
var partitionKey = new PartitionKeyBuilder().Add(topLevelPkValue).Build();
var requestOptions = new QueryRequestOptions();

requestOptions.PartitionKey = pk;


List<JObject> results = new List<JObject>();
using var iterator = container.GetItemQueryIterator<JObject>(queryDefinition, requestOptions: requestOptions);

while (iterator.HasMoreResults)
{
    var response = await iterator.ReadNextAsync();
}

FIX:

var queryDefinition = new QueryDefinition(
    $"SELECT * FROM c "
    + $"WHERE c.topLevelPkValue = @pk1")
     .WithParameter("@pk1", topLevelPkValue);

using var iterator = container.GetItemQueryIterator<JObject>(queryDefinition);

while (iterator.HasMoreResults)
{
    var response = await iterator.ReadNextAsync();
}

[vit-svoboda]

Would've been lovely to be able to see a known issue such as this one in the known issues list.