Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cosmos DB: OrderBy on nullable property, with mixed null and non-null entries, throws exception #18722

Closed
JonPSmith opened this issue Nov 2, 2019 · 5 comments

Comments

@JonPSmith
Copy link

@JonPSmith JonPSmith commented Nov 2, 2019

If I have a nullable property in a class mapped to a Cosmos database, and the database contains a mixture of null and non-null values then when I use an OrderBy I get an exception.

NOTE: Where clause works OK with mixed null/non-null data.

Steps to reproduce

Entity class

public class CosmosBook
{
    public int CosmosBookId { get; set; }
    public string Title { get; set; }
    public double Price { get; set; }
    public DateTime PublishedDate { get; set; }

    //Used for checking on null handling
    public int? NullableInt { get; set; }

    //----------------------------------
    //relationships 

    public ICollection<CosmosReview> Reviews { get; set; }
}

DbContext

public class CosmosDbContext : DbContext
{
    public DbSet<CosmosBook> Books { get; set; }

    public CosmosDbContext(DbContextOptions<CosmosDbContext> options)
        : base(options) { }
}

Unit tests - both int? and string

[Fact]
public async Task TestNullableIntOrderByOk()
{
    //SETUP
    var options = this.GetCosmosDbToEmulatorOptions<CosmosDbContext>();
    using (var context = new CosmosDbContext(options))
    {
        await context.Database.EnsureDeletedAsync();
        await context.Database.EnsureCreatedAsync();

        var cBook1 = new CosmosBook { CosmosBookId = 1, NullableInt = null };
        var cBook2 = new CosmosBook { CosmosBookId = 2, NullableInt = 1 };
        context.AddRange(cBook1, cBook2);
        await context.SaveChangesAsync();
    }
    using (var context = new CosmosDbContext(options))
    {
        //ATTEMPT
        var ex = await Assert.ThrowsAsync<NotSupportedException>(async ()
            => await context.Books.OrderBy(x => x.NullableInt).ToListAsync());

        //VERIFY
        ex.Message.ShouldStartWith("Cannot execute cross partition order-by queries on mix types. " +
                                   "Consider using IS_STRING/IS_NUMBER to get around this exception.");
    }
}

[Fact]
public async Task TestStringWithNullOrderByOk()
{
    //SETUP
    var options = this.GetCosmosDbToEmulatorOptions<CosmosDbContext>();
    using (var context = new CosmosDbContext(options))
    {
        await context.Database.EnsureDeletedAsync();
        await context.Database.EnsureCreatedAsync();

        var cBook1 = new CosmosBook { CosmosBookId = 1,  };
        var cBook2 = new CosmosBook { CosmosBookId = 2, Title = "Book2"};
        context.AddRange(cBook1, cBook2);
        await context.SaveChangesAsync();
    }
    using (var context = new CosmosDbContext(options))
    {
        //ATTEMPT
        var ex = await Assert.ThrowsAsync<NotSupportedException>(async () =>
            await context.Books.OrderBy(x => x.Title).ToListAsync());

        //VERIFY
        ex.Message.ShouldStartWith("Cannot execute cross partition order-by queries on mix types. " +
                                   "Consider using IS_STRING/IS_NUMBER to get around this exception.");
    }
}

The exception for the string example is

System.NotSupportedException : Cannot execute cross partition order-by queries on mix types. Consider using IS_STRING/IS_NUMBER to get around this exception. Expect type: Null. Actual type: String. Item value: Microsoft.Azure.Cosmos.Query.OrderByItem.
   at Microsoft.Azure.Cosmos.Query.ParallelQuery.OrderByConsumeComparer.CheckTypeMatching(IList`1 items1, IList`1 items2)
   at Microsoft.Azure.Cosmos.Query.ParallelQuery.OrderByConsumeComparer.CompareOrderByItems(IList`1 items1, IList`1 items2)
   at Microsoft.Azure.Cosmos.Query.CosmosOrderByItemQueryExecutionContext.OrderByEqualityComparer.Equals(CosmosElement x, CosmosElement y)
   at Microsoft.Azure.Cosmos.Query.ItemProducer.MoveNextAsync(CancellationToken token)
   at Microsoft.Azure.Cosmos.Query.ItemProducerTree.TryMoveNextAsyncImplementationAsync(CancellationToken token)
   at Microsoft.Azure.Cosmos.Query.ItemProducerTree.ExecuteWithSplitProofingAsync(Func`2 function, Boolean functionNeedsBeReexecuted, CancellationToken cancellationToken)
   at Microsoft.Azure.Cosmos.Query.ItemProducerTree.MoveNextAsync(CancellationToken token)
   at Microsoft.Azure.Cosmos.Query.CosmosCrossPartitionQueryExecutionContext.MoveNextHelperAsync(ItemProducerTree itemProducerTree, CancellationToken cancellationToken)
   at Microsoft.Azure.Cosmos.Query.CosmosOrderByItemQueryExecutionContext.InternalDrainAsync(Int32 maxElements, CancellationToken cancellationToken)
   at Microsoft.Azure.Cosmos.Query.CosmosCrossPartitionQueryExecutionContext.DrainAsync(Int32 maxElements, CancellationToken cancellationToken)
   at Microsoft.Azure.Cosmos.Query.PipelinedDocumentQueryExecutionContext.ExecuteNextAsync(CancellationToken token)
   at Microsoft.Azure.Cosmos.Query.CosmosQueryExecutionContextFactory.ExecuteNextHelperAsync(CancellationToken cancellationToken)
   at Microsoft.Azure.Cosmos.Query.CosmosQueryExecutionContextFactory.ReadNextAsync(CancellationToken cancellationToken)
   at Microsoft.EntityFrameworkCore.Cosmos.Storage.Internal.CosmosClientWrapper.DocumentAsyncEnumerable.AsyncEnumerator.MoveNextAsync()
   at Microsoft.EntityFrameworkCore.Cosmos.Query.Internal.CosmosShapedQueryCompilingExpressionVisitor.AsyncQueryingEnumerable`1.AsyncEnumerator.MoveNextAsync()
   at Microsoft.EntityFrameworkCore.EntityFrameworkQueryableExtensions.ToListAsync[TSource](IQueryable`1 source, CancellationToken cancellationToken)
   at Microsoft.EntityFrameworkCore.EntityFrameworkQueryableExtensions.ToListAsync[TSource](IQueryable`1 source, CancellationToken cancellationToken)
   at Test.UnitTests.ExploreCosmosDb.TestCosmosBasics.TestStringWithNullOrderByOk() in C:\Users\Jon\Source\Repos\EfCoreSqlAndCosmos\Test\UnitTests\ExploreCosmosDb\TestCosmosBasics.cs:line 168
--- End of stack trace from previous location where exception was thrown ---

Further technical details

EF Core version: 3.0
Database provider: Microsoft.EntityFrameworkCore.Cosmos
Target framework: .NET Core 3.0
Operating system: Windows
IDE: Visual Studio 2019 16.3

@JonPSmith JonPSmith added the type-bug label Nov 2, 2019
@AndriySvyryd

This comment has been minimized.

Copy link
Member

@AndriySvyryd AndriySvyryd commented Nov 4, 2019

@JonPSmith We expected it to not return the items with null values, but it should not throw. What is the exception that you get?

@JonPSmith

This comment has been minimized.

Copy link
Author

@JonPSmith JonPSmith commented Nov 4, 2019

Hi @AndriySvyryd,

Sorry for the blank bug entry. I did put the explanation in there but I must and moved away before saving it. I have the unit tests so it was easy to redo.

@AndriySvyryd

This comment has been minimized.

Copy link
Member

@AndriySvyryd AndriySvyryd commented Nov 4, 2019

This looks like an external issue since you are not using different partitions (EF Core sends a null partition key value if none is specified), so at least the exception message in incorrect.
@j82w Is this a known issue?

@AndriySvyryd

This comment has been minimized.

Copy link
Member

@AndriySvyryd AndriySvyryd commented Nov 4, 2019

@JonPSmith You can work around this by filtering out nulls before the orderby or by mapping an explicit partition key and adding a partition filter to the query.

@AndriySvyryd

This comment has been minimized.

Copy link
Member

@AndriySvyryd AndriySvyryd commented Nov 5, 2019

Found the Cosmos issue: Azure/azure-cosmos-dotnet-v3#733
It might have been fixed in 3.4.0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.