Skip to content

Support hierarchical partition keys for Azure Cosmos #9513

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
May 29, 2025

Conversation

Kumima
Copy link
Contributor

@Kumima Kumima commented May 26, 2025

Description

Support hierarchical partition keys for Azure Cosmos and make both AzureInfrastructure and emulator respect the hierarchical partition keys.

As @davidfowl suggested, this won't introduce any break changes. A nullable property which is PartitionKeyPaths has been introduced, and when initializes a new instance of the AzureCosmosDBContainerResource, the first key will be used as the required parameter.

Similar with the Azure Cosmos SDK, only the PartitionKeyPaths will provide the partition key info.

The way to determine whether to use hierarchical partition keys to create the container is to check whether PartitionKeyPaths is null.

If there are further tests need to be added, let me know😊. Another thing I'm not sure is whether we should check the partition keys count, it should be <= 3.

Fixes #9462

Checklist

  • Is this feature complete?
    • Yes. Ready to ship.
    • No. Follow-up changes expected.
  • Are you including unit tests for the changes and scenario tests if relevant?
    • Yes
    • No
  • Did you add public API?
    • Yes
      • If yes, did you have an API Review for it?
        • Yes
        • No
      • Did you add <remarks /> and <code /> elements on your triple slash comments?
        • Yes
        • No
    • No
  • Does the change make any security assumptions or guarantees?
    • Yes
      • If yes, have you done a threat model and had a security review?
        • Yes
        • [] No
    • No
  • Does the change require an update in our Aspire docs?

@github-actions github-actions bot added the area-integrations Issues pertaining to Aspire Integrations packages label May 26, 2025
@dotnet-policy-service dotnet-policy-service bot added the community-contribution Indicates that the PR has been added by a community member label May 26, 2025
Copy link
Member

@davidfowl davidfowl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Kumima can update the cosmos playground example to use this feature?

@dotnet-policy-service dotnet-policy-service bot added the needs-author-action An issue or pull request that requires more info or actions from the author. label May 28, 2025
@dotnet-policy-service dotnet-policy-service bot removed the needs-author-action An issue or pull request that requires more info or actions from the author. label May 28, 2025
@Kumima Kumima requested a review from davidfowl May 28, 2025 08:51
// Use the resource name as the container name if it's not provided
containerName ??= name;

var container = new AzureCosmosDBContainerResource(name, containerName, partitionKeyPathsArray[0], builder.Resource)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sebastienros @eerhardt thoughts here? This is the only sketchy part of this change. If there are many this property will only show the first one.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could also obsolete this ctor and expose one with the keys. Then add a unit test to have coverage on both even if one is deprecated.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we follow this, if hierarchical, single getter will throw. Or we can introduce a private/public ContainerProperties from AzureCosmosSdk to host the both type keys, so that we don't need to duplicated checking from AzureCosmosSdk.
https://github.com/Azure/azure-cosmos-dotnet-v3/blob/5e6232d25f84074d5008e97670d3e0dc07891f0e/Microsoft.Azure.Cosmos/src/Resource/Settings/ContainerProperties.cs#L408-L419

With public API of ContainerProperties seems bring more flexibility. Currently we can only custom the partition keys, not all those properties. And if this, we may also need to map all those properties for AzureProvisioning.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I prefer @sebastienros's suggestion, it's simpler.

Copy link
Contributor Author

@Kumima Kumima May 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@davidfowl, please check the latest commits, I'm not sure whether those argument checking can be simplified. Are they a bit of duplicated?
And for:

[Obsolete("Use the overload that supports the hierarchical partition keys instead.")]
public AzureCosmosDBContainerResource(string name, string containerName, string partitionKeyPath, AzureCosmosDBDatabaseResource parent)
    : this(name, containerName, [partitionKeyPath], parent) { }

The argument exception comes from the other ctor. So given partitionKeyPath is null, we got partitionKeyPaths argument exception. Are we going to duplicated ctor logic here again?

@davidfowl davidfowl self-requested a review May 28, 2025 18:24
// Use the resource name as the container name if it's not provided
containerName ??= name;

var container = new AzureCosmosDBContainerResource(name, containerName, partitionKeyPathsArray[0], builder.Resource)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could also obsolete this ctor and expose one with the keys. Then add a unit test to have coverage on both even if one is deprecated.

@Kumima Kumima requested a review from sebastienros May 29, 2025 02:43
Copy link
Member

@davidfowl davidfowl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

@Kumima
Copy link
Contributor Author

Kumima commented May 29, 2025

Looks good!

Can you take a look at the failed Linux test?

@davidfowl davidfowl enabled auto-merge (squash) May 29, 2025 05:32
@davidfowl davidfowl merged commit 9e4ed4f into dotnet:main May 29, 2025
500 of 502 checks passed
@davidfowl
Copy link
Member

Thanks @Kumima !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area-integrations Issues pertaining to Aspire Integrations packages community-contribution Indicates that the PR has been added by a community member
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support Hierarchical Partition Keys for Azure Cosmos
4 participants