ls: push max limit down with SELECT TOP when no client-side filter#70
Merged
Conversation
Fixes #66. Previously 'ls' inside a container always issued an unbounded 'SELECT * FROM c' and capped the result client-side via QueryRequestOptions.MaxItemCount (which is per-page, not a total cap) plus a manual loop break. When no client-side substring filter is in play (no 'filter' argument or '*' wildcard), the SDK now sends 'SELECT TOP <n> * FROM c' so the server stops scanning once the requested row count has been produced. With a substring filter we still fall back to 'SELECT * FROM c' because the match is applied in the shell against the partition or custom key and capping rows server-side could silently drop matches. Added unit tests covering the helper across all branches.
Contributor
There was a problem hiding this comment.
Pull request overview
Updates the ls container-items listing behavior to apply the effective max item limit server-side (via SELECT TOP n) when doing so is safe, reducing unnecessary paging/scan work in Cosmos DB.
Changes:
- Introduces
ListCommand.BuildItemQueryText(...)to choose betweenSELECT *andSELECT TOP n *based on limit/filter conditions. - Updates
ListContainerItemsAsyncto use the new query builder for item listing. - Adds unit tests covering all branches of the query builder logic.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| CosmosDBShell/Azure.Data.Cosmos.Shell.Commands/ListCommand.cs | Adds query-text helper and switches item listing to use SELECT TOP n when no client-side filter applies. |
| CosmosDBShell.Tests/CommandTests/ListCommandTests.cs | Adds unit tests validating query-text selection across limit/filter combinations. |
Treat '*' as no client-side filter in ExecuteAsync, matching the TOP pushdown helper's assumptions. Preserve the limit-reached message for the server-side TOP path when the requested cap is hit even if the iterator has no continuation token. Add unit tests for filter classification and limit-message behavior.
sevoku
approved these changes
May 4, 2026
…t-top # Conflicts: # CosmosDBShell.Tests/CommandTests/ListCommandTests.cs
sevoku
approved these changes
May 4, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #66.
Background
ListCommand.ListContainerItemsAsync issued an unbounded
SELECT * FROM cquery and relied on:QueryRequestOptions.MaxItemCount(per-page hint, not a total cap).breakonce the client-side accumulator reached the user's-mlimit.The server kept producing pages even though the shell only ever consumed up to
effectiveMaxItemCountrows. The issue asks us to considerSELECT TOP <n> * FROM cso the cap is applied server-side.Change
Introduced
ListCommand.BuildItemQueryText(int? effectiveMaxItemCount, string? filter):SELECT TOP <n> * FROM cwhen there is a finite limit and no client-side filter that could discard rows. The shell treats both no-filter and the*wildcard as 'match everything', so both are safe to push down.SELECT * FROM cotherwise, because the substring filter is applied in the shell against the partition or custom key. Capping rows server-side in that case would silently drop matching items.QueryRequestOptions.MaxItemCountis still set as the page-size hint in both paths, and the existing client-side break is unchanged.Tests
New unit tests in CosmosDBShell.Tests/CommandTests/ListCommandTests.cs cover every branch of the helper:
SELECT TOP n*wildcard →SELECT TOP nSELECT TOP nExisting emulator-backed integration tests (
DataOperationTests,QueryCommandTests,EndToEndScriptTests) continue to exerciselsend-to-end and pass locally.