Skip to content

Latest commit

 

History

History
210 lines (147 loc) · 7.1 KB

paging.dotnet.markdown

File metadata and controls

210 lines (147 loc) · 7.1 KB
Raw

Page Query Results

{NOTE: }

  • Paging:
    Paging is the process of fetching a subset (a page) of results from a dataset, rather than retrieving the entire results at once. This method enables processing query results one page at a time.

  • Default page size:

    • Querying Lucene indexes:
      If the client's query definition does Not explicitly specify the page size, the server will default to int.MaxValue (2,147,483,647). In such case, all results will be returned in a single server call.

    • Querying Corax indexes:
      The default page size is the same as the one employed by Lucene.
      Note: when using Corax as the search engine, indexes with more than int.MaxValue entries can be created and used. To match this capacity, queries over Corax indexes can skip a number of results that exceed this max value and take documents from that location.

  • Performance:
    Using paging is beneficial when handling large result datasets, contributing to improved performance.
    See paging and performance here below.

  • In this page:

{NOTE/}

{PANEL: No paging example}

{CODE-TABS} {CODE-TAB:csharp:Query paging_0_1@Indexes\Querying\Paging.cs /} {CODE-TAB:csharp:Query_async paging_0_2@Indexes\Querying\Paging.cs /} {CODE-TAB:csharp:DocumentQuery paging_0_3@Indexes\Querying\Paging.cs /} {CODE-TAB:csharp:Index index_0@Indexes\Querying\Paging.cs /} {CODE-TAB-BLOCK:sql:RQL} from index "Products/ByUnitsInStock" where UnitsInStock > 10 {CODE-TAB-BLOCK/} {CODE-TABS/}

{PANEL/}

{PANEL: Paging examples}

{NOTE: }

Retrieve a specific page:

{CODE-TABS} {CODE-TAB:csharp:Query paging_1_1@Indexes\Querying\Paging.cs /} {CODE-TAB:csharp:Query_async paging_1_2@Indexes\Querying\Paging.cs /} {CODE-TAB:csharp:DocumentQuery paging_1_3@Indexes\Querying\Paging.cs /} {CODE-TAB:csharp:Index index_0@Indexes\Querying\Paging.cs /} {CODE-TAB-BLOCK:sql:RQL} from index "Products/ByUnitsInStock" where UnitsInStock > 10 limit 20, 10 // skip 20, take 10 {CODE-TAB-BLOCK/} {CODE-TABS/}

{NOTE/}

{NOTE: }

Page through all results:

{CODE-TABS} {CODE-TAB:csharp:Query paging_2_1@Indexes\Querying\Paging.cs /} {CODE-TAB:csharp:Query_async paging_2_2@Indexes\Querying\Paging.cs /} {CODE-TAB:csharp:DocumentQuery paging_2_3@Indexes\Querying\Paging.cs /} {CODE-TAB:csharp:Index index_0@Indexes\Querying\Paging.cs /} {CODE-TAB-BLOCK:sql:RQL} from index "Products/ByUnitsInStock" where UnitsInStock > 10 limit 0, 10 // First loop will skip 0, take 10

// The next loops in the code will each generate the above RQL with an increased 'skip' value: // limit 10, 10 // limit 20, 10 // limit 30, 10 // ... {CODE-TAB-BLOCK/} {CODE-TABS/}

{NOTE/} {PANEL/}

{PANEL: Paging and performance}

Better performance:

It is recommended to explicitly set a page size when making a query that is expected to generate a significant number of results. This practice has several benefits:

  • Optimizes bandwidth usage by reducing data transfer between the server and client.
  • Prevents delays in response times caused by sending too much data over the network.
  • Avoids high memory consumption when dealing with numerous documents.
  • Ensures a more manageable user experience by not overwhelming users with massive datasets at once.

Performance hints:

  • By default, if the number of returned results exceeds 2048, the server will issue a "Page size too big" notification (visible in the Studio) with information about the query.

  • This threshold can be customized by modifying the value of the PerformanceHints.MaxNumberOfResults configuration key.

  • As suggested by the hint, you may consider using Streaming query results instead of paging.

Figure 1. Performance Hint

{PANEL/}

{PANEL: Paging through tampered results}

  • The QueryStatistics object contains the TotalResults property,
    which represents the total number of matching documents found in the query results.

  • The QueryStatistics object also contains the SkippedResults property.
    Whenever this property is greater than 0, that implies the server has skipped that number of results from the index.

  • The server will skip duplicate results internally in the following two scenarios:

    1. When making a Projection query with Distinct.

    2. When querying a Fanout index.

  • In those cases:

    • The SkippedResults property from the stats object will hold the count of skipped (duplicate) results.

    • The TotalResults property will be invalidated -
      it will Not deduct the number of skipped results from the total number of results.

  • In order to do proper paging in those scenarios:
    include the SkippedResults value when specifying the number of documents to skip for each page using:
    (currentPage * pageSize) + SkippedResults.

  • See the following examples:

{NOTE: }

A projection query with Distinct:

{CODE-TABS} {CODE-TAB:csharp:Query paging_3_1@Indexes\Querying\Paging.cs /} {CODE-TAB:csharp:Query_async paging_3_2@Indexes\Querying\Paging.cs /} {CODE-TAB:csharp:DocumentQuery paging_3_3@Indexes\Querying\Paging.cs /} {CODE-TAB:csharp:Index index_0@Indexes\Querying\Paging.cs /} {CODE-TAB:csharp:Projected_class projected_class@Indexes\Querying\Paging.cs /} {CODE-TAB-BLOCK:sql:RQL} from index "Products/ByUnitsInStock" where UnitsInStock > 10 select distinct Category, Supplier limit 0, 10 // First loop will skip 0, take 10, etc.
{CODE-TAB-BLOCK/} {CODE-TABS/}

{NOTE/}

{NOTE: }

Querying a Fanout index:

{CODE-TABS} {CODE-TAB:csharp:Query paging_4_1@Indexes\Querying\Paging.cs /} {CODE-TAB:csharp:Query_async paging_4_2@Indexes\Querying\Paging.cs /} {CODE-TAB:csharp:DocumentQuery paging_4_3@Indexes\Querying\Paging.cs /} {CODE-TAB:csharp:Index index_1@Indexes\Querying\Paging.cs /} {CODE-TAB-BLOCK:sql:RQL} from index "Orders/ByProductName" limit 0, 50 // First loop will skip 0, take 50, etc. {CODE-TAB-BLOCK/} {CODE-TABS/}

{NOTE/} {PANEL/}

Related Articles

Querying

Indexes