Skip to content

Commit

Permalink
guidance of offset vs cursor
Browse files Browse the repository at this point in the history
  • Loading branch information
gbilder committed Nov 7, 2016
1 parent 40ffccb commit d8030c7
Showing 1 changed file with 6 additions and 2 deletions.
8 changes: 6 additions & 2 deletions rest_api.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@
- v30, 2016-09-26, Highlight issue tracker
- v31, 2016-10-05, document `has-clinical-trial-number` and `has-abstract` filters
- v32, 2016-10-27, document rate limit headers
- v33, 2016-11-07, guidance on when to use `offset` vs `cursor`

## Background

Expand Down Expand Up @@ -182,11 +183,12 @@ Parameters can be used to query, filter and control the results returned by the
| `query` | limited [DisMax](https://wiki.apache.org/solr/DisMax) query terms |
| `filter={filter_name}:{value}`| filter results by specific fields |
| `rows={#}` | results per per page |
| `offset={#}` | result offset |
| `offset={#}` (mak 10k) | result offset (user `cursor` for larger `/works` result sets) |
| `sample={#}` | return random N results |
| `sort={#}` | sort results by a certain field |
| `order={#}` | set the sort order to `asc` or `desc` |
| `facet=t` | enable facet information in responses |
| `cursor={#}` | deep page through `/works` result sets |

Multiple filters can be specified by separating name:value pairs with a comma:

Expand Down Expand Up @@ -351,7 +353,7 @@ every time there is a change to metadata requiring a reindex.

## Result controls

You can control the delivery and selection results using the `rows`, `offset` and `sample` parameters.
You can control the delivery and selection results using the `rows`, `offset` and `sample` parameters. If you are expecting results beyond 10K, then use a `cursor` to deep page through the results.

### Rows

Expand All @@ -371,6 +373,8 @@ The number of returned items is controlled by the `rows` parameter, but you can

http://api.crossref.org/works?query=allen+renear&rows=5&offset=5

Offsets for `/works` are limited to 10K. Use `cursor` (see below) for larger `/works` results sets.

### Deep Paging with Cursors

Using large `offset` values can result in extremely long response times. Offsets in the 100,000s and beyond will likely cause a timeout before the API is able to respond. An alternative to paging through very large result sets (like a corpus used for text and data mining) it to use the API's exposure of Solr's deep paging cursors. Any combination of query, filters and facets may be used with deep paging cursors. While `rows` may be specified along with `cursor`, `offset` and `sample` cannot be used. To use deep paging make a query as normal, but include the `cursor` parameter with a value of `*`. In this example we will page through all `journal-article` works from member `311`:
Expand Down

0 comments on commit d8030c7

Please sign in to comment.