Add Link: pagination HTTP headers

[simonw]

Spun off from #782. These can go on all of the JSON endpoints that support pagination.

[simonw]

The GitHub API has these:

link: <https://api.github.com/repositories/107914493/commits?page=2>; rel="next", <https://api.github.com/repositories/107914493/commits?page=46>; rel="last"

WordPress API: https://css-tricks.com/wp-json/wp/v2/posts

link: <https://css-tricks.com/wp-json/wp/v2/posts?page=2>; rel="next"

It also has these:

x-wp-total: 5162
x-wp-totalpages: 517

[simonw]

This should be documented on https://docs.datasette.io/en/stable/json_api.html

[simonw]

I'm just going to implement the 'next' one.

[simonw]

I won't offer a total-number header. That's expensive to calculate - no need to calculate it unless it's explicitly asked for.

[simonw]

First example page to add this to: ?_shape=array, e.g. https://latest.datasette.io/fixtures/compound_three_primary_keys.json?_shape=array

[simonw]

Demo using paginate-json:

% paginate-json 'https://latest.datasette.io/fixtures/compound_three_primary_keys.json?_shape=array' | jq '. | length' 
https://latest.datasette.io/fixtures/compound_three_primary_keys.json?_shape=array
http://latest.datasette.io/fixtures/compound_three_primary_keys.json?_shape=array&_next=a%2Cd%2Cv
http://latest.datasette.io/fixtures/compound_three_primary_keys.json?_shape=array&_next=a%2Ch%2Cr
http://latest.datasette.io/fixtures/compound_three_primary_keys.json?_shape=array&_next=a%2Cl%2Cn
http://latest.datasette.io/fixtures/compound_three_primary_keys.json?_shape=array&_next=a%2Cp%2Cj
http://latest.datasette.io/fixtures/compound_three_primary_keys.json?_shape=array&_next=a%2Ct%2Cf
http://latest.datasette.io/fixtures/compound_three_primary_keys.json?_shape=array&_next=a%2Cx%2Cb
http://latest.datasette.io/fixtures/compound_three_primary_keys.json?_shape=array&_next=b%2Ca%2Cx
http://latest.datasette.io/fixtures/compound_three_primary_keys.json?_shape=array&_next=b%2Ce%2Ct
http://latest.datasette.io/fixtures/compound_three_primary_keys.json?_shape=array&_next=b%2Ci%2Cp
http://latest.datasette.io/fixtures/compound_three_primary_keys.json?_shape=array&_next=b%2Cm%2Cl
1001

New documentation: https://docs.datasette.io/en/latest/json_api.html#pagination