Posts feed pagination issue

[quentez]

I think that the current way pagination is specified in the protocol could be improved to be more consistent and to make better use of the pages object returned by the posts_feed endpoint.

At the moment, the query parameters before/since/until are different from other parameters like mentions and types in that they don't affect the pages object. To me, this means that the set of posts I'm working on, as an app, is all the posts that responds to the non-pagination criteria. The first, next, previous and last links will therefore apply to this set (the first page of the set, the next page in the set, etc).

The first problem here is that the count header returned by HEAD and GET requests on the posts_feed endpoint doesn't apply to this set I just mentioned since it can be affected by the pagination parameters. This means that the count returned when querying the next link won't be the same as the count I received for my first request. This is confusing.

This second issue is that the pagination links can't be used if I'm trying to retrieve a set of posts that are constrained by one or multiple timestamps. One example would be to try and retrieve all the posts between ts1 and ts2, if those can't be retrieved in a single request then paging will be required. Problem is: the paging links won't apply to the set of posts between ts1 and ts2, but to the whole posts collection.

My main concern with this is the inconsistency between the pages object and the count header, and there's two ways out of it:

1. Make the count header consistent with the pages object and make the before/until/since parameters have no effects on it.
2. Add two new parameters for time conditions (something like before/until), those will be used by apps to select a subset of posts, count that subsets, etc. And keep the three current parameters only for pagination within this subset (therefore not affecting the count, nor the pages object). Here's a few examples of what simple queries could look like with that solution:

Get a count of all the posts received between ts1 and ts2:

HEAD posts_feed?before=ts1&until=ts2

Retrieve the first page of the previous request:

GET posts_feed?before=ts1&until=ts2

And the second page (assuming ts3 is the date/time of the first page's last post)

GET posts_feed?before=ts1&until=ts2&page_before=ts3

This also makes it easier to compute the pages object:

• The first page is always the bare query, without any paging parameter.
• The last page has page_before or page_since with extreme values.
• The next/previous pages have page_before/page_since with the date/times of the first/last posts returned in the current request.
• Only the paging parameters are modified, not modifying the date criteria used by the app to select a subset of posts.

And I'll add that this solution doesn't make it more difficult for a server to proxy its request to one or more remote servers.