/messages does not provide a way of saying that you cannot paginate further

[ara4n]

We need to define the error code or mechanism to inform the user that they can't back-paginate further via /messages because the server has run out of DAG.

[ara4n]

see matrix-org/synapse#1623 and element-hq/element-web#2566

[KitsuneRal]

Do we really need to invent anything here if /messages returns state changes and m.room.create is (supposed to be) still the very first event no matter how you purge messages?

[richvdh]

it's not really a spec-bug imho: it's something we need to agree on and spec.

@KitsuneRal: that is one solution, but there are others which would make just as much (or more) sense. It's worth remembering that /messages lets you go both ways so we could do with a solution that works in the other direction too.

[turt2live]

err, yes - it should be an improvement not a bug. I wonder which issue I was trying to label as a bug....

[ara4n]

According to #1763 (comment) @turt2live thinks that the absence of a next_batch token should be specced to mean "you cannot paginate further", which feels very sane to me. Shall we do that (if it's not already?)

[richvdh]

/messages doesn't have a next_batch. I think you mean end, in which case +1

[joepie91]

Do we really need to invent anything here if /messages returns state changes and m.room.create is (supposed to be) still the very first event no matter how you purge messages?

Yes. The same problem occurs when you specify a to parameter (to fetch missing backlog in a limited /sync response for example), in which case you don't know what the final event is going to be. The only solutions right now are:

1. Checking whether the amount of results is less than the limit (fragile, will break in 1/Nth of cases)
2. Continuing to paginate until an empty chunk is produced (seemingly reliable, but requires a pointless extra request)

According to #1763 (comment) @turt2live thinks that the absence of a next_batch token should be specced to mean "you cannot paginate further", which feels very sane to me. Shall we do that (if it's not already?)

That sounds entirely reasonable to me, and would be in line with how pagination is designed in most HTTP APIs.

[richvdh]

Continuing to paginate until an empty chunk is produced (seemingly reliable, but requires a pointless extra request)

not entirely reliable, sadly. Synapse will sometimes return empty chunks if there is a section of timeline you're not allowed to see (eg because you weren't in the room).

[richvdh]

For the record, it looks like matrix-js-sdk currently decides that it's at the end of the timeline if start == end in the response.

[richvdh]

This got fixed by #3353.

[bakkot]

To be clear, accepted solution is "you can detected the end of the line by the absence of an end token in the response", right?

I note that this is not the behavior on matrix.org currently, which will always return an end token. That's fine, but it's a bit annoying that someone just reading the spec [e.g. me] is going to end up in an infinite loop when trying to write a client against the matrix.org homeserver.

[richvdh]

I note that this is not the behavior on matrix.org currently, which will always return an end token.

please file an issue against https://github.com/matrix-org/synapse