Add endpoint to get an event at a given timestamp - MSC3030

[MadLittleMods]

Add endpoint to get an event at a given timestamp (MSC3030)

Complement tests: matrix-org/complement#178

Client API endpoint. This will also go and fetch from the federation API endpoint if unable to find an event locally or we found an extremity with possibly a closer event we don't know about.

GET /_matrix/client/unstable/org.matrix.msc3030/rooms/<roomID>/timestamp_to_event?ts=<timestamp>&dir=<direction>
{
    "event_id": ...
    "origin_server_ts": ...
}

Federation API endpoint:

GET /_matrix/federation/unstable/org.matrix.msc3030/timestamp_to_event/<roomID>?ts=<timestamp>&dir=<direction>
{
    "event_id": ...
    "origin_server_ts": ...
}

---

The initial implementation spawned from resurrecting old commits from @erikjohnston, https://gitlab.matrix.org/new-vector/synapse/-/compare/develop...kraken

Dev notes

COMPLEMENT_DIR=../complement ./scripts-dev/complement.sh TestJumpToDateEndpoint

• get_events_around(room_id, event_id, before_limit, after_limit, event_filter)
• find_first_stream_ordering_after_ts_txn(txn, ts)
• get_room_event_before_stream_ordering

---

all_backward_extremity_query = """
    SELECT event_id FROM event_backward_extremities
    WHERE room_id = ?
    LIMIT 100
"""

all_forward_extremity_query = """
    SELECT event_id FROM event_forward_extremities
    WHERE room_id = ?
    LIMIT 100
"""

txn.execute(all_backward_extremity_query, (room_id,))
all_backward_extremties = txn.fetchall()
txn.execute(all_forward_extremity_query, (room_id,))
all_forward_extremties = txn.fetchall()
logger.info(
    "all_forward_extremties=%s all_backward_extremties=%s",
    all_forward_extremties,
    all_backward_extremties,
)

---

Function overloads:

• https://treyhunner.com/2018/10/asterisks-in-python-what-they-are-and-how-to-use-them/#Keyword-only_arguments_without_positional_arguments

Todo

• Make sure this also works with historical messages from MSC2716
• If your local server has only backfilled a little bit of the room and you ask for messages in the distant past (past the backwards extremities), the server should probably try to backfill some messages from back then. Or just ask another homeserver
• Do we want to add a ?dir=[f|b] query parameter so you can specify which direction from the date you want to get the message from. For example when I want to use this in combination with MSC2716 to find a place to import messages, I want to get before/after a certain date vs the closest date.

Pull Request Checklist

• Pull request is based on the develop branch
• Pull request includes a changelog file. The entry should:
  • Be a short description of your change which makes sense to users. "Fixed a bug that prevented receiving messages from other servers." instead of "Moved X method from EventStore to EventWorkerStore.".
  • Use markdown where necessary, mostly for code blocks.
  • End with either a period (.) or an exclamation mark (!).
  • Start with a capital letter.
• Pull request includes a sign off
• Code style is correct (run the linters)

[erikjohnston]

I think we probably need to think a little bit about that query performance, I don't it's terribly efficient as it stands. I think its entirely solvable though, so this is fine to prove that the MSC is possible 👍

[erikjohnston]

I don't think there is anything to do here until the MSC progresses?

[MadLittleMods]

@erikjohnston I'm going to be exploring this MSC next after MSC2716 in order to make static archives more efficient. Use server-side rendered Hydrogen with this endpoint to show what happened on that day in the archive.

[MadLittleMods]

One problem I'm seeing with going with a /timestamp_to_event endpoint is that there is no way to actually paginate with /messages after getting the event_id.

We probably want to consider also returning a pagination stream token to use with the from/to on /messages

[MadLittleMods]

Or just not worry about it, we can make a separate request to get one:

https://github.com/matrix-org/complement/blob/581cb0b26e602763c4ec52e3c5f5047a2a4d95b7/tests/msc2716_test.go#L1323-L1326

// Get a pagination token for a single event
contextRes := c.MustDoFunc(t, "GET", []string{"_matrix", "client", "r0", "rooms", roomID, "context", someEventId}, client.WithContentType("application/json"), client.WithQueries(url.Values{
	"limit": []string{"0"},
}))

[MadLittleMods]

Just de-nesting this function so we can re-use it

[squahtx]

LGTM!

Sorry for the lengthy review process. I really appreciate the updates to the comments and documentation you're doing!

[MadLittleMods]

Thanks for the review @squahtx! 🐓 The daily regular drum of review was fantastic to blast this forward quickly and got this to a much better state 🤘