Message reordering with multi-relay

[link2xt]

1. First device receives messages and reads them all.
2. Second device receives messages from secondary relay.
   Secondary relay only have messages of some users
   who use newer version and are aware of this relay.
   MDNs to self are also received and all messages get marked as read.
3. Second device receives messages from primary relay.
   There all messages arrive, from other group members.
   But they are sorted to the end of the chat because
   messages are always added to the chat after the last read message.

This will not be entirely fixed by everyone upgrading, it is always possible that some users are only aware of part of the relays or send message while being offline, or the relays are not gossiped because we don't gossip keys in every message.

[link2xt]

I think we should drop the logic for sorting the message before the last read one, and fix the problem of accidentally missing late message that gets sorted to the top as discussed in #7965

[link2xt]

I am now frequently getting messages on Android first and in the correct order, but then open desktop and get messages from users of the new core somewhere at the top of the chat because messages of others get sorted to the end.

Will try to run #8031 for a while and see if there are problems or if I overlooked something, but it seems to only make everything better. Desktop already opens the chat at the correct place when new message arrives, the only problem I expect is that a message may arrive late into the chat that I already have open. But with the changes #7965 i should be able to get out of chat, get back in, and get scrolled to the late message that is somewhere at the top.

[iequidoo]

MDNs to self are also received and all messages get marked as read.

There's one more problem with MDNs: with multi-relay they may be received earlier than referenced messages and then such MDNs are ignored in mimeparser::handle_mdn().

[link2xt]

We send to all our relays, so at least from one relay it should not be ignored because it will arrive after the message. If the message arrives to unpublished relay, this may happen.

[iequidoo]

We send to all our relays, so at least from one relay it should not be ignored because it will arrive after the message. If the message arrives to unpublished relay, this may happen.

Still smth like this may happen:

• Device 1 receives an incoming message from the primary relay
• Device 1 sends a self-MDN to all relays
• Device 2 receives the MDN from the secondary relay and trashes it
• Device 2 receives the message from the primary relay
• Device 2 sees the MDN on the primary relay, but it's ignored because of the tombstone

[iequidoo]

I think that the right way to fix all these sorting issues is to do smth similar to what we did for inbox/mvbox message ordering, but w/o moving messages:

• Remember how many messages each relay has (i.e. message UIDs)
• Receive all messages from the primary relay ("mvbox")
• Receive all messages up to the remembered UID values from secondary relays ("inboxes")

...
But the primary relay needs to be available then. If it's not available, after a small timeout, we have to download messages from the secondary relays and thus get message reordering again.

[ell1e]

Is this the same problem? Today for some rooms I'm getting a completely scrambled ordering, and when I compare two of my clients I get different ordering on both of them. Also, I don't see the one message I sent on my one client on the other (edit: it did show up, eventually!). It's quite the adventure. This is with desktop v2.43.0. It seems to get more chaotic and mixed up whenever the wifi gets worse.

I don't think anybody involved is using more than one relay.

[link2xt]

I think that the right way to fix all these sorting issues is to do smth similar to what we did for inbox/mvbox message ordering, but w/o moving messages

For inbox/mvbox we moved all messages to mvbox and then downloaded only from mvbox to get the same order on all devices. This cannot be done with multiple relays. We also cannot wait for one relay before downloading from another, it may be not available.