Sending to-device messages looks incredibly fragile, causing UISIs

[ara4n]

mildly terrified that matrix-js-sdk calls this.baseApis.sendToDevice("m.room.encrypted") 7 times
but sendToDevice has no retry mechanism
but almost none of those calls to sendToDevice appear to do retries
so if the PUT fails... the whole thing fails, whether that's keysharing or re-sharing or requesting or... anything.

That said, encryptAndSendKeysToDevices() has uses Promise.all().then() to send its messages... so if any of the sends fail, the whole attempt to encrypt will fail, so we will re-send those messages if it ever retries.

[turt2live]

A fix for the function would be:

const promises = [];
for (const userId of userIds) {
    // do not await - we want these to run concurrently
    promises.push(this.encryptAndSend(userId, props...).then(() => markSent()).catch(e => retry())); // don't let `retry()` throw
}
await Promise.all(promises);
// continue event sending

this is a lot easier on paper, though.

[BillCarsonFr]

Should also check for rate limits and use retry_after_ms

[BillCarsonFr]

See #1866

[richvdh]

duplicate of #1866