Improve syncing of big accounts: add caching, increase timeout

[MayeulC]

Problem

• My Matrix account is quite big. Even on a 8-core xeon E31270 x86_64 cpu and 32 GB of RAM, I run into timeouts during initial sync and subsequent syncs with the default config.
• With the default config, running into an nginx timeout (default 60s) means that the sync computation is thrown away, and the server starts from scratch: cache sync results for a while after the request completes matrix-org/synapse#3880 Default sync_response_cache_duration can lead to clients never being able to initial sync matrix-org/synapse#13027

Solution

• Increase the default nginx timeout for the sync endpoint, from 60s. 7 minutes is arbitrary, but should provide a lot more time for syncing before it is attempted again, while still being a reasonable waiting time for big accounts.
• Introduce caching of sync response. If connection is cut while syncing, the server doesn't have to re-compute everything. This will use slightly more memory (but not that much, it seems). It can also be disabled in the config.

I also left commented out other caching options I use as I have plenty of RAM. The comments are taken from upstream homeserver.yaml. This should make it easier for curious user to play with cache settings, and can be activated by default at a later time.

PR Status

• Code finished and ready to be reviewed/tested
• The fix/enhancement were manually tested (if applicable)

Automatic tests

Automatic tests can be triggered on https://ci-apps-dev.yunohost.org/ after creating the PR, by commenting "!testme", "!gogogadgetoci" or "By the power of systemd, I invoke The Great App CI to test this Pull Request!". (N.B. : for this to work you need to be a member of the Yunohost-Apps organization)

[lapineige]

This should make it easier for curious user to play with cache settings, and can be activated by default at a later time.

Maybe we should shortly document that in this repository ReadMe (for instance), to improve discoverability of such features ?

[MayeulC]

Actually my comments are a bit outdated, it seems (I've been running the current config for about one year) : upstream changed the default caching period to 2 min a few months ago: matrix-org/synapse#13042 so maybe the change isn't that necessary after all.

Element timeouts after 90s though, which is longer than nginx's default of (IIRC) 60s.

Hmm, about the documentation, yes, but I would rather point to upstream docs, and/or the config file comments.

[Josue-T]

Haven't tested but look like nice

[lapineige]

Hmm, about the documentation, yes, but I would rather point to upstream docs, and/or the config file comments.

I don't mean a full tutorial : mentioning the existence of these settings, pointing to the doc and/or the specific file to modify, is enough :)

[lapineige]

By the way, should this be pointing to Testing instead ?

[MayeulC]

Sorry, I clicked too fast in the GitHub interface. Edit: changed base

Also, I had forgotten part of the nginx config... And I was wondering why Matrix was so silent today :P

[MayeulC]

I added a commit that mentions cache tuning. I am not sure the preferred way to edit the configuration is by editing homeserver.yaml, but that's what I've been doing (overwritten on upgrade, hence the PR).

[Josue-T]

Maybe it would be good to put this in a specific file like /etc/matrix-synapse/conf.d/cache.yaml to avoid that this settings is always override by the update.

And the best solution is to give the possibility to edit this from the config pannel.

[Josue-T]

Closing as there are still some work to make it working.