Add environment variable SYNAPSE_CACHE_FACTOR, default it to 0.1

[erikjohnston]

(And a couple of other things)

[ara4n]

@erikjohnston did you see https://hveem.no/mj%C3%B8lnir.tors.cache.hammer.diff.txt ? Tor tweaked a few other things in order to bring the footprint on his HS down to ~512MB tops RSZ

[NegativeMjark]

LGTM

[ara4n]

does the env var not need doc?

[erikjohnston]

does the env var not need doc?

I'm not sure where to doc this, or even how much I want to publicise it. This is such an awful (and hopefully temporary) hack that I'd be quite happy to keep it under wraps and only use it for our bigger deployments.

[erikjohnston]

@matrixbot test this please

[ara4n]

The reason for implementing this is to give folks who complain about RAM
an coarse and easy knob to twist as a short-term solution. I will
absolutely cry if I spend my life having to tell people manually about
this in HQ or matrix-dev. Can we please just put a paragraph at the end
of the Synapse README of the form:

Halp!! Synapse eats all my RAM!

Synapse's architecture is quite RAM hungry currently - we deliberately
cache a lot of recent room data and metadata in RAM in order to speed up
common requests. We'll improve this in future, but for now the easiest
way to either reduce the RAM usage (at the risk of slowing things down)
is to set the almost-undocumented SYNAPSE_CACHE_FACTOR environment
variable. Roughly speaking, a SYNAPSE_CACHE_FACTOR of 1.0 will max out
at around 3-4GB of resident memory - this is what we currently run the
matrix.org on. The default setting is currently 0.1, which is probably
around a ~700MB footprint. You can dial it down further to 0.02 if
desired, which targets roughly ~512MB. Conversely you can dial it up if
you need performance for lots of users and have a box with a lot of RAM.

On 02/03/2016 11:51, Erik Johnston wrote:

does the env var not need doc?

I'm not sure where to doc this, or even how much I want to publicise
it. This is such an awful (and hopefully temporary) hack that I'd be
quite happy to keep it under wraps and only use it for our bigger
deployments.

— Reply to this email directly or view it on GitHub
#612 (comment).

[erikjohnston]

@matrixbot test this please

[erikjohnston]

Fine. Added to README.

I think its a mistake to publicise this hack given that lowering the SYNAPSE_CACHE_FACTOR further is unlikely to have much effect, and hopefully the hack is temporary. I'd much rather we just spent the time and did this properly if we actually wanted to expose this option to people.

[erikjohnston]

The sytest that fails seems to be just that the last test is timing out on jenkins since its doing so much