Provide better guidelines on self-hosted limits

[chadwhitacre]

Our intention with self-hosted is to provide a turnkey solution for small workloads, and to leave the provisioning of Sentry for larger workloads as an exercise for the reader. We should be clearer in our docs about what exactly constitutes a "small" workload.

Immediate impetus is @kanadaj's comments:

Sentry has way too many separately moving parts to be easy to maintain on your own
onpremise has way too many ways to break on a sufficiently large deployment
if zookeeper breaks, you basically need a PhD in ZK to fix it
and probably a review of the ZK config cause iirc CH has some recommended configs for ZK
I don't use docker-compose or really any of that
I'm transcribing /onpremise to a kubernetes yml

+1

1. Am2020 - forum

[max-wittig]

Another solution would be to provide a kubernetes yaml instead, as docker-compose doesn't really work for productive use on it's own

[kanadaj]

@max-wittig I've discussed it with Chad, if the docker-compose is rewritten to be using volume mounts instead of custom compiling Dockerfiles to embed the config, it'd be trivial to convert it to kubernetes yaml files, probably even by existing CLI tools assuming they can pick up and embed the required files into ConfigMaps/Secrets.

Even if they can't, it's not hard to automate that.

Another option would be an operator, which is way more overhead to work out, but could help with updates between Sentry versions in the future if a more complex upgrade path is required.

[chadwhitacre]

Filing for later ...

The 2400MB min RAM requirement in the readme for onpremise seems waaaay below what I'd realistically expect to use

https://discord.com/channels/621778831602221064/621783515423440927/795686074851524700

Intel(R) Xeon(R) Silver 4108 CPU @ 1.80GHz with 4 cores
8 GB of ram

https://discord.com/channels/621778831602221064/621783515423440927/795673428287684639

[kanadaj]

If it helps any with the numbers, with a freshly wiped ClickHouse, our own Sentry deployment uses 3.74GB RAM atm, highest I've seen was around the 5-6GB mark. Ingesting ~50-100 errors and 5400 transactions an hour right now, though I expect this to go way up once we have integration for other languages.

EDIT: Correction, it's now sitting at the 6GB mark, mostly stable now.

[chadwhitacre]

@kanadaj Is that RAM for just ClickHouse? Or the whole kit and kaboodle?

[kanadaj]

The whole setup @chadwhitacre

[BYK]

@kanadaj

if the docker-compose is rewritten to be using volume mounts instead of custom compiling Dockerfiles to embed the config, it'd be trivial to convert it to kubernetes yaml files, probably even by existing CLI tools assuming they can pick up and embed the required files into ConfigMaps/Secrets.

This is already the case, we have stopped doing this long ago: #407.

[lucas-zimerman]

@kanadaj Is that RAM for just ClickHouse? Or the whole kit and kaboodle?

If that helps, here're some numbers from production:


(each bar is 1h)


^ hdd

[chadwhitacre]

I bumped the RAM requirement on the README to 8 GB, seems safe and easy. Since resource requirements are a function of throughput I'd like to document both sides of the equation, possibly for both real-world and test scenarios. Keeping this open for that.

[kohenkatz]

Another solution would be to provide a kubernetes yaml instead, as docker-compose doesn't really work for productive use on it's own

For smaller deployments, docker-compose works perfectly. It would be a mistake to assume that everyone has a Kubernetes infrastructure. The small business I work for has GitLab (Omnibus Docker), Sonatype Nexus, and Sentry deployed on a single Linode instance (16GB, 6 cores) using around 9.5GB RAM for all of them, serving a dozen developers and collecting logs/crashes from around 1,000 customer users. It works perfectly, still has room to grow, and we have no reason to need anything more complex for the foreseeable future.

I have no problem with also providing a kubernetes YAML file, except that I suspect it's a bad idea to have two options that will end up diverging from each other.