Make Finch pools lighter for self-hosting #2250
Conversation
BundleMonUnchanged files (7)
No change in files bundle size Final result: ✅ View report in BundleMon website ➡️ |
| defp maybe_add_sentry_pool(pool_config) do | ||
| case Application.get_env(:sentry, :dsn) do | ||
| dsn when is_binary(dsn) -> | ||
| Map.put(pool_config, dsn, size: 50) |
There was a problem hiding this comment.
This is the only pool where we don't currently specify :http2
I'm not sure what the benefits of using it elsewhere is, and why we don't use http2 for Sentry. Didn't want to dive into this rabbit hole for now but something I noticed
There was a problem hiding this comment.
Maybe it's https://forum.sentry.io/t/sentry-requests-using-http-1-1-instead-of-h2/13371
I am looking to learn what is the rationale for using http2 everywhere else? Is there an obvious benefit for our use cases?
There was a problem hiding this comment.
I am looking to learn what is the rationale for using http2 everywhere else? Is there an obvious benefit for our use cases?
No idea
There was a problem hiding this comment.
Perhaps we shouldn't use it until we find a good reason then? AFAIU http2 characteristics is the reason for this PR 😬
There was a problem hiding this comment.
Sorry to butt in, but sentry.io is a docs/marketing host. Ingest hosts are different and support h2. I've been using h2 with sentry for some time (~6 months) with no issues.
Try https://sentry.io and then https://o1012425.ingest.sentry.io/ on https://tools.keycdn.com/http2-test
I am looking to learn what is the rationale for using http2 everywhere else
In my experience using h2 results in lower memory usage (single tcp connection vs 50 by default) and higher throughput. But I agree that it's a less tested approach to http clients in elixir and there are issues sometimes, so for stability http/1 is probably better.
There was a problem hiding this comment.
Thanks for the insights. I think it's best to aim for stability so I'll try removing http2 config and using normal http1 pools with size 50.
| conn_opts: [transport_opts: [timeout: 15_000]] | ||
| ] | ||
| } | ||
|
|
||
| sentry_dsn = Application.get_env(:sentry, :dsn) | ||
| base_config | ||
| |> maybe_add_sentry_pool() |
There was a problem hiding this comment.
Not all these pools are needed for self-hosting so I'm leaving them out here if the relevant integrations have not been configured
| defp maybe_add_sentry_pool(pool_config) do | ||
| case Application.get_env(:sentry, :dsn) do | ||
| dsn when is_binary(dsn) -> | ||
| Map.put(pool_config, dsn, size: 50) |
There was a problem hiding this comment.
Maybe it's https://forum.sentry.io/t/sentry-requests-using-http-1-1-instead-of-h2/13371
I am looking to learn what is the rationale for using http2 everywhere else? Is there an obvious benefit for our use cases?
|
Bumped my self-hosted version to the 1.5 RC with this, absolutely decimated the memory usage by almost a whole gig. Which is pretty huge when the entire VPS is only 2GB of RAM. Before that, with some minor services running (nginx, weechat), plausible with clickhouse ended up getting total system memory usage to around 1.7GB |
|
Did you enable any new features like city-level geolocation? If so, you might want to disable them to get the memory usage to v1.4 level. |
|
Oh, I meant that the 1.5 is using 1GB less memory than previous version! And huh, is that enabled by default? |
Changes
Previous discussion: #2224
The current pool config eats a lot of memory for self-hosted installations. The pool count seems excessive so this is an attempt to make the app more lean by default.
If it turns out we need beefier pool counts for Plausible.Cloud now or in the future, we can add a config option but keep the lighter pools in the default configuration for easy self-hosting.
Tests
Changelog
Documentation
Dark mode