The document about memory usage need be clarify

[netroby]

The document : https://prometheus.io/docs/operating/storage/#memory-usage

Prometheus keeps all the currently used chunks in memory. In addition, it keeps the most recently used chunks in memory up to a threshold configurable via the storage.local.memory-chunks flag. If you have a lot of RAM available, you might want to increase it above the default value of 1048576 (and vice versa, if you run into RAM problems, you can try to decrease it). Note that the actual RAM usage of your server will be much higher than what you would expect from multiplying storage.local.memory-chunks by 1024 bytes. There is inevitable overhead for managing the sample data in the storage layer. Also, your server is doing many more things than just storing samples. The actual overhead depends on your usage pattern. In extreme cases, Prometheus has to keep more chunks in memory than configured because all those chunks are in use at the same time. You have to experiment a bit. The metrics prometheus_local_storage_memory_chunks and process_resident_memory_bytes, exported by the Prometheus server, will come in handy. As a rule of thumb, you should have at least three times more RAM available than needed by the memory chunks alone.

I have a dev vm, it only have 512MB memory (Most like the common cloud hosting provided us like DigitalOcean, Vultr or others ). How to set this flag?

And if i want to deploy premetheus to production server, which may have 4GB or 8GB or 16GB memory.
How to set the flags?

Would you please gave a real example to show us ? We have not experience on how to configure the flags. and do not quite understand how the flag will affect the result memory usage.

for example. may we configure

storage.local.memory-chunks=4096

The default unit was byte or kilobyte?

[netroby]

I can search many issues reported by user, will caused by the lack of clarify document .

https://github.com/prometheus/prometheus/search?q=storage.local.memory-chunks&type=Issues&utf8=%E2%9C%93

[beorn7]

We tried to make it as clear as possible. Most user issues were reported by users who weren't aware of that part of documentation at all.

In any case, if you have suggestion to clarify the documentation, PRs are welcome.

[netroby]

For 512MB vm, what's the best value of the storage.local.memory-chunks ?
If could add the example value on the document. will be better for new users who want to try prometheus in a cloud box or local vm

[beorn7]

The doc says "As a rule of thumb, you should have at least three times more RAM available than needed by the memory chunks alone." So that's at most 512MiB/1kiB/3 = 174762 memory chunks at most. (Talking to people working in tech, I'd assume I don't have to do the math for various example values.) The doc also says "Also, your server is doing many more things than just storing samples. The actual overhead depends on your usage pattern. In extreme cases, Prometheus has to keep more chunks in memory than configured because all those chunks are in use at the same time. You have to experiment a bit."

In short: There is no single best value. It's plainly impossible to tell users "set this flag to that value, and everything is fine". See also #455, which will, once solved, get things closer to "use that much RAM, and adjust everything else accordingly".

[grobie]

Please send your questions to the https://groups.google.com/forum/#!forum/prometheus-users mailing list.

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.