From a8a50858f584829f9f98eba45a3a552a7ac80551 Mon Sep 17 00:00:00 2001 From: lecaros Date: Tue, 7 Oct 2025 11:53:42 -0300 Subject: [PATCH 1/3] administration: buffering-and-storage: fix typo in mem_buf_limit Signed-off-by: lecaros --- administration/buffering-and-storage.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/administration/buffering-and-storage.md b/administration/buffering-and-storage.md index 7d433e4a8..b245348fd 100644 --- a/administration/buffering-and-storage.md +++ b/administration/buffering-and-storage.md @@ -89,7 +89,7 @@ If this input uses more than 50 MB memory to buffer logs, you will get a wa {% hint style="info" %} -`m em_buf_Limit` applies only when `storage.type` is set to the default value of `memory`. +`mem_buf_Limit` applies only when `storage.type` is set to the default value of `memory`. {% endhint %} From 481bc6e18bd0137777bffacfe52e048ede4414d1 Mon Sep 17 00:00:00 2001 From: lecaros Date: Tue, 7 Oct 2025 11:54:06 -0300 Subject: [PATCH 2/3] administration: buffering-and-storage: add missing ')'s Signed-off-by: lecaros --- administration/buffering-and-storage.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/administration/buffering-and-storage.md b/administration/buffering-and-storage.md index b245348fd..a3bb44362 100644 --- a/administration/buffering-and-storage.md +++ b/administration/buffering-and-storage.md @@ -112,8 +112,8 @@ If `storage.pause_on_chunks_overlimit` is enabled (default is off), the input pl Look for messages in the Fluent Bit log output like: ```text -[input] tail.1 paused (storage buf overlimit -[input] tail.1 resume (storage buf overlimit +[input] tail.1 paused (storage buf overlimit) +[input] tail.1 resume (storage buf overlimit) ``` ##### Limiting filesystem space for chunks From 5341405031c80b3431c6237b8711310b8fa7d40c Mon Sep 17 00:00:00 2001 From: lecaros Date: Tue, 7 Oct 2025 15:18:24 -0300 Subject: [PATCH 3/3] administration: buffering-and-storage: fix chunks up indication Signed-off-by: lecaros --- administration/buffering-and-storage.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/administration/buffering-and-storage.md b/administration/buffering-and-storage.md index a3bb44362..3ba34a6a6 100644 --- a/administration/buffering-and-storage.md +++ b/administration/buffering-and-storage.md @@ -103,7 +103,7 @@ Enabling filesystem buffering changes the behavior of the engine. Upon chunk cre Fluent Bit controls the number of chunks that are `up` in memory by using the filesystem buffering mechanism to deal with high memory usage and backpressure. -By default, the engine allows a total of 128 chunks `up` in memory in total, considering all chunks. This value is controlled by the service property `storage.max_chunks_up`. The active chunks that are `up` are ready for delivery and are still receiving records. Any other remaining chunk is in a `down` state, which means that it's only in the filesystem and won't be `up` in memory unless it's ready to be delivered. Chunks are never much larger than 2 MB, so with the default `storage.max_chunks_up` value of 128, each input is limited to roughly 256 MB of memory. +By default, the engine allows a total of 128 chunks `up` in memory in total, considering all chunks. This value is controlled by the service property `storage.max_chunks_up`. The active chunks that are `up` are either ready for delivery (marked busy and locked), or are still receiving records. Any other remaining chunk is in a `down` state, which means that it's only in the filesystem and won't be `up` in memory unless it's ready to be delivered. Chunks are never much larger than 2 MB, so with the default `storage.max_chunks_up` value of 128, each input is limited to roughly 256 MB of memory. If the input plugin has enabled `storage.type` as `filesystem`, when reaching the `storage.max_chunks_up` threshold, instead of the plugin being paused, all new data will go to chunks that are `down` in the filesystem. This lets you control memory usage by the service and also provides a guarantee that the service won't lose any data. By default, the enforcement of the `storage.max_chunks_up` limit is best-effort. Fluent Bit can only append new data to chunks that are `up`. When the limit is reached chunks will be temporarily brought `up` in memory to ingest new data, and then put to a `down` state afterwards. In general, Fluent Bit works to keep the total number of `up` chunks at or under `storage.max_chunks_up`.