Skip to content

Commit 4a1c741

Browse files
authored
Merge pull request #1505 from fluent/lynettemiles/sc-105731/update-administration-memory-management-md
2 parents 2b88b72 + 0767ac6 commit 4a1c741

File tree

2 files changed

+32
-15
lines changed

2 files changed

+32
-15
lines changed

administration/memory-management.md

Lines changed: 30 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -1,30 +1,46 @@
1-
# Memory Management
1+
# Memory management
22

33
<img referrerpolicy="no-referrer-when-downgrade" src="https://static.scarf.sh/a.png?x-pxid=5cc3ce54-e910-4ebf-85f5-f02530b3e11b" />
44

5-
In certain scenarios it would be ideal to estimate how much memory Fluent Bit could be using, this is very useful for containerized environments where memory limits are a must.
5+
You might need to estimate how much memory Fluent Bit could be using in scenarios
6+
like containerized environments where memory limits are essential.
67

7-
In order to that we will assume that the input plugins have set the **Mem\_Buf\_Limit** option \(you can learn more about it in the [Backpressure](backpressure.md) section\).
8+
To make an estimate, in-use input plugins must set the `Mem_Buf_Limit`option.
9+
Learn more about it in [Backpressure](backpressure.md).
810

911
## Estimating
1012

11-
Input plugins append data independently, so in order to do an estimation, a limit should be imposed through the **Mem\_Buf\_Limit** option. If the limit was set to _10MB_ we need to estimate that in the worse case, the output plugin likely could use _20MB_.
13+
Input plugins append data independently. To make an estimation, impose a limit with
14+
the `Mem_Buf_Limit` option. If the limit was set to `10MB`, you can estimate that in
15+
the worst case, the output plugin likely could use `20MB`.
1216

13-
Fluent Bit has an internal binary representation for the data being processed, but when this data reaches an output plugin, it will likely create its own representation in a new memory buffer for processing.
14-
The best examples are the [InfluxDB](../pipeline/outputs/influxdb.md) and [Elasticsearch](../pipeline/outputs/elasticsearch.md) output plugins, both need to convert the binary representation to their respective custom JSON formats before it can be sent to the backend servers.
17+
Fluent Bit has an internal binary representation for the data being processed. When
18+
this data reaches an output plugin, it can create its own representation in a new
19+
memory buffer for processing. The best examples are the
20+
[InfluxDB](../pipeline/outputs/influxdb.md) and
21+
[Elasticsearch](../pipeline/outputs/elasticsearch.md) output plugins, which need to
22+
convert the binary representation to their respective custom JSON formats before
23+
sending data to the backend servers.
1524

16-
So, if we impose a limit of _10MB_ for the input plugins and consider the worse case scenario of the output plugin consuming _20MB_ extra, as a minimum we need \(_30MB_ x 1.2\) = **36MB**.
25+
When imposing a limit of `10MB` for the input plugins, and a worst case scenario of
26+
the output plugin consuming `20MB`, you need to allocate a minimum (`30MB` x 1.2) =
27+
`36MB`.
1728

18-
## Glibc and Memory Fragmentation
29+
## Glibc and memory fragmentation
1930

20-
It is well known that in intensive environments where memory allocations happen in the orders of magnitude, the default memory allocator provided by Glibc could lead to high fragmentation, reporting a high memory usage by the service.
31+
In intensive environments where memory allocations happen in the orders of magnitude,
32+
the default memory allocator provided by Glibc could lead to high fragmentation,
33+
reporting a high memory usage by the service.
2134

22-
It's strongly suggested that in any production environment, Fluent Bit should be built with [jemalloc](http://jemalloc.net/) enabled \(e.g. `-DFLB_JEMALLOC=On`\). Jemalloc is an alternative memory allocator that can reduce fragmentation \(among others things\) resulting in better performance.
35+
It's strongly suggested that in any production environment, Fluent Bit should be
36+
built with [jemalloc](http://jemalloc.net/) enabled (`-DFLB_JEMALLOC=On`).
37+
The jemalloc implementation of malloc is an alternative memory allocator that can
38+
reduce fragmentation, resulting in better performance.
2339

24-
You can check if Fluent Bit has been built with Jemalloc using the following command:
40+
Use the following command to determine if Fluent Bit has been built with jemalloc:
2541

26-
```text
27-
$ bin/fluent-bit -h | grep JEMALLOC
42+
```bash
43+
bin/fluent-bit -h | grep JEMALLOC
2844
```
2945

3046
The output should look like:
@@ -36,5 +52,4 @@ FLB_HAVE_PROXY_GO FLB_HAVE_JEMALLOC JEMALLOC_MANGLE FLB_HAVE_REGEX
3652
FLB_HAVE_C_TLS FLB_HAVE_SETJMP FLB_HAVE_ACCEPT4 FLB_HAVE_INOTIFY
3753
```
3854

39-
If the FLB\_HAVE\_JEMALLOC option is listed in _Build Flags_, everything will be fine.
40-
55+
If the `FLB_HAVE_JEMALLOC` option is listed in `Build Flags`, jemalloc is enabled.

vale-styles/FluentBit/Spelling-exceptions.txt

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -58,6 +58,7 @@ Fargate
5858
Firehose
5959
FluentBit
6060
Fluentd
61+
glibc
6162
Golang
6263
golib
6364
Grafana
@@ -73,6 +74,7 @@ Hostname
7374
Ingester
7475
Keepalive
7576
Istio
77+
jemalloc
7678
keepalive
7779
Kinesis
7880
kubectl

0 commit comments

Comments
 (0)