Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions pipeline/inputs/blob.md
Original file line number Diff line number Diff line change
@@ -1,21 +1,21 @@
# Blob

The _Blob_ input plugin monitors a directory and processes binary (blob) files. It scans the specified path at regular intervals, reads binary files, and forwards them as records through the Fluent Bit pipeline. This plugin is useful for processing binary log files, artifacts, or any binary data that needs to be collected and forwarded to outputs.
The _Blob_ input plugin monitors a directory and processes binary (blob) files. It scans the specified path at regular intervals, reads binary files, and forwards them as records through the Fluent Bit pipeline. This plugin processes binary log files, artifacts, or any binary data that needs to be collected and forwarded to outputs.

## Configuration parameters

The plugin supports the following configuration parameters:

| Key | Description | Default |
|:------------------------------------|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:---------|
| `alias` | Sets an alias for multiple instances of the same input plugin. Useful when you need to run multiple blob input instances with different configurations. | _none_ |
| `alias` | Sets an alias for multiple instances of the same input plugin. This helps when you need to run multiple blob input instances with different configurations. | _none_ |
| `database_file` | Specify a database file to keep track of processed files and their state. This enables the plugin to resume processing from the last known position if Fluent Bit is restarted. | _none_ |
| `exclude_pattern` | Set one or multiple shell patterns separated by commas to exclude files matching certain criteria. For example, `exclude_pattern *.tmp,*.bak` will exclude temporary and backup files from processing. | _none_ |
| `log_level` | Specifies the log level for this input plugin. If not set here, the plugin uses the global log level specified in the `service` section. Valid values: `off`, `error`, `warn`, `info`, `debug`, `trace`. | `info` |
| `log_suppress_interval` | Suppresses log messages from this input plugin that appear similar within a specified time interval. Set to `0` to disable suppression. The value must be specified in seconds. This helps reduce log noise when the same error or warning occurs repeatedly. | `0` |
| `mem_buf_limit` | Set a memory buffer limit for the input plugin instance in bytes. If the limit is reached, the plugin will pause until the buffer is drained. If set to `0`, the buffer limit is disabled. If the plugin has enabled filesystem buffering, this limit won't apply. The value must be according to the [Unit Size](../../administration/configuring-fluent-bit/unit-sizes.md) specification. | `0` |
| `path` | Path to scan for blob (binary) files. Supports wildcards and glob patterns. For example, `/var/log/binaries/*.bin` or `/data/artifacts/**/*.dat`. This is a required parameter. | _none_ |
| `routable` | If `true`, the data generated by the plugin can be forwarded to other plugins or outputs. If `false`, the data will be discarded. Use this for testing or when you want to process data but not forward it. | `true` |
| `routable` | If `true`, the data generated by the plugin can be forwarded to other plugins or outputs. If `false`, the data will be discarded. This is used for testing or when you want to process data but not forward it. | `true` |
| `scan_refresh_interval` | Set the interval time to scan for new files. The plugin periodically scans the specified path for new or modified files. The value must be specified according to the [Unit Size](../../administration/configuring-fluent-bit/unit-sizes.md) specification (`2s`, `30m`, `1h`). | `2s` |
| `storage.pause_on_chunks_overlimit` | Enable pausing on an input when it reaches its chunks limit. When enabled, the plugin will pause processing if the number of chunks exceeds the limit, preventing memory issues during backpressure scenarios. | `false` |
| `storage.type` | Sets the storage type for this input. Options: `filesystem` (persists data to disk), `memory` (stores data in memory only), or `memrb` (memory ring buffer). For production environments with high data volumes, consider using `filesystem` to prevent data loss during restarts. | `memory` |
Expand Down