Skip to content
Closed
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
101 changes: 101 additions & 0 deletions observability/otel/otel-collector/otel-collector-extensions.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -161,6 +161,107 @@ include::snippets/technology-preview.adoc[]
<7> Buffers the OTLP Exporter data on the local file system.
<8> Starts the File Storage Extension by the Collector.

.Parameters used by the File Storage Extension
[options="header"]
[cols="a,a,a"]
|===
|Parameter |Description |Default

|`directory`
|The relative or absolute path to the dedicated data storage directory. The default is `%ProgramData%\Otelcol\FileStorage` on Windows and `/var/lib/otelcol/file_storage` otherwise.
|Platform-dependent

|`timeout`
|Maximum time to wait for a file lock, typically does not need modification.
|`1s`

|`fsync`
|When enabled, forces the database to perform an `fsync` after each write for data integrity, at the cost of performance.
|`false`

|`create_directory`
|Creates the data storage and compaction directory if it does not already exist, with default permissions of `0750`.
|`false`

|`directory_permissions`
|Sets custom permissions for directory creation if `create_directory` is enabled.
|`0750`

|`compaction.on_start`
|Triggers compaction when the Collector starts.
|`false`

|`compaction.on_rebound`
|Triggers online (rebound) compaction when certain conditions are met (see details below).
|`false`

|`compaction.directory`
|Specifies the directory used as a staging area during compaction.
|`/tmp/`

|`compaction.max_transaction_size`
|Defines the maximum size for a compaction transaction. Set to zero to ignore transaction sizes.
|`65536`

|`compaction.cleanup_on_start`
|Removes any temporary compaction files left from a previous interrupted compaction process.
|`false`

|`compaction.rebound_needed_threshold_mib`
|If storage exceeds this threshold, the "compaction needed" flag is enabled, indicating potential need for rebound compaction.
|`100`

|`compaction.rebound_trigger_threshold_mib`
|When the "compaction needed" flag is set and storage falls below this threshold, compaction is triggered.
|`10`

|`compaction.check_interval`
|Frequency at which conditions for compaction are checked.
|`5s`
|===

=== Troubleshooting

The File Storage Extension supports storing telemetry data locally and requires read and write access to a directory. The extension can use a default or custom directory, but the directory must exist if `create_directory` is set to `false`.

When troubleshooting components that use the File Storage extension, it can be helpful to examine the raw contents of the files generated by the extension. The easiest way to read these files is by using the `strings` utility, available on both link:https://man7.org/linux/man-pages/man1/strings.1.html[Linux] and link:https://learn.microsoft.com/en-us/sysinternals/downloads/strings[Windows].

==== Telemetry data not being stored or persisted

If telemetry data is not being persisted as expected, consider the following:

.Procedure

- Verify the `directory`: ensure that the directory specified in `directory` exists and is accessible by the OpenTelemetry Collector.
+
- Check `create_directory` setting: if `create_directory` is set to `false`, confirm that the directory exists prior to starting the Collector. Enable `create_directory` if you want the Collector to create the directory automatically.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 [error] RedHat.TermsErrors: Use 'before' rather than 'prior to'. For more information, see RedHat.TermsErrors.

+
- Check permissions: verify that the OpenTelemetry Collector has appropriate read and write permissions for the specified directory. Adjust `directory_permissions` if necessary.

==== Issues with compaction

If compaction is not working as expected, consider the following:

.Procedure

- Verify `compaction.on_start`: if compaction should start automatically when the Collector starts, ensure `compaction.on_start` is set to `true`.
+
- Check `compaction.directory`: confirm that the `compaction.directory` is correctly set and accessible.
+
- Review `compaction.max_transaction_size`: if compaction is slow or fails, check the `compaction.max_transaction_size` value. Setting it to zero will ignore transaction size, which can help with large data volumes.

==== Storage thresholds for compaction

If compaction is not triggered as expected based on storage size, consider the following:

.Procedure

- Check `compaction.rebound_needed_threshold_mib`: ensure this value accurately reflects when storage should flag a "compaction needed" state.
+
- Adjust `compaction.rebound_trigger_threshold_mib`: if compaction is not activating as expected, adjust this threshold to better match the conditions under which compaction should trigger.
+
- Set `compaction.check_interval`: confirm the interval is sufficient for monitoring storage conditions frequently enough to detect compaction needs.

[id="oidcauth-extension_{context}"]
== OIDC Auth Extension

Expand Down