diff --git a/observability/otel/otel-collector/otel-collector-extensions.adoc b/observability/otel/otel-collector/otel-collector-extensions.adoc index daaa9f3b59fe..7d73c302f7fa 100644 --- a/observability/otel/otel-collector/otel-collector-extensions.adoc +++ b/observability/otel/otel-collector/otel-collector-extensions.adoc @@ -161,6 +161,107 @@ include::snippets/technology-preview.adoc[] <7> Buffers the OTLP Exporter data on the local file system. <8> Starts the File Storage Extension by the Collector. +.Parameters used by the File Storage Extension +[options="header"] +[cols="a,a,a"] +|=== +|Parameter |Description |Default + +|`directory` +|The relative or absolute path to the dedicated data storage directory. The default is `%ProgramData%\Otelcol\FileStorage` on Windows and `/var/lib/otelcol/file_storage` otherwise. +|Platform-dependent + +|`timeout` +|Maximum time to wait for a file lock, typically does not need modification. +|`1s` + +|`fsync` +|When enabled, forces the database to perform an `fsync` after each write for data integrity, at the cost of performance. +|`false` + +|`create_directory` +|Creates the data storage and compaction directory if it does not already exist, with default permissions of `0750`. +|`false` + +|`directory_permissions` +|Sets custom permissions for directory creation if `create_directory` is enabled. +|`0750` + +|`compaction.on_start` +|Triggers compaction when the Collector starts. +|`false` + +|`compaction.on_rebound` +|Triggers online (rebound) compaction when certain conditions are met (see details below). +|`false` + +|`compaction.directory` +|Specifies the directory used as a staging area during compaction. +|`/tmp/` + +|`compaction.max_transaction_size` +|Defines the maximum size for a compaction transaction. Set to zero to ignore transaction sizes. +|`65536` + +|`compaction.cleanup_on_start` +|Removes any temporary compaction files left from a previous interrupted compaction process. +|`false` + +|`compaction.rebound_needed_threshold_mib` +|If storage exceeds this threshold, the "compaction needed" flag is enabled, indicating potential need for rebound compaction. +|`100` + +|`compaction.rebound_trigger_threshold_mib` +|When the "compaction needed" flag is set and storage falls below this threshold, compaction is triggered. +|`10` + +|`compaction.check_interval` +|Frequency at which conditions for compaction are checked. +|`5s` +|=== + +=== Troubleshooting + +The File Storage Extension supports storing telemetry data locally and requires read and write access to a directory. The extension can use a default or custom directory, but the directory must exist if `create_directory` is set to `false`. + +When troubleshooting components that use the File Storage extension, it can be helpful to examine the raw contents of the files generated by the extension. The easiest way to read these files is by using the `strings` utility, available on both link:https://man7.org/linux/man-pages/man1/strings.1.html[Linux] and link:https://learn.microsoft.com/en-us/sysinternals/downloads/strings[Windows]. + +==== Telemetry data not being stored or persisted + +If telemetry data is not being persisted as expected, consider the following: + +.Procedure + +- Verify the `directory`: ensure that the directory specified in `directory` exists and is accessible by the OpenTelemetry Collector. ++ +- Check `create_directory` setting: if `create_directory` is set to `false`, confirm that the directory exists prior to starting the Collector. Enable `create_directory` if you want the Collector to create the directory automatically. ++ +- Check permissions: verify that the OpenTelemetry Collector has appropriate read and write permissions for the specified directory. Adjust `directory_permissions` if necessary. + +==== Issues with compaction + +If compaction is not working as expected, consider the following: + +.Procedure + +- Verify `compaction.on_start`: if compaction should start automatically when the Collector starts, ensure `compaction.on_start` is set to `true`. ++ +- Check `compaction.directory`: confirm that the `compaction.directory` is correctly set and accessible. ++ +- Review `compaction.max_transaction_size`: if compaction is slow or fails, check the `compaction.max_transaction_size` value. Setting it to zero will ignore transaction size, which can help with large data volumes. + +==== Storage thresholds for compaction + +If compaction is not triggered as expected based on storage size, consider the following: + +.Procedure + +- Check `compaction.rebound_needed_threshold_mib`: ensure this value accurately reflects when storage should flag a "compaction needed" state. ++ +- Adjust `compaction.rebound_trigger_threshold_mib`: if compaction is not activating as expected, adjust this threshold to better match the conditions under which compaction should trigger. ++ +- Set `compaction.check_interval`: confirm the interval is sufficient for monitoring storage conditions frequently enough to detect compaction needs. + [id="oidcauth-extension_{context}"] == OIDC Auth Extension