From fbc6c63d713c1d9084cf2fc6d6d0d0ed8e731573 Mon Sep 17 00:00:00 2001 From: Brian Floersch Date: Thu, 14 May 2026 17:14:12 -0400 Subject: [PATCH 1/3] Document opt-in recursive glob support for log file tailing Adds documentation for logs_config.enable_recursive_glob, available in Agent 7.76.0+, which lets file log sources match nested directories with the ** glob pattern in path and exclude_paths. Co-Authored-By: Claude Opus 4.7 (1M context) --- .../en/agent/logs/advanced_log_collection.md | 47 +++++++++++++++++++ 1 file changed, 47 insertions(+) diff --git a/content/en/agent/logs/advanced_log_collection.md b/content/en/agent/logs/advanced_log_collection.md index ece1692ce6c..cbcaad7e8f5 100644 --- a/content/en/agent/logs/advanced_log_collection.md +++ b/content/en/agent/logs/advanced_log_collection.md @@ -37,6 +37,7 @@ After you set up [log collection][1], you can customize your collection configur - [Automatically aggregate multi-line logs](#automatically-aggregate-multi-line-logs) - [Commonly used log processing rules](#commonly-used-log-processing-rules) - [Tail directories using wildcards](#tail-directories-using-wildcards) + - [Tail nested directories using recursive globs](#tail-nested-directories-using-recursive-globs) - [Prioritize tailed files by modification time](#prioritize-tailed-files-by-modification-time) - [Log file encodings](#log-file-encodings) - [Global processing rules](#global-processing-rules) @@ -615,6 +616,52 @@ The example above matches `C:\\MyApp\\MyLog.log` and excludes `C:\\MyApp\\MyLog. - The Agent requires read and execute permissions on a directory to list all the available files in it. - The path and exclude_paths values are case sensitive. +### Tail nested directories using recursive globs + +This feature requires Agent version 7.76.0 or above. + +To match files across an arbitrary number of nested directories, use the recursive glob (`**`) in the `path` or `exclude_paths` attributes. Recursive globs are opt-in and must be enabled by setting `logs_config.enable_recursive_glob` to `true` in your Agent configuration. + +Enable recursive globs in `datadog.yaml`: + +```yaml +logs_enabled: true + +logs_config: + enable_recursive_glob: true +``` + +Then use `**` in a log source's `path`: + +```yaml +logs: + - type: file + path: /var/log/myapp/**/*.log + service: mywebapp + source: go +``` + +With recursive globs enabled, the example above matches `.log` files at any depth, such as: +- `/var/log/myapp/app.log` +- `/var/log/myapp/sub/app.log` +- `/var/log/myapp/sub/nested/app.log` + +You can also use `**` in `exclude_paths` to exclude matching files at any depth: + +```yaml +logs: + - type: file + path: /var/log/myapp/**/*.log + exclude_paths: + - /var/log/myapp/**/debug.log + service: mywebapp + source: go +``` + +**Note**: +- When `logs_config.enable_recursive_glob` is not set or is `false`, paths containing `**` are not expanded and no files are matched. The default value is `false` to preserve existing behavior. +- The `open_files_limit` and `file_wildcard_selection_mode` settings described below also apply to files matched by recursive globs. + ### Prioritize tailed files by modification time This feature requires Agent version 7.40.0 or above. From 538dbfa84ddb2485c9ee736f22686d5d21f4fc67 Mon Sep 17 00:00:00 2001 From: Brian Floersch Date: Thu, 14 May 2026 17:38:46 -0400 Subject: [PATCH 2/3] Simplify recursive glob docs to a single note Co-Authored-By: Claude Opus 4.7 (1M context) --- .../en/agent/logs/advanced_log_collection.md | 48 +------------------ 1 file changed, 1 insertion(+), 47 deletions(-) diff --git a/content/en/agent/logs/advanced_log_collection.md b/content/en/agent/logs/advanced_log_collection.md index cbcaad7e8f5..56e3d8cafa3 100644 --- a/content/en/agent/logs/advanced_log_collection.md +++ b/content/en/agent/logs/advanced_log_collection.md @@ -37,7 +37,6 @@ After you set up [log collection][1], you can customize your collection configur - [Automatically aggregate multi-line logs](#automatically-aggregate-multi-line-logs) - [Commonly used log processing rules](#commonly-used-log-processing-rules) - [Tail directories using wildcards](#tail-directories-using-wildcards) - - [Tail nested directories using recursive globs](#tail-nested-directories-using-recursive-globs) - [Prioritize tailed files by modification time](#prioritize-tailed-files-by-modification-time) - [Log file encodings](#log-file-encodings) - [Global processing rules](#global-processing-rules) @@ -615,52 +614,7 @@ The example above matches `C:\\MyApp\\MyLog.log` and excludes `C:\\MyApp\\MyLog. **Note**: - The Agent requires read and execute permissions on a directory to list all the available files in it. - The path and exclude_paths values are case sensitive. - -### Tail nested directories using recursive globs - -This feature requires Agent version 7.76.0 or above. - -To match files across an arbitrary number of nested directories, use the recursive glob (`**`) in the `path` or `exclude_paths` attributes. Recursive globs are opt-in and must be enabled by setting `logs_config.enable_recursive_glob` to `true` in your Agent configuration. - -Enable recursive globs in `datadog.yaml`: - -```yaml -logs_enabled: true - -logs_config: - enable_recursive_glob: true -``` - -Then use `**` in a log source's `path`: - -```yaml -logs: - - type: file - path: /var/log/myapp/**/*.log - service: mywebapp - source: go -``` - -With recursive globs enabled, the example above matches `.log` files at any depth, such as: -- `/var/log/myapp/app.log` -- `/var/log/myapp/sub/app.log` -- `/var/log/myapp/sub/nested/app.log` - -You can also use `**` in `exclude_paths` to exclude matching files at any depth: - -```yaml -logs: - - type: file - path: /var/log/myapp/**/*.log - exclude_paths: - - /var/log/myapp/**/debug.log - service: mywebapp - source: go -``` - -**Note**: -- When `logs_config.enable_recursive_glob` is not set or is `false`, paths containing `**` are not expanded and no files are matched. The default value is `false` to preserve existing behavior. -- The `open_files_limit` and `file_wildcard_selection_mode` settings described below also apply to files matched by recursive globs. +- Starting in Agent 7.76.0, recursive globs (`**`) are supported in `path` and `exclude_paths` when you set `logs_config.enable_recursive_glob: true` in `datadog.yaml`. This is disabled by default. ### Prioritize tailed files by modification time From 6179b977e903c2a654779c2f903fb8b27f587763 Mon Sep 17 00:00:00 2001 From: Brian Floersch Date: Fri, 15 May 2026 09:26:37 -0400 Subject: [PATCH 3/3] Update content/en/agent/logs/advanced_log_collection.md Co-authored-by: Janine Chan <64388808+janine-c@users.noreply.github.com> --- content/en/agent/logs/advanced_log_collection.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/en/agent/logs/advanced_log_collection.md b/content/en/agent/logs/advanced_log_collection.md index 56e3d8cafa3..9a6d4acaee0 100644 --- a/content/en/agent/logs/advanced_log_collection.md +++ b/content/en/agent/logs/advanced_log_collection.md @@ -614,7 +614,7 @@ The example above matches `C:\\MyApp\\MyLog.log` and excludes `C:\\MyApp\\MyLog. **Note**: - The Agent requires read and execute permissions on a directory to list all the available files in it. - The path and exclude_paths values are case sensitive. -- Starting in Agent 7.76.0, recursive globs (`**`) are supported in `path` and `exclude_paths` when you set `logs_config.enable_recursive_glob: true` in `datadog.yaml`. This is disabled by default. +- Starting in Agent version 7.76.0, recursive globs (`**`) are supported in `path` and `exclude_paths` when you set `logs_config.enable_recursive_glob: true` in `datadog.yaml`. This is disabled by default. ### Prioritize tailed files by modification time