NewFactory for beatreceivers takes default home and data dirs

[leehinman]

Proposed commit message

Change NewFactory for filebeat receiver and metricbeat receiver to take default paths for home and data path. This is just the default. When the receiver is instantiated the configuration setting for path.home and path.data will take precedence if given. This will allow the otel collector to provide a "sane" default for these settings so the user doesn't have to explicitly give them in the configuration.

For example elastic-agent can set home to be https://github.com/elastic/elastic-agent/blob/e3b9b4fba4f3c0900ccb751da994da76a2218d0c/internal/pkg/agent/application/paths/common.go#L215 and data to be https://github.com/elastic/elastic-agent/blob/e3b9b4fba4f3c0900ccb751da994da76a2218d0c/internal/pkg/agent/application/paths/common.go#L196-L198

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works. Where relevant, I have used the stresstest.sh script to run them under stress conditions and race detector to verify their stability.
• I have added an entry in ./changelog/fragments using the changelog tool.

Disruptive User Impact

This does change the signature of NewFactory, callers will have to supply arguments. Giving the empty string will give the same behavior as before.

Author's Checklist

• [ ]

How to test this PR locally

1. Build elastic-agent deb package with this PR, and use suggestions from above for determining default paths
2. Install elastic-agent with deb package
3. Write a simple otel collector configuration file that uses metricbeat receiver and activemq lite module.

exporters:
  file:
    path: /<valid path>/mbreceiver.out
    flush_interval: 1s
    create_directory: true

receivers:
  metricbeatreceiver:
    metricbeat:
      modules:
        - data_stream:
            dataset: activemq.broker
          index: metrics-activemq.broker-default
          metricsets:
            - broker
          module: activemq
          period: 10s
          hosts:
            - 'localhost:8161'
          username: bob
          password: alice
    logging:
      level: debug
      selectors:
        - '*'
    output:
      otelconsumer: null
service:
  pipelines:
    logs:
      exporters:
        - file
      receivers:
        - metricbeatreceiver
  telemetry:
    metrics:
      level: detailed

4. run elastic-agent otel --config <path to config>,

With PR the activemq module will be found, without it won't.

Related issues

• Relates [beats receivers] Using Filebeat modules or Metricbeat lightweight modules requires manually setting home path to the components directory elastic-agent#11734

Use cases

Screenshots

Logs

[github-actions]

🤖 GitHub comments

Just comment with:

• run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)

[mergify]

This pull request does not have a backport label.
If this is a bug or security fix, could you label this PR @leehinman? 🙏.
For such, you'll need to label your PR with:

• The upcoming major version of the Elastic Stack
• The upcoming minor version of the Elastic Stack (if you're not pushing a breaking change)

To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

• backport-8./d is the label to automatically backport to the 8./d branch. /d is the digit
• backport-active-all is the label that automatically backports to all active branches.
• backport-active-8 is the label that automatically backports to all active minor branches for the 8 major.
• backport-active-9 is the label that automatically backports to all active minor branches for the 9 major.

[github-actions]

Workflow run 22866647362 failed in job lint (ubuntu-latest) (the macOS/Windows matrix jobs were canceled after the first failure).

Root cause from logs:

• x-pack/metricbeat/mbreceiver/receiver_test.go:251:20: net.Dial must not be called. use (*net.Dialer).DialContext (noctx)
• x-pack/metricbeat/mbreceiver/config.go:22:18: ST1016: methods on the same type should have the same receiver name (seen 1x "c", 1x "cfg")

Recommended minimal fix:

1. In x-pack/metricbeat/mbreceiver/receiver_test.go around line 251, replace net.Dial("unix", socketPath) with (&net.Dialer{}).DialContext(ctx, "unix", socketPath) (thread the context from the DialContext callback).
2. In x-pack/metricbeat/mbreceiver/config.go, use one receiver name consistently across methods (e.g. rename func (cfg *Config) Validate() to func (c *Config) Validate() and update references).

Suggested verification before re-run:

• golangci-lint run ./x-pack/metricbeat/mbreceiver/...

Tests run by me in this investigation: none locally; analysis was from Actions logs and PR diff only.

---

What is this? | From workflow: PR Actions Detective

Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.

[github-actions]

Workflow run 22869929288 (check-default) completed successfully, so there is no failing job to diagnose.

• Associated PR: #49327 (leehinman:11734_beatreciever_default_data_and_home -> elastic:main)
• Job summary: 1/1 job succeeded (check, job 66346682522)
• Failed step/root cause: none observed (no failed jobs, and failed-job log query returned zero results)

Recommended action: no fix needed for this run. If you want proactive hardening, keep this workflow as-is and monitor subsequent runs after new commits for regressions.

Tests/log checks run:

• Retrieved workflow run metadata and job conclusions via GitHub Actions API tools
• Queried failed job logs for run 22869929288 (result: no failed jobs)

---

What is this? | From workflow: PR Actions Detective

Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.

[github-actions]

golangci-lint failed in workflow run 22869929307 due to a confirmed noctx violation in lint (ubuntu-latest); macOS and Windows jobs were then cancelled.

Root cause from the log:

• x-pack/metricbeat/mbreceiver/receiver_test.go:251:20: net.Dial must not be called. use (*net.Dialer).DialContext (noctx)
• Offending line: return net.Dial("unix", socketPath)

Recommended minimal fix:

• In getFromSocket (x-pack/metricbeat/mbreceiver/receiver_test.go), change the transport dialer callback to use the provided context and net.Dialer:
  • DialContext: func(ctx context.Context, _, _ string) (net.Conn, error) {
  • return (&net.Dialer{}).DialContext(ctx, "unix", socketPath)
  • }

PR context captured:

• PR: #49327
• Author: leehinman
• Base/head: elastic/beats:main <- leehinman/beats:11734_beatreciever_default_data_and_home (fork PR)

Tests run here:

• Workflow-log analysis only (downloaded and inspected run logs); no local test/lint execution in this read-only detective pass.

---

What is this? | From workflow: PR Actions Detective

Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.

[github-actions]

I investigated workflow run 22870597186 for this PR and there is no failed check to triage.

• Workflow: pre-commit
• Job: pre-commit (66349026885) — success
• Failing jobs/steps: none found

Root cause analysis: no failure occurred in this run, so no remediation is required.

Tests/checks observed:

• GitHub Actions pre-commit workflow run completed successfully.

Follow-up:

• No action needed for this run. If a different run failed, share that run ID and I can analyze the failing step logs.

---

What is this? | From workflow: PR Actions Detective

Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.

[github-actions]

Workflow run 22870597225 (golangci-lint) completed successfully, so there is no failed check to triage.

Summary:

• PR: NewFactory for beatreceivers takes default home and data dirs #49327 (leehinman:11734_beatreciever_default_data_and_home -> elastic:main, fork PR)
• Jobs: lint (ubuntu-latest) ✅, lint (macos-latest) ✅, lint (windows-latest) ✅
• Failed jobs/steps found in run logs: none (0 failed jobs)

Root cause of failure: not applicable (no failure present).
Recommended action: no remediation needed for this run; if a different failing run is intended, share that run ID and I can analyze its failing step output.

---

What is this? | From workflow: PR Actions Detective

Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.

[github-actions]

Checked workflow run 22873440759 for this PR: there are no failures to investigate.

• Workflow: check-default
• Conclusion: success
• Job summary: 1 job (check), conclusion success
• Failed step analysis: none (no failed jobs detected)

Root cause of failure: N/A (run passed).
Recommended remediation: none required.
Tests/log checks run: reviewed run metadata, job list, and failed-job logs query (failed_only=true), which returned 0 failed jobs.

No follow-up action is needed unless a different run ID is failing.

---

What is this? | From workflow: PR Actions Detective

Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 5fa179e6-f6e1-482d-a902-044771cbfc20

📥 Commits

Reviewing files that changed from the base of the PR and between 9517fb0 and cb04ab8.

📒 Files selected for processing (4)

• x-pack/filebeat/fbreceiver/config.go
• x-pack/libbeat/cmd/instance/config.go
• x-pack/libbeat/cmd/instance/config_test.go
• x-pack/metricbeat/mbreceiver/config.go

🚧 Files skipped from review as they are similar to previous changes (3)

• x-pack/libbeat/cmd/instance/config_test.go
• x-pack/filebeat/fbreceiver/config.go
• x-pack/libbeat/cmd/instance/config.go

---

📝 Walkthrough

Walkthrough

This PR changes filebeat and metricbeat receiver factories to NewFactory(home, data string), supplying default beat.path.home and beat.path.data via the factory. Config structs now include a custom Unmarshal(conf *confmap.Conf) that calls DeDotKeys to expand dotted keys into nested maps; Beatconfig fields changed from map[string]interface{} to map[string]any. Call sites and tests were updated to pass explicit home/data arguments; new tests cover DeDotKeys and setNested behavior.

Possibly related PRs

• fix(diskqueue): use per-beat paths instead of global paths #48834: Modifies the same receiver factory API to accept explicit home/data paths and updates call sites to provide those paths.

Suggested labels

Filebeat, enhancement

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment

• 🛠️ Update Documentation: Commit on current branch
• 🛠️ Update Documentation: Create PR

Tip

Try Coding Plans. Let us write the prompt for your AI agent so you can ship faster (with fewer bugs).
Share your feedback on Discord.

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[github-actions]

Workflow run 22877915764 passed; there is no failing check to debug.

Summary of what I verified:

• Associated PR: NewFactory for beatreceivers takes default home and data dirs #49327 (leehinman:11734_beatreciever_default_data_and_home -> elastic:main, fork PR).
• Workflow: pre-commit completed with success.
• Jobs: only pre-commit (66373932487) and it completed successfully.
• Log inspection: the pre-commit step finished with Passed (including check for merge conflicts...Passed) and cleanup only; no error signatures were present.

Recommended action:

• No remediation needed for this run.
• If you want deeper validation beyond Actions, run local targeted checks for changed areas (receiver factory + tests), but there is no CI failure from this run to fix.

Tests run in this investigation:

• Local tests: not run (read-only workflow analysis).
• CI evidence reviewed: job list + job log tail for run 22877915764.

---

What is this? | From workflow: PR Actions Detective

Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.

[github-actions]

Workflow run 22877915757 completed successfully; I found no failing checks to debug.

• Associated PR: NewFactory for beatreceivers takes default home and data dirs #49327 (leehinman:11734_beatreciever_default_data_and_home -> elastic:main, fork: leehinman/beats)
• Jobs reviewed: 1 (check) with conclusion success
• Failed-job log query result: 0 failed jobs
• Log inspection (Run check-default) showed successful completion and no error/failure/panic lines

No remediation is required for this run. If you want, I can instead analyze a failing run ID for this PR.

---

What is this? | From workflow: PR Actions Detective

Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.

[elasticmachine]

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)