Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Filestream inputs with duplicated ID cause Filebeat to panic #35964

Closed
belimawr opened this issue Jun 29, 2023 · 1 comment · Fixed by #35972
Closed

Filestream inputs with duplicated ID cause Filebeat to panic #35964

belimawr opened this issue Jun 29, 2023 · 1 comment · Fixed by #35972
Labels
bug needs_team Indicates that the issue/PR needs a Team:* label

Comments

@belimawr
Copy link
Contributor

First things first: having two Filestream inputs configured with the same ID will lead to data duplication, this is not a valid configuration.

  • Version: main, 8.9 branch
  • Operating System: all

Steps to Reproduce

  1. Create a filebeat.yml with the following content
    filebeat:
      inputs:
        - type: filestream
          id: foo
          paths:
            - /tmp/foo
        - type: filestream
          id: foo
          paths:
            - /tmp/foo2
    output:
      file:
        enabled: true
  2. Run Filebeat
  3. It will panic:
panic: name foo already used

goroutine 90 [running]:
github.com/elastic/elastic-agent-libs/monitoring.panicErr(...)
        /home/tiago/go/pkg/mod/github.com/elastic/elastic-agent-libs@v0.3.9/monitoring/registry.go:257
github.com/elastic/elastic-agent-libs/monitoring.(*Registry).Add(0xc0003a9840, {0xc0003a46b8, 0x0}, {0x45f3b60, 0xc00072d380}, 0x30)
        /home/tiago/go/pkg/mod/github.com/elastic/elastic-agent-libs@v0.3.9/monitoring/registry.go:155 +0xda
github.com/elastic/elastic-agent-libs/monitoring.(*Registry).NewRegistry(0xc0003a9840, {0xc0003a46b8, 0x3}, {0x0?, 0x0?, 0x4078f36?})
        /home/tiago/go/pkg/mod/github.com/elastic/elastic-agent-libs@v0.3.9/monitoring/registry.go:94 +0x165
github.com/elastic/beats/v7/libbeat/monitoring/inputmon.NewInputRegistry({0x407584e, 0xa}, {0xc0003a46b8, 0x3}, 0xc00013bb10?)
        /home/tiago/devel/beats/libbeat/monitoring/inputmon/input.go:64 +0x3e6
github.com/elastic/beats/v7/filebeat/input/filestream/internal/input-logfile.NewMetrics({0xc0003a46b8, 0x3})
        /home/tiago/devel/beats/filebeat/input/filestream/internal/input-logfile/metrics.go:66 +0xa5
github.com/elastic/beats/v7/filebeat/input/filestream/internal/input-logfile.(*managedInput).Run(_, {0xc000488c30, {0xc0003a46b8, 0x3}, {{0x40708ec, 0x8}, {0x40708ec, 0x8}, {0x406b23e, 0x6}, ...}, ...}, ...)
        /home/tiago/devel/beats/filebeat/input/filestream/internal/input-logfile/input.go:64 +0x1bd
github.com/elastic/beats/v7/filebeat/input/v2/compat.(*runner).Start.func1()
        /home/tiago/devel/beats/filebeat/input/v2/compat/compat.go:122 +0x215
created by github.com/elastic/beats/v7/filebeat/input/v2/compat.(*runner).Start
        /home/tiago/devel/beats/filebeat/input/v2/compat/compat.go:119 +0xd8

The problem comes from registering metrics for Filestream inputs using the ID:

reg, unreg := inputmon.NewInputRegistry("filestream", id, nil)

when the second input tries to registry its metrics, the monitoring/metrics code panics.

We need to find a way to detect this case and prevent the panic from happening.

@belimawr belimawr added the bug label Jun 29, 2023
@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Jun 29, 2023
@botelastic
Copy link

botelastic bot commented Jun 29, 2023

This issue doesn't have a Team:<team> label.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug needs_team Indicates that the issue/PR needs a Team:* label
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant