Skip to content

otel(packetbeat): add OTel status reporting and integration tests#51428

Merged
mauri870 merged 11 commits into
elastic:mainfrom
mauri870:otel-packetbeat-wiring
Jun 23, 2026
Merged

otel(packetbeat): add OTel status reporting and integration tests#51428
mauri870 merged 11 commits into
elastic:mainfrom
mauri870:otel-packetbeat-wiring

Conversation

@mauri870

@mauri870 mauri870 commented Jun 22, 2026

Copy link
Copy Markdown
Member

Proposed commit message

Thread the OTel status factory wrapper so the receiver can report per-input StatusOK events to the OTel host. Add SetStatusReporter to processor so the wrapper can inject a sub-reporter after runner creation.

Also add integration test coverage for the packetbeat OTel receiver:

  • Rewrite pbreceiver unit tests to replay pre-recorded pcap files (no root/npcap required); add MockHost event history so TestReceiverStatus can verify StatusOK was reported even when the pcap finishes fast

  • Add docker-compose.yml, GoIntegTest mage target, and CI pipeline step

  • Add tests/integration/otel_test.go with pcap-based E2E tests (single receiver, multiple receivers, standalone-vs-receiver ES comparison) and explicit live-interface tests that skip without root

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works. Where relevant, I have used the stresstest.sh script to run them under stress conditions and race detector to verify their stability.
  • I have added an entry in ./changelog/fragments using the changelog tool.

How to test this PR locally

cd x-pack/packetbeat
mage docker:composeUp
mage goIntegTest

# Live tests that don't run on CI, capturing packets from a network interface (requires sudo)
(go test -c -tags integration -o x-pack/packetbeat/tests/integration/pb_integration.test ./x-pack/packetbeat/tests/integration/ && cd x-pack/packetbeat/tests/integration && sudo ./pb_integration.test -test.run 'TestPacketbeatOTelLiveInterfaceE2E|TestPacketbeatOTelLiveInterfaceMultipleReceiversE2E' && echo $?)

Related issues

Thread the OTel status factory wrapper so the receiver can report
per-input StatusOK events to the OTel host. Add SetStatusReporter
to processor so the wrapper can inject a sub-reporter after runner
creation.

Also add integration test coverage for the packetbeat OTel receiver:

- Rewrite pbreceiver unit tests to replay pre-recorded pcap files (no
  root/npcap required); add MockHost event history so TestReceiverStatus
  can verify StatusOK was reported even when the pcap finishes fast

- Add docker-compose.yml, GoIntegTest mage target, and CI pipeline step

- Add tests/integration/otel_test.go with pcap-based E2E tests (single
  receiver, multiple receivers, standalone-vs-receiver ES comparison)
  and explicit live-interface tests that skip without root
@mauri870 mauri870 self-assigned this Jun 22, 2026
@mauri870 mauri870 added enhancement backport-skip Skip notification from the automated backport with mergify Team:Elastic-Agent-Data-Plane Label for the Agent Data Plane team skip-changelog labels Jun 22, 2026
@botelastic botelastic Bot added needs_team Indicates that the issue/PR needs a Team:* label and removed needs_team Indicates that the issue/PR needs a Team:* label labels Jun 22, 2026
@github-actions

Copy link
Copy Markdown
Contributor

🤖 GitHub comments

Just comment with:

  • run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)
  • /test : Run the Buildkite pipeline.

@mauri870 mauri870 marked this pull request as ready for review June 22, 2026 18:35
@mauri870 mauri870 requested review from a team as code owners June 22, 2026 18:35
@mauri870 mauri870 requested review from leehinman and rdner June 22, 2026 18:35
@infra-vault-gh-plugin-prod

Copy link
Copy Markdown

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

@coderabbitai

coderabbitai Bot commented Jun 22, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

The PR wires OTel status reporting through the Packetbeat beater layer: packetbeat.go stores the beat's logger, builds a cfgfile.RunnerFactory with optional OTel wrapping, and passes it into refactored runStatic/runManaged (both now accept the interface type); runManaged replaces Manager.Start() with PreInit/PostInit. processor.go adds a sync.RWMutex around status reporter access. reloader.go accepts cfgfile.RunnerFactory. The oteltest.MockHost gains full event history tracking and CheckReceivers validates against that history. pbreceiver tests switch to pcap-based inputs with LogsSink assertions and a new TestReceiverStatus. A new E2E integration suite, Docker Compose setup, GoIntegTest mage target, and CI pipeline step complete the change.

🚥 Pre-merge checks | ✅ 2
✅ Passed checks (2 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • 🛠️ Update Documentation

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai

coderabbitai Bot commented Jun 23, 2026

Copy link
Copy Markdown
Contributor

Caution

Failed to replace (edit) comment. This is likely due to insufficient permissions or the comment being deleted.

Error details
{}

@mauri870

Copy link
Copy Markdown
Member Author

@elastic/sec-linux-platform Can I get a review from your team? Thanks.

Comment thread x-pack/packetbeat/magefile.go
@mauri870 mauri870 requested a review from nicholasberlin June 23, 2026 14:14
Comment thread packetbeat/beater/packetbeat.go
Comment thread x-pack/packetbeat/tests/integration/otel_test.go
@mauri870 mauri870 requested a review from nicholasberlin June 23, 2026 15:43
Comment thread packetbeat/beater/packetbeat.go Outdated
@mauri870 mauri870 requested a review from nicholasberlin June 23, 2026 16:17

@nicholasberlin nicholasberlin left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM (and the bot)

@mauri870 mauri870 enabled auto-merge (squash) June 23, 2026 19:35
@mauri870

Copy link
Copy Markdown
Member Author

@leehinman Could you take a look when you have a chance? Thanks!

@mauri870 mauri870 merged commit 2706a77 into elastic:main Jun 23, 2026
76 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport-skip Skip notification from the automated backport with mergify enhancement skip-changelog Team:Elastic-Agent-Data-Plane Label for the Agent Data Plane team

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants