Skip to content

Add filebeat_input to beat receiver agent monitoring integration test #8338

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Jun 12, 2025

Conversation

leehinman
Copy link
Contributor

What does this PR do?

Adds last part of comparing process based monitoring vs beat receiver based monitoring

  • Adds Windows as test case
  • adds filebeat_input metrics to config
  • adds filebeat_input metrics to integration test

Why is it important?

We need to make sure that both implementations produce the same documents.

Checklist

  • I have read and understood the pull request guidelines of this project.
  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in ./changelog/fragments using the changelog tool
  • I have added an integration test or an E2E test

Disruptive User Impact

None. Unit test and changes to producing configuration when experimental runtime is used.

How to test this PR locally

mage integration:single TestClassicAndReceiverAgentMonitoring

Related issues

Questions to ask yourself

  • How are we going to support this in production?
  • How are we going to measure its adoption?
  • How are we going to debug this?
  • What are the metrics I should take care of?
  • ...

@leehinman leehinman requested a review from a team as a code owner June 4, 2025 18:11
Copy link
Contributor

mergify bot commented Jun 4, 2025

This pull request does not have a backport label. Could you fix it @leehinman? 🙏
To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

  • backport-./d./d is the label that automatically backports to the 8./d branch. /d is the digit
  • backport-active-all is the label that automatically backports to all active branches.
  • backport-active-8 is the label that automatically backports to all active minor branches for the 8 major.
  • backport-active-9 is the label that automatically backports to all active minor branches for the 9 major.

@leehinman leehinman requested review from khushijain21, mauri870 and ycombinator and removed request for ycombinator and michel-laterman June 4, 2025 18:12
@leehinman leehinman added backport-9.0 Automated backport to the 9.0 branch backport-8.19 Automated backport to the 8.19 branch skip-changelog labels Jun 4, 2025
Copy link
Contributor

@swiatekm swiatekm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't get why changes to the existing test cases are necessary in this PR. Why does uncommenting the filebeat input test case not suffice?

@leehinman
Copy link
Contributor Author

I don't get why changes to the existing test cases are necessary in this PR. Why does uncommenting the filebeat input test case not suffice?

2 things.

  1. Because I found that the single query case wasn't sufficient, so I changed it to a slice
query           map[string]any
to
query           []map[string]any
  1. Added Windows

@pierrehilbert pierrehilbert added Team:Elastic-Agent-Control-Plane Label for the Agent Control Plane team Team:Elastic-Agent-Data-Plane Label for the Agent Data Plane team labels Jun 6, 2025
@elasticmachine
Copy link
Collaborator

Pinging @elastic/elastic-agent-control-plane (Team:Elastic-Agent-Control-Plane)

@elasticmachine
Copy link
Collaborator

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

Copy link
Contributor

mergify bot commented Jun 6, 2025

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b 4876_agent_monitoring_tests upstream/4876_agent_monitoring_tests
git merge upstream/main
git push upstream 4876_agent_monitoring_tests

@leehinman leehinman force-pushed the 4876_agent_monitoring_tests branch from f6d024a to c786566 Compare June 9, 2025 18:44
@leehinman
Copy link
Contributor Author

@mauri870 @swiatekm @khushijain21 please take another look, should be clearer now.

@swiatekm
Copy link
Contributor

@leehinman you need to update the monitoring config generation unit tests.

@leehinman leehinman force-pushed the 4876_agent_monitoring_tests branch from c786566 to d38eed5 Compare June 10, 2025 14:45
@leehinman
Copy link
Contributor Author

@leehinman you need to update the monitoring config generation unit tests.

🤦 done.

@swiatekm
Copy link
Contributor

FYI, the serverless tests are failing due to an unrelated infrastructure problem right now.

swiatekm
swiatekm previously approved these changes Jun 11, 2025
mauri870
mauri870 previously approved these changes Jun 11, 2025
@leehinman leehinman force-pushed the 4876_agent_monitoring_tests branch from d38eed5 to b16e612 Compare June 11, 2025 14:22
@swiatekm
Copy link
Contributor

Looks like the modified test fails right now.

@leehinman leehinman dismissed stale reviews from mauri870 and swiatekm via 0e2024f June 11, 2025 16:47
@leehinman leehinman force-pushed the 4876_agent_monitoring_tests branch from 0e2024f to d403bf9 Compare June 11, 2025 18:45
Copy link

@elasticmachine
Copy link
Collaborator

💛 Build succeeded, but was flaky

Failed CI Steps

History

cc @leehinman

@swiatekm swiatekm removed the backport-9.0 Automated backport to the 9.0 branch label Jun 12, 2025
Copy link
Contributor

@khushijain21 khushijain21 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me!

@leehinman leehinman merged commit b8882aa into elastic:main Jun 12, 2025
12 checks passed
mergify bot pushed a commit that referenced this pull request Jun 12, 2025
…#8338)

(cherry picked from commit b8882aa)

# Conflicts:
#	testing/integration/beat_receivers_test.go
swiatekm pushed a commit that referenced this pull request Jun 13, 2025
…#8338)

(cherry picked from commit b8882aa)

# Conflicts:
#	testing/integration/beat_receivers_test.go
swiatekm pushed a commit that referenced this pull request Jun 16, 2025
…#8338)

(cherry picked from commit b8882aa)

# Conflicts:
#	testing/integration/beat_receivers_test.go
swiatekm pushed a commit that referenced this pull request Jun 18, 2025
…#8338)

(cherry picked from commit b8882aa)

# Conflicts:
#	testing/integration/beat_receivers_test.go
swiatekm pushed a commit that referenced this pull request Jun 20, 2025
…#8338)

(cherry picked from commit b8882aa)

# Conflicts:
#	testing/integration/beat_receivers_test.go
leehinman added a commit that referenced this pull request Jun 24, 2025
…#8338)

(cherry picked from commit b8882aa)

# Conflicts:
#	testing/integration/beat_receivers_test.go
swiatekm added a commit that referenced this pull request Jun 24, 2025
…itoring integration test (#8473)

* Add filebeat_input to beat receiver agent monitoring integration test (#8338)

(cherry picked from commit b8882aa)

# Conflicts:
#	testing/integration/beat_receivers_test.go

* Fix conflicts

---------

Co-authored-by: Lee E Hinman <57081003+leehinman@users.noreply.github.com>
Co-authored-by: Mikołaj Świątek <mail@mikolajswiatek.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport-8.19 Automated backport to the 8.19 branch skip-changelog Team:Elastic-Agent-Control-Plane Label for the Agent Control Plane team Team:Elastic-Agent-Data-Plane Label for the Agent Data Plane team
Projects
None yet
Development

Successfully merging this pull request may close these issues.

runtime_experimental: otel does not gather elastic_agent.filebeat_input datastream
6 participants