Allow agent to monitor endpoint

[fearful-symmetry]

What does this PR do?

closes #4083

This is a large and fairly wide-ranging set of changes that allows agent to monitor and report usage metrics for endpoint.
Specifically:

• adds code to the coordinator and runtime controllers to filter PID changes up to the config system
• Adds logic to the metrics input generator to add a system/process metricset to monitor endpoint
• Add an integration test
• Change the behavior of the local env var in mage, so local=agentbeat will only build agentbeat, then pull other binaries from upstream. Needed this for testing, but kept it because it seemed vaguely useful. However, we don't need to merge it.

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works
• I have added an entry in ./changelog/fragments using the changelog tool
• I have added an integration test or an E2E test

[mergify]

This pull request does not have a backport label. Could you fix it @fearful-symmetry? 🙏
To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

• backport-v./d./d./d is the label to automatically backport to the 8./d branch. /d is the digit

NOTE: backport-skip has been added to this pull request.

[fearful-symmetry]

So, TestCoordinatorReportsInvalidPolicy is broken, trying to figure out if I broke the coordinator, or just the test.

[elasticmachine]

Pinging @elastic/elastic-agent-control-plane (Team:Elastic-Agent-Control-Plane)

[fearful-symmetry]

CI may be various kinds of broken until the snapshot beats build are updated, taking out of draft since technically everything needed is merged.

[fearful-symmetry]

Still tinkering with the tests...

[fearful-symmetry]

So, I ended up adding a decent bit of logic to the integration test, hopefully to better shake out any problems that might exist with the metric update logic. HOWEVER, I'm waiting for the elastic_agent integration to get pushed to prod in fleet, which is blocking me from making some of the test improvements Craig suggested. Should be in the next few days?

EDIT: tests are done, and working.

[blakerouse]

Overall this looks good.

I would really like to see checkinpid to just be pid, but not a blocker.

[faec]

Some suggestions on atomic handling, otherwise looks good

[mergify]

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b service-monitoring upstream/service-monitoring
git merge upstream/main
git push upstream service-monitoring

[elastic-sonarqube]

Quality Gate passed

Issues
7 New issues
1 Fixed issue
0 Accepted issues

Measures
0 Security Hotspots
76.3% Coverage on New Code
7.1% Duplication on New Code

See analysis details on SonarQube