x-pack/plugin/apm: introduce x-pack-apm plugin

[axw]

Introduce the x-pack-apm plugin, which installs index templates, component templates, and ingest pipelines for Elastic APM.

This replaces the need for installing any Fleet integration package for Elastic APM, unless central configuration/fleet management is desired, simplifying setup for users of the apm-server binary. This will also simplify APM Server by removing the possibility of index templates being missing on startup.

[felixbarny]

How do we ensure rollovers when settings or mappings change? Does this PR depend on #96521?

[axw]

@felixbarny we don't. We would either need #96521, or do it in the plugin:

https://github.com/elastic/elasticsearch/pull/97546/files#diff-652b74c665f6638e6113270681056a55bca105de6a22a297f83c0312634c841dR45-R47

// TODO(axw) when the template version is updated, data streams using the old
// version should be rolled over after upgrading. Ideally this would be done
// automatically: #96521

😄

[axw]

@danielmitterdorfer provided some helpful pointers on Slack:

#97380 adds support for upgrading Universal Profiling data streams and indices. Maybe we can take some inspiration from there to address #96521?

#97773 adds support for migrations. I haven't looked at all the details, but perhaps this could be useful for adding runtime fields to old backing indices for schema evolution.

[eyalkoren]

@jbaiera I addressed all review comments. As agreed with @axw and @felixbarny, full integration tests for APM components will be added in a followup effort. I removed the rollover test from the APM data plugin and added a simple one that tests the basic functionality to core instead.
Please go over the non-resolved comments and see if you can resolve them. Thanks for your great feedback 🙏

[jbaiera]

Thanks for iterating on this! Given that the index template registry refactoring is on the todo list for future consideration, I think things look good to me!

[axw]

Thank you for finishing this up @eyalkoren, and for your support @jbaiera! ❤️