Improve Elastic Agent integrations logging

[mbudge]

We were affected by this bug when various Elastic Agent integrations would stop sending data yet the agent appeared healthy in the Fleet UI. There were no error/debug logs in the Elastic Agent logs to show the integration failed to start or the integration had stopped working.

elastic/beats#30533

The state of the integrations could become difficult to monitor in a complex enterprise environment if the integration health isn't monitored as well as the agent health.

We're also finding it very time consuming having to log onto remote systems to collect Elastic Agent diagnostic logs, and remove all API and secret keys before proving them to support. On the prod network we don't have access to remote systems so a future improvement might be collecting Elastic Agent diagnostics through the Fleet interface in Kibana.

[botelastic]

This issue doesn't have a Team:<team> label.

[jlind23]

@mbudge thanks for raising this, can I close this one as it seems to be a duplication of #100 ?

[mbudge]

Yes as long as we can monitor the state of the integrations from a single interface.

Also collecting elastic-agent diagnostic logs is very time consuming in an enterprise environment.

[jlind23]

@nimarezainia do you know if we have any feature issue to collect agent diagnostic from fleet ui interface?