[Elastic Agent] Implement the shipper PersistedIndex API in the Filebeat shipper input

[cmacknz]

This is a follow up from #35135. The shipper event protocol defines a PersistedIndex API for end to end acknowledgement from the output back to the input. This is critically important to ensuring that the Filebeat registry will continue to function as designed when using the shipper.

The shipper PersistedIndex API is described in detail in https://github.com/elastic/elastic-agent-shipper-client/tree/main/api#publication-acknowledgements. Implement the PersistedIndex API in the Filebeat shipper input.

Acceptance Criteria

• The end to end acknowledgement integration tests continue to pass with the Filebeat shipper input: Add integration tests for end to end acknowledgement elastic-agent-shipper#264
• An automated test exists to prove that the Filebeat registry continues to function as designed.

[elasticmachine]

Pinging @elastic/elastic-agent (Team:Elastic-Agent)

[botelastic]

Hi!
We just realized that we haven't looked into this issue in a while. We're sorry!

We're labeling this issue as Stale to make it hit our filters and make sure we get back to it as soon as possible. In the meantime, it'd be extremely helpful if you could take a look at it as well and confirm its relevance. A simple comment with a nice emoji will be enough :+1.
Thank you for your contribution!