[8.19](backport #49734) Fix: Implement backup_to_bucket_ and delete_after_backup for aws-s3 polling mode

[mergify]

When Filebeat aws-s3 input runs in polling mode (e.g. non_aws_bucket_name for MinIO/S3-compatible), it can fully process an object (events published and registry state stored) but never perform backup_to_bucket_* and delete_after_backup.

User-visible symptom

• The object is parsed and events are published.
• The registry shows the object state as stored: true.
• S3 traces show no CopyObject / DeleteObject calls for the processed object.

Root cause (bug)

Object finalization (backup/delete) is implemented in (*s3ObjectProcessor).FinalizeS3Object() in s3_objects.go.

• In the SQS execution path (sqs_s3_event.go), FinalizeS3Object() is collected and executed after successful processing / ACK.
• In the poller execution path (s3_input.go), the ACK callback used to only checkpoint state (in.registry.AddState(state)) and increment metrics. It did not call FinalizeS3Object().

As a result, polling mode could mark objects as completed in the registry without ever copying/deleting them.

Fix

In s3_input.go (poller worker loop), call FinalizeS3Object() inside the ACK callback after all events for the object are ACKed, and only when processing succeeded (state.Stored == true).

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works. Where relevant, I have used the stresstest.sh script to run them under stress conditions and race detector to verify their stability.
• I have added an entry in ./changelog/fragments using the changelog tool.

Disruptive User Impact

None. It fixes the problem.

How to test this PR locally

• Configure filebeat input

filebeat.inputs:
  - type: aws-s3
    # Polling mode (AWS S3)
    bucket_arn: "arn:aws:s3:::my-source-bucket"
    region: "us-east-1"
    number_of_workers: 5
    bucket_list_interval: 60s
    bucket_list_prefix: ""
    access_key_id: "*******"
    secret_access_key: "*******"
    ignore_older: 168h
    # Backup + delete after ACK
    backup_to_bucket_arn: "arn:aws:s3:::my-backup-bucket"
    backup_to_bucket_prefix: "processed/"
    delete_after_backup: true

• Confirm the source object is removed and a copy exists under the configured backup prefix.

Related issues

• Closes Filebeat/input aws-s3: processed objects not backed up and not deleted #46672

Use cases

Screenshots

Logs

---

This is an automatic backport of pull request #49734 done by [Mergify](https://mergify.com).

[mergify]

Cherry-pick of e3ddd67 has failed:

On branch mergify/bp/8.19/pr-49734
Your branch is up to date with 'origin/8.19'.

You are currently cherry-picking commit e3ddd67c0.
  (fix conflicts and run "git cherry-pick --continue")
  (use "git cherry-pick --skip" to skip this patch)
  (use "git cherry-pick --abort" to cancel the cherry-pick operation)

Changes to be committed:
	new file:   changelog/fragments/1783515049-fix-s3-poller-backup-delete.yaml

Unmerged paths:
  (use "git add <file>..." to mark resolution)
	both modified:   x-pack/filebeat/input/awss3/s3_input.go
	both modified:   x-pack/filebeat/input/awss3/s3_test.go

To fix up this pull request, you can check it out locally. See documentation: https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/checking-out-pull-requests-locally

[github-actions]

🤖 GitHub comments

Just comment with:

• run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)
• /test : Run the Buildkite pipeline.

[infra-vault-gh-plugin-prod]

Pinging @elastic/obs-ds-hosted-services (Team:obs-ds-hosted-services)