fix(buffers): disk buffers v2 reopen buffer after restart crash loop

[blt]

Summary

We have observed in Antithesis scenario vector_to_vector_e2e_disk that
the disk buffer can be driven to a state where Vector will crash loop
without recovery. This happens when the reader is set to resume from a
data file that no longer exists and will not be created.

Strictly, the disk buffer reader is meant to chase the writer. This
happens because the deletion of completed data files is not atomic. When
a reader finishes an acks a data file delete_completed_data_file:

• unlinks the .dat file
• advances the reader id in the ledger
• fsyncs the ledger

If Vector crashes after the unlink but before the ledger is fsync'ed
this will cause the next instance of Vector to attempt to read a .dat
that will never appear, crashing said instance. Mostly. OS page cache
complicates this some, as the newly introduced
reopen_recovers_with_reader_resume_data_file_is_missing discusses.

Vector configuration

How did you test this PR?

Change Type

• Bug fix
• New feature
• Dependencies
• Non-functional (chore, refactoring, docs)
• Performance

Is this a breaking change?

• Yes
• No

Does this PR include user facing changes?

• Yes. Please add a changelog fragment based on our guidelines.
• No. A maintainer will apply the no-changelog label to this PR.

References

• SMPTNG-749
• Vector sink with disk buffer stuck after pod restart #19155 likely

Notes

• Please read our Vector contributor resources.
• Do not hesitate to use @vectordotdev/vector to reach out to us regarding this PR.
• Some CI checks run only after we manually approve them.
  • We recommend adding a pre-push hook, please see this template.
  • Alternatively, we recommend running the following locally before pushing to the remote branch:
    • make fmt
    • make check-clippy (if there are failures it's possible some of them can be fixed with make clippy-fix)
    • make test
• After a review is requested, please avoid force pushes to help us review incrementally.
  • Feel free to push as many commits as you want. They will be squashed into one before merging.
  • For example, you can run git merge origin master and git push.
• If this PR introduces changes Vector dependencies (modifies Cargo.lock), please
  run make build-licenses to regenerate the license inventory and commit the changes (if any). More details on the dd-rust-license-tool.

[blt]

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

• fix(buffers): disk buffers v2 reopen buffer after restart crash loop #25570 👈 (View in Graphite)
• chore(tests): Antithesis Vector end-to-end ack scenario #25571
• chore(tests): antithesis harness, durability harness scenario #25562 : 1 other dependent PR (#25525 )
• master

This stack of pull requests is managed by Graphite. Learn more about stacking.

[datadog-vectordotdev]

🎉 All green!

❄️ No new flaky tests detected
🧪 All tests passed

_{This comment will be updated automatically if new data arrives.
🔗 Commit SHA: fe305fa | Docs | Give us feedback!}