tests: sync_after_each_test -> sync_between_tests

[jcsp]

Problem

We are seeing frequent pageserver startup timelines while it calls syncfs(). There is an existing fixture that syncs after tests, but not before the first one. We hypothesize that some failures are happening on the first test in a job.

Summary of changes

• extend the existing sync_after_each_test to be a sync between all tests, including sync'ing before running the first test. That should remove any ambiguity about whether the sync is happening on the correct node.

This is an alternative to #8957 -- I didn't realize until I saw Alexander's comment on that PR that we have an existing hook that syncs filesystems and can be extended.

Checklist before requesting a review

• I have performed a self-review of my code.
• If it is a core feature, I have added thorough tests.
• Do we need to implement analytics? if so did you add the relevant metrics to the dashboard?
• If this PR requires public announcement, mark it with /release-notes label and add several sentences in this section.

Checklist before merging

• Do not forget to reformat commit message to not include the above checklist

[github-actions]

5110 tests run: 4942 passed, 0 failed, 168 skipped (full report)

---

Flaky tests (1)

Postgres 17

• test_pageserver_compaction_smoke: release-arm64

Code coverage* (full report)

• functions: 31.4% (7489 of 23887 functions)
• lines: 49.6% (60094 of 121239 lines)

* collected from Rust tests only

---

_{The comment gets automatically updated with the latest test results
75899c7 at 2024-10-02T15:38:59.444Z :recycle:}

[jcsp]

This made it through two benchmark runs without failures, so cautiously optimistic that it at least does no harm

[bayandin]

LGTM, just a minor style-related comment

[koivunej]

Let's try this.

Per #8957 (comment) the earlier attempt didn't look promising though. Because we only run single process with I don't see why this would be different, but lets try.