storage_controller: wait for db on startup

[jcsp]

Problem

In some dev/test environments, there aren't health checks to guarantee the database is available before starting the controller. This creates friction for the developer.

Summary of changes

• Wait up to 5 seconds for the database to become available on startup

Checklist before requesting a review

• I have performed a self-review of my code.
• If it is a core feature, I have added thorough tests.
• Do we need to implement analytics? if so did you add the relevant metrics to the dashboard?
• If this PR requires public announcement, mark it with /release-notes label and add several sentences in this section.

Checklist before merging

• Do not forget to reformat commit message to not include the above checklist

[github-actions]

2760 tests run: 2642 passed, 0 failed, 118 skipped (full report)

---

Flaky tests (1)

Postgres 16

• test_vm_bit_clear_on_heap_lock: debug

Code coverage* (full report)

• functions: 28.1% (6465 of 23045 functions)
• lines: 46.8% (45678 of 97609 lines)

* collected from Rust tests only

---

_{The comment gets automatically updated with the latest test results
cb22372 at 2024-04-23T13:09:29.855Z :recycle:}