store: block until Ceph is healthy

[carmstrong]

When starting store components, we have accounted for dependencies and used a specific start order as of #2307. However, we still have the case when all of the daemons are up, but the cluster isn't yet healthy - sometimes (especially during upgrades), some recovery is necessary (see #2300 (comment)) which may happen automatically with time. A better solution here rather than letting the store-volume service fail and restart a few times before it continues, we could have a check_store_health script which scripts could block on in their ExecStartPre to wait until the cluster reports HEALTH_OK.

There would still be cases where manual intervention would be necessary, but we may be able to do some grepping of the ceph -s output and suggest a remediation for the user rather than letting them wonder what's going on.

[carmstrong]

After giving this more thought, I'm not sure the little clarify is worth the effort involved. We would really only be able to say the same thing that ceph -s does.

[carmstrong]

Closing - the components will continue to restart until the cluster is healthy, so I really don't think this work is necessary.