Review table_status format

[timmaxw]

The current table_status format in Raft is kind of awkward. We should think about improving it. One idea is to eliminate the state arrays by just choosing the most conservative state.

[danielmewes]

👍 on

One idea is to eliminate the state arrays by just choosing the most conservative state.

The second thing worth discussing is if we should turn the primary_replicas array for a given shard back into a single value somehow.
I'm not aware of a good way of doing that, so possibly not.

[timmaxw]

OK. I think should display whichever state is earliest in the following list: backfilling, waiting_for_quorum, waiting_for_primary, transitioning, ready, nothing. If a replica would display the nothing state then we'll simply hide it instead. There's one more state (disconnected) but it can never occur in conjunction with any other state, so it doesn't have a meaningful position in the list.

Rationale: backfilling has a performance impact so we should display it whenever it's happening. waiting_for_quorum can cause another machine to be waiting_for_primary, so we should prioritize displaying waiting_for_quorum to make it clear why the primary is stuck. transitioning is the least informative error condition, so we shouldn't display it at all unless it's the only thing keeping us from being ready.

[danielmewes]

That ordering sounds very reasonable.

[VeXocide]

I agree, the ordering sounds great. Turning the primary_replicas array into a single value is probably hard -- I don't see a good way of going about it either.

[VeXocide]

In CR 2894 by @timmaxw.

[deontologician]

I realize this is an issue about parts of a shard being in multiple states and not wanting to show that as an array because it's confusing, but it might be reasonable to use an array for the state to allow things like state: ["backfilling", "ready"] where we indicate it's ready but may have degraded performance due to backfilling.

This allows clients to just look for the presence of the "ready" state, rather than have to mirror the logic of the state ordering to determine if a shard is ready. It also means if we add new possible states in the future, the client logic wouldn't need to be updated (i.e. looser coupling to exactly which states we use internally and their ordering).

[VeXocide]

Merged into raft via commit eea2c57.