segment/unavailable/count metric is misleading during handoff

[jihoonson]

Affected Version

Probably all versions.

Description

The segment/unavailable/count metric is computed as the number of published segments that are not being served by historicals. But this is not the correct definition of "available": it should include realtime tasks as well. The effect is that the metric is misleading during handoff: it appears that unavailability spikes up before the new segments are loaded by historicals, even if all segments actually are continuously available on some combination of realtime tasks and historicals.

See https://druid.apache.org/docs/latest/design/architecture.html#indexing-and-handoff for definitions of terms that we should be using here.

[glasser]

I see this mentioned in the 0.18 release notes, but I should take your "affected version" above to mean that you don't believe this to be a recent regression?

[jihoonson]

@glasser no, I don't think it's a regression. I believe this issue has been there ever since that metric was supported.