Normalize waffle map values

[exekias]

When using Waffle visualization with a metric that doesn't spawn across its full range, it would be nice to normalize existing values to make use of the whole color gradient.

As an example, values for this graph go between 0 and ~10%, where they can potentially go up to 100%. This results in only gray to light blue colors being used:

By normalizing values range, elements in the map can use the whole color spectrum, giving more visual information to the user.

cc @gingerwizard

[elasticmachine]

Pinging @elastic/infrastructure-ui

[kaiyan-sheng]

Just curious, is there any other place in Kibana that uses color spectrum does normalization?

[weltenwort]

The only ones I can think of are the heatmap visualization and the "blob" colors of the tile map visualization. In most other cases the color is picked from a palette AFAIK.

[roncohen]

personally, my preference would be to have 10% show up as it is today. IMO we should try to use bright red only when it calls for attention, whenever possible. That's just my preference though.

When we don't have a max. value (for example for network or disk i/o), there's no good option, and we have to choose the max as the top value we've seen.

Since the infrastructure overview here is a snapshot in time, when looking at non-percentage metrics, an option could be to take the max. value seen for a longer time period. E.g. we're using the metrics from the current snapshot in time, but the max. value is set from all the values going back an hour.

This would potentially make it so that it's more rare to see something red unless there's an actual problem, but could also be quite confusing and complicated to implement.

[makwarth]

On the colors: We don't really need green/yellow/red indicators, right? As I see it, we just need some visual indicator to highlight difference between instances. How about choosing a neutral gradient, like light grey to saturated blue instead of green-yellow-red?

Once we we have user defined thresholds or automated ML bounds, we can use green-yellow-red.

[kaiyan-sheng]

I think a more neutral gradient for the color bar will work better since red/yellow color might give false alarms to users.

[gingerwizard]

It’s entirely possible 90%+ cpu is intended and in a lot of cases desirable given it shows good utilization of hardware. In a lot of usecases, good usage profile is everything is approximately the same. Although I’d prefer neutral colors, as high CPU isn’t bad (only if it’s unusual), it would still struggle to differentiate in this case. A button to normalize within the range would allow the users to switch to a relative view and back to an absolute view - both are equally useful.

[weltenwort]

Aside from the semantic interpretation of the colors there are also accessibility concerns to keep in mind (see the feedback in https://discuss.elastic.co/t/infrastructure-ui-colouring-feedback/159152).

[simianhacker]

Quick update, the colors have been updated to a gradient from gray to blue (6.7 via #28206) to eliminate any qualification of good or bad based on the color.

As far as the bounds for the data, are we still looking for a way to change the bounds from 0% to 100% on the percent based metrics?

@roncohen @makwarth @formgeist Where should we add a control for this? Maybe have some kind of "cog" settings icon near the legend, all the way to the right.

It would be pretty pretty trivial to implement this change and not too difficult to add the ability to choose a different color (if we wanted to). In the back of my mind I always had the idea of allowing the user to customize the legend by changing colors, adding their own color stops, changing the bounds.

[formgeist]

Hey Chris,

I think placing a settings cog wheel in that area makes sense. Keep it close to the element you're messing with. I imagine the cog wheel invokes a popover with the bounds and colour settings? Not sure if you're aware of the EUI Color Palettes utility that allows one to generate i.e. gradient maps like these pretty easily. I think it was the Canvas team that created it. I tend to think to have a few colour map options rather than allowing for complete custom colours.

[simianhacker]

@formgeist @makwarth How about we just stick with the custom bounds for now? We can add more enhancements later for the color.

[makwarth]

@simianhacker SGTM