ESQL should support a humanize option to quickly render values in human readable way.

[philippkahr]

Description

When you deal with any sort of value that is in a unit that is not easy to comprehend as a human, like duration_in_nanos it is annoying to always have to manually perform some calculations like eval duration = duration_in_nanos / 1000 / 1000 to get to seconds resolution.

I would like to have a simple to_human(field, unit) and a to_human(field, unit, target_unit) conversion. It should support:

• Duration
• Bits (1000)
• Bytes (1024)
• Percent

We need two signatures for that function since sometimes I might not want to rely on the auto conversation and I want to fix everything to be displayed in GB. Otherwise I might end up with values of KB, MB, TB and so on in the same column.

[elasticsearchmachine]

Pinging @elastic/es-analytical-engine (Team:Analytics)

[fediazgon]

I want to work on this. I might need some time to ramp-up but I would like it to be assigned to me to avoid double work.

[nik9000]

It sounds like:

• TO_HUMAN(field, unit) - takes numeric fields, attach a unit to them, and use our standard unit rendering logic on them
• TO_HUMAN(field, unit, target_unit) - takes numeric fields, attaches a unit, then displays them in locked to a target unit

So, examples:

ROW TO_HUMAN(100, MS)
# 100ms
ROW TO_HUMAN(10000, MS)
# 10s
ROW TO_HUMAN(10000, MS, S)
# 10s
ROW TO_HUMAN(100, MS, S)
# 0.1s

Are these what you want?

Interestingly, we already have something like this in the form of date and date_nanos those are whole field types that can take millis (or nanos) since epoch. It's a whole type. So for dates the equivalent is:

ROW 10000::DATE
# 1970-01-01T00:00:10.000Z

This is, at least a little bit, a short cut to making a new duration and bytes type.

[fediazgon]

Is it really a good idea to have a one-catch all function instead of multiple functions? Users can easier discover in docs functions that are named in the form:

• formatReadableDuration.
• formatReadableSize.
• formatReadableDecimalSize.

You can also argue that, in case we wanted to add the inverse functions, they would be also easier to discover:

• parseReadableDuration.
• parseReadableSize.

It seems that a lot of DB vendors follow this approach.

[varsh00l]

Hi!!I'd love to work on this issue.
Here's my plan:
1.Implement a humanize option in ESQL to format numeric values into human readable sizes(kB,MB,GB) and durations(ms,s,min,h).
2.Follow the suggested unified function approach and add support for bytes and durations initially.
3.Add Tests to Validate formatting and ensure consistent output.
Could you please assign this issue to me?
Thanks!!

[philippkahr]

I don't have a strong opinion on either a single function or having X functions. I personally find a single thing that I need to remember much much easier than having to have lookup or rely on autocomplete which fails to show me the right function.

I might also be the wrong person because I use ES|QL only in the context of Kibana, thus it needs to look perfect and super usable in Discover and or for visualisations.

What I do have an opinion is this:

Even better would be if the to_human or humanize function could take arguments out of a field name like if the field is named abc_ms_duration it has ms in there. Therefore I can simply say to_human(abc_ms_duration) and I do not need to type the same information twice.

I would also expect the function to automatically know every ECS field and format it under the hood for us. So when I use event.duration which is in nanos, that I can type to_human(event.duration) and I do not need to specify anything, because it is specified in a schema already.

I want as least effort as there is to get good readable and useable values. Especially since I could do any of this within seconds one time in a Kibana Data View and now I would need to write it over and over again in an ESQL statement.

[fediazgon]

I am not sure about that approach because it makes it ambiguous by design (e.g., in latency_ms_ratio). I think we need some input from the team (cc @nik9000 @benwtrent). From what I can see, none of the currently supported functions take this implicit route, they all seem to favor explicit, well-defined parameters instead.

[philippkahr]

I am fine with not reading it out from the field name.

What about ECS, SemConv though? This should be baked in, it's our definition of what a field is per default. That could also cause ambiguity if someone reuses event.duration and stores it as a MS instead of Nanoseconds. Still you can always run humanize(event.duration, ms) then.

[nik9000]

This sounds like it needs to be designed before anyone starts typing stuff. It's not shovel ready. For a bunch of reasons below.

Even better would be if the to_human or humanize function could take arguments out of a field name like if the field is named abc_ms_duration it has ms in there. Therefore I can simply say to_human(abc_ms_duration) and I do not need to type the same information twice.

I think I'd personally be surprised by this behavior and find it very confusing. I imagine we could make some rules that aren't confusing, but I'd be terrified that stuff like TO_HUMAN(yams) would try to render my count of yams as a duration in millis.

What about ECS, SemConv though? This should be baked in, it's our definition of what a field is per default. That could also cause ambiguity if someone reuses event.duration and stores it as a MS instead of Nanoseconds. Still you can always run humanize(event.duration, ms) then.

I'm not entirely sure what you mean by this, but it does sound like we need to talk to someone who better understands ECS.

On the good side: once we figure out what we want none of this is particularly hard. We have functions in ES for most of these renderings already (TimeValue, ByteSizeValue, DistanceUnit, etc).

#136533 (comment) seems like a much more contained thing. Maybe we could make FORMAT_BYTE_SIZE or FORMAT_DISTANCE or FORMAT_DECIMAL_BYTE_SIZE or something. And we could design TO_HUMAN in the mean time and have it surrogate to these based on, like, rules.