Skip to content

ESQL: Behavior of multi value fields in STATS #134792

@quackaplop

Description

@quackaplop

Today, STATS handles multi value fields that are a part of the BY grouping in the following way:

  1. Any group key is expanded and deduped for the purpose of any expression in STATS that is not inside an aggregate function
  2. Any field (including group fields) are never expanded in aggregate function, and remains multivalued.

These behaviors are consistent, however may be seen as quite confusing, particularly rule 2 - one can argue that this rule makes sense for non-group fields, but is mind boggling for group fields.

Actions required:

  • Review current behavior and decide whether this is useful for the users
  • If changes are needed, they will be breaking, so will need to decide how to drive them through (a SET option is a possibility)

Example 1

       a       |       b       
---------------+---------------
[1, 2]         |[1, 2]         
[1, 2]         |[1, 2]    

STATS count(a) by b :

   count(a)    |       b       
---------------+---------------
4              |1              
4              |2      

Here each MV is expanded, and the command behaves as if it was written as if MV_EXPAND/MV_DEDUPE was applied to each field before STATS.

Example 2

Perhaps more confusingly:

       a       
----------
[1, 2, 3] 

Query STATS a+0*SUM(a), SUM(a), a*SUM(a) BY a returns:

  a+0*SUM(a)   |    SUM(a)     |   a*SUM(a)    |       a      
---------------+---------------+---------------+---------------
1              |6              |6              |1              
2              |6              |12             |2              
3              |6              |18             |3    

Note that here we expand a as it is a group key, which results in 3 rows, however SUM(a) is actually SUM(1, 2, 3) in each row, as opposed to the current expanded value.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions