Skip to content

Commit

Permalink
Add global ordinal info to stats APIs. (#94500)
Browse files Browse the repository at this point in the history
This change adds:
* Total global ordinal build time for all fields and per field.
* Max shard value count per field. The value count is per shard and of the shard with the highest count. Reporting value on index level or across indices is too expensive to report or keep track of.

This is added to common stats, which
is exposed in several stats APIs.

The following api call:

```
GET /_nodes/stats?filter_path=nodes.*.indices.fielddata&fields=key,key2
```

Returns:

```
{
    "nodes": {
        "pcMNy4GsQ8ef6Rw-bI2EFg": {
            "indices": {
                "fielddata": {
                    "memory_size_in_bytes": 2552,
                    "evictions": 0,
                    "fields": {
                        "key2": {
                            "memory_size_in_bytes": 1320
                        },
                        "key": {
                            "memory_size_in_bytes": 1232
                        }
                    },
                    "global_ordinals": {
                        "build_time_in_millis": 8,
                        "fields": {
                            "key2": {
                                "build_time_in_millis": 4,
                                "shard_max_value_count": 4
                            },
                            "key": {
                                "build_time_in_millis": 4,
                                "shard_max_value_count": 4
                            }
                        }
                    }
                }
            }
        }
    }
}
```
  • Loading branch information
martijnvg committed Apr 24, 2023
1 parent 69951fd commit 6566bb4
Show file tree
Hide file tree
Showing 19 changed files with 706 additions and 34 deletions.
5 changes: 5 additions & 0 deletions docs/changelog/94500.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
pr: 94500
summary: Add global ordinal info to stats APIs
area: Aggregations
type: enhancement
issues: []
26 changes: 25 additions & 1 deletion docs/reference/cluster/stats.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -285,6 +285,26 @@ shards assigned to selected nodes.
(integer)
Total number of evictions from the field data cache across all shards assigned
to selected nodes.

`global_ordinals.build_time`::
(<<time-units,time unit>>)
The total time spent building global ordinals for all fields.

`global_ordinals.build_time_in_millis`::
(integer)
The total time, in milliseconds, spent building global ordinals for all fields.

`global_ordinals.fields.[field-name].build_time`::
(<<time-units,time unit>>)
The total time spent building global ordinals for field with specified name.

`global_ordinals.fields.[field-name].build_time_in_millis`::
(integer)
The total time, in milliseconds, spent building global ordinals for field with specified name.

`global_ordinals.fields.[field-name].shard_max_value_count`::
(long)
The total time spent building global ordinals for field with specified name.
=====
`query_cache`::
Expand Down Expand Up @@ -1514,7 +1534,11 @@ The API returns the following response:
"fielddata": {
"memory_size": "0b",
"memory_size_in_bytes": 0,
"evictions": 0
"evictions": 0,
"global_ordinals": {
"build_time" : "0s",
"build_time_in_millis" : 0
}
},
"query_cache": {
"memory_size": "0b",
Expand Down

0 comments on commit 6566bb4

Please sign in to comment.