Cat endpoint for data streams

[joegallo]

Along similar lines as _cat/aliases and _cat/indices, we should have a _cat/data_streams.

Perhaps something like:

GET _cat/data_streams&v
data_stream    status generation template      ilm_policy  [...]
my-data-stream yellow          1 some-template some-policy [...]

Also, there's already _cat/templates, but there's no _cat API for ILM Policies (#29690 has a checkbox for it). It probably makes sense to add that, too, perhaps as a followup.

/cc @dakrone @martijnvg

[elasticmachine]

Pinging @elastic/es-core-features (Team:Core/Features)

[danhermann]

I'd suggest including a count of backing indices. And maybe excluding the generation as it's of marginal value.

[joegallo]

+1, but doesn't the generation tell you the number of backing indices?

[danhermann]

+1, but doesn't the generation tell you the number of backing indices?

No, the generation has turned out to be just a counter for the number of times the rollover API has been called on the data stream. That will diverge from the number of backing indices as older backing indices are deleted or if the data stream migration API is used to create a new data stream from an alias pointing to more than one index.

[joegallo]

Ahhhh yes, of course. Thanks!

[VimCommando]

I would be very useful to include the current write index for the data stream. That is not easy to determine in the current API output.

[danhermann]

+1, but doesn't the generation tell you the number of backing indices?

No, the generation has turned out to be just a counter for the number of times the rollover API has been called on the data stream. That will diverge from the number of backing indices as older backing indices are deleted or if the data stream migration API is used to create a new data stream from an alias pointing to more than one index.

Just an update for posterity -- the generation is no longer even a counter for the number of rollover operations since there are now other operations that increment the generation. It's an implementation that shouldn't be relied upon for anything other than a monotonically increasing value attached to the names generated for backing indices.

[danhermann]

I would be very useful to include the current write index for the data stream. That is not easy to determine in the current API output.

That's something that would make sense to add to the GET _data_stream API. It wouldn't fit as neatly into a cat API, especially since, unlike the JSON responses to most APIs, the fixed-width text responses from cat APIs are intended explicitly for human consumption and not machine consumption.