The downsample operation doesn't set any tier preference for the downsampled index

[andreidan]

Elasticsearch Version

8.7+

Installed Plugins

No response

Java Version

bundled

OS Version

Darwin

Problem Description

The downsample transport action ( https://github.com/elastic/elasticsearch/blob/main/x-pack/plugin/rollup/src/main/java/org/elasticsearch/xpack/downsample/TransportDownsampleAction.java ) creates a rollup index without setting any tier preference.

This means the rollup index will receive the automatic tier preference of data_content irrespective of where the source index is located.
This means that even if the source index is in the warm tier, the rollup index will be created in the hot tier (so we'd be downsampling from warm to hot).

This has some confusing consequences when using ILM if, say, the downsample action is configured in the cold phase the hot tier could run out of space (or at least see increased, unexpected (?) usage)

Maybe this makes sense for performance reasons? If that's the case I think we should document this behaviour.
We should otherwise configure the _tier_preference on the rollup index.

Steps to Reproduce

Spin up a deployment with hot/warm/cold tiers.
Create an index with the _tier_preference: data_warm and create an ILM policy with only the warm phase that contains the downsample action ( https://www.elastic.co/guide/en/elasticsearch/reference/current/downsampling-ilm.html )
Notice the downsample-* index is created in the hot tier.

Logs (if relevant)

No response

[elasticsearchmachine]

Pinging @elastic/es-analytics-geo (Team:Analytics)