NPE in downsample transport action

[martijnvg]

[2025-11-07T20:18:30,753][ERROR][o.e.d.l.DataStreamLifecycleService][node_s0][masterService#updateTask][T#1] Data stream lifecycle encountered an error trying to downsample index [downsample-5m-.ds-metrics-baz-2025.11.07-000001]. Data stream lifecycle will attempt to downsample the index on its next run.	
Failing since [1762535910446], operation retried [10] times	
java.lang.NullPointerException: Cannot invoke "org.elasticsearch.cluster.metadata.IndexMetadata.getSettings()" because "downsampleIndex" is null	
	at org.elasticsearch.xpack.downsample.TransportDownsampleAction$RefreshDownsampleIndexActionListener$1.execute(TransportDownsampleAction.java:1164) ~[main/:?]	
	at org.elasticsearch.xpack.downsample.TransportDownsampleAction$1.executeTask(TransportDownsampleAction.java:142) ~[main/:?]	
	at org.elasticsearch.xpack.downsample.TransportDownsampleAction$1.executeTask(TransportDownsampleAction.java:138) ~[main/:?]	
	at org.elasticsearch.cluster.SimpleBatchedExecutor.execute(SimpleBatchedExecutor.java:71) ~[elasticsearch-9.3.0-SNAPSHOT.jar:9.3.0-SNAPSHOT]	
	at org.elasticsearch.cluster.service.MasterService.innerExecuteTasks(MasterService.java:1100) ~[elasticsearch-9.3.0-SNAPSHOT.jar:9.3.0-SNAPSHOT]	
	at org.elasticsearch.cluster.service.MasterService.executeTasks(MasterService.java:1063) ~[elasticsearch-9.3.0-SNAPSHOT.jar:9.3.0-SNAPSHOT]	
	at org.elasticsearch.cluster.service.MasterService.executeAndPublishBatch(MasterService.java:246) ~[elasticsearch-9.3.0-SNAPSHOT.jar:9.3.0-SNAPSHOT]	
	at org.elasticsearch.cluster.service.MasterService$BatchingTaskQueue$Processor.lambda$run$2(MasterService.java:1710) ~[elasticsearch-9.3.0-SNAPSHOT.jar:9.3.0-SNAPSHOT]	
	at org.elasticsearch.action.ActionListener.run(ActionListener.java:465) ~[elasticsearch-9.3.0-SNAPSHOT.jar:9.3.0-SNAPSHOT]	
	at org.elasticsearch.cluster.service.MasterService$BatchingTaskQueue$Processor.run(MasterService.java:1707) ~[elasticsearch-9.3.0-SNAPSHOT.jar:9.3.0-SNAPSHOT]	
	at org.elasticsearch.cluster.service.MasterService$5.lambda$doRun$0(MasterService.java:1308) ~[elasticsearch-9.3.0-SNAPSHOT.jar:9.3.0-SNAPSHOT]	
	at org.elasticsearch.action.ActionListener.run(ActionListener.java:465) ~[elasticsearch-9.3.0-SNAPSHOT.jar:9.3.0-SNAPSHOT]	
	at org.elasticsearch.cluster.service.MasterService$5.doRun(MasterService.java:1287) ~[elasticsearch-9.3.0-SNAPSHOT.jar:9.3.0-SNAPSHOT]	
	at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:1076) ~[elasticsearch-9.3.0-SNAPSHOT.jar:9.3.0-SNAPSHOT]	
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:27) ~[elasticsearch-9.3.0-SNAPSHOT.jar:9.3.0-SNAPSHOT]	
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1090) ~[?:?]	
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:614) ~[?:?]	
	at java.lang.Thread.run(Thread.java:1474) ~[?:?]

Test failure: https://gradle-enterprise.elastic.co/s/lrb4evifhrn6e/tests/task/:x-pack:plugin:downsample:internalClusterTest/details/org.elasticsearch.xpack.downsample.DataStreamLifecycleDownsampleIT/testUpdateDownsampleSamplingMode?top-execution=1

Reported reproduction does reproduce. The linked test failure occurred on a PR branch, but this also reproduces on main branch.

[elasticsearchmachine]

Pinging @elastic/es-storage-engine (Team:StorageEngine)

[gmarouli]

I was not able to reproduce this, but I found the root cause. The tests waits for the cluster state to show that the data stream has now two 10-minute downsampled backing indices, this times out which triggers the clean up of the test which eventually deletes all indices.

In the mean time the downsampling transport action is still in progress and when its time to update the status of the target index, it cannot find the metadata because the test clean up has already happened and throws an NPE.

I will open a PR to better handle this.