[CI] DownsampleActionSingleNodeTests testDuplicateDownsampleRequest failing

[elasticsearchmachine]

Build Scans:

• elasticsearch-intake #17674 / part2
• elasticsearch-pull-request #55080 / part-2

Reproduction Line:

./gradlew ":x-pack:plugin:downsample:test" --tests "org.elasticsearch.xpack.downsample.DownsampleActionSingleNodeTests.testDuplicateDownsampleRequest" -Dtests.seed=F5C8F9E8A4AEA95A -Dtests.locale=hr-BA -Dtests.timezone=Africa/Conakry -Druntime.java=23

Applicable branches:
main

Reproduces locally?:
N/A

Failure History:
See dashboard

Failure Message:

com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=104, name=Thread-9, state=RUNNABLE, group=TGRP-DownsampleActionSingleNodeTests]

Issue Reasons:

• [main] 2 failures in test testDuplicateDownsampleRequest (0.2% fail rate in 811 executions)

Note:
This issue was created using new test triage automation. Please report issues or feedback to es-delivery.

[elasticsearchmachine]

This has been muted on branch main

Mute Reasons:

• [main] 2 failures in test testDuplicateDownsampleRequest (0.2% fail rate in 811 executions)

Build Scans:

• elasticsearch-intake #17674 / part2
• elasticsearch-pull-request #55080 / part-2

[elasticsearchmachine]

Pinging @elastic/es-data-management (Team:Data Management)

[elasticsearchmachine]

Pinging @elastic/es-storage-engine (Team:StorageEngine)

[kkrik-es]

Seems like there's a warning while writing the shard

  1> [2025-02-10T07:58:52,390][INFO ][o.e.x.d.DownsampleShardIndexer] [node_s_0] Downsampling task [downsample-gfwenlvjoa-0-942d on shard [mjuxilnmoydcim][0] started
  1> [2025-02-10T07:58:52,413][INFO ][o.e.x.d.DownsampleShardIndexer] [node_s_0] Shard [mjuxilnmoydcim][0] processed [512] docs, created [104] downsample buckets
  1> [2025-02-10T07:58:52,448][ERROR][o.e.x.d.DownsampleShardIndexer] [node_s_0] Shard [[mjuxilnmoydcim][0]] failed to populate downsample index. Failures: [{null=org.elasticsearch.cluster.block.ClusterBlockException: index [gfwenlvjoa] blocked by: [FORBIDDEN/8/index write (api)];}]
  1> [2025-02-10T07:58:52,453][INFO ][o.e.x.d.DownsampleShardIndexer] [node_s_0] Shard [[mjuxilnmoydcim][0]] successfully sent [512], received source doc [104], indexed downsampled doc [104], failed [104], took [0s]
  1> [2025-02-10T07:58:52,457][INFO ][o.e.x.d.DownsampleShardIndexer] [node_s_0] Downsampling task [downsample-gfwenlvjoa-0-942d] on shard [mjuxilnmoydcim][0] failed indexing [104]
  1> [2025-02-10T07:58:52,458][ERROR][o.e.x.d.DownsampleShardPersistentTaskExecutor] [node_s_0] Downsampling task [downsample-gfwenlvjoa-0-942d non retriable failure [Downsampling task [downsample-gfwenlvjoa-0-942d] on shard [mjuxilnmoydcim][0] failed indexing [104]]

Suite: Test class org.elasticsearch.xpack.downsample.DownsampleActionSingleNodeTests
  2> WARNING: A restricted method in   2> java.lang.foreign.Linker  2>  has been called
  2> WARNING:   2> java.lang.foreign.Linker::downcallHandle  2>  has been called by   2> org.elasticsearch.nativeaccess.jdk.LinkerHelper  2>  in   2> an unnamed module
  2> WARNING: Use --enable-native-access=  2> ALL-UNNAMED  2>  to avoid a warning for callers in this module
  2> WARNING: Restricted methods will be blocked in a future release unless native access is enabled

REPRODUCE WITH: ./gradlew ":x-pack:plugin:downsample:test" --tests "org.elasticsearch.xpack.downsample.DownsampleActionSingleNodeTests.testDuplicateDownsampleRequest" -Dtests.seed=F5C8F9E8A4AEA95A -Dtests.locale=hr-BA -Dtests.timezone=Africa/Conakry -Druntime.java=23

[martijnvg]

1> [2025-02-10T07:58:52,448][ERROR][o.e.x.d.DownsampleShardIndexer] [node_s_0] Shard [[mjuxilnmoydcim][0]] failed to populate downsample index. Failures: [{null=org.elasticsearch.cluster.block.ClusterBlockException: index [gfwenlvjoa] blocked by: [FORBIDDEN/8/index write (api)];}]

This is the target index? That shouldn't be read only. Was there cluster instability?