[BUG]org.opensearch.remotestore.RemoteStoreStatsIT.testDownloadStatsCorrectnessSinglePrimarySingleReplica #10193

Poojita-Raj · 2023-09-22T20:34:36Z

Describe the bug
org.opensearch.remotestore.RemoteStoreStatsIT.testDownloadStatsCorrectnessSinglePrimarySingleReplica failure seen on autocut gradle check failure to main. (#10149).

Expected behavior
Test should pass always.

Stacktrace:

java.lang.AssertionError
	at __randomizedtesting.SeedInfo.seed([5784F3A051D8B7A3:61806557B4685E4E]:0)
	at org.junit.Assert.fail(Assert.java:87)
	at org.junit.Assert.assertTrue(Assert.java:42)
	at org.junit.Assert.assertTrue(Assert.java:53)
	at org.opensearch.remotestore.RemoteStoreStatsIT.lambda$testDownloadStatsCorrectnessSinglePrimarySingleReplica$6(RemoteStoreStatsIT.java:310)
	at org.opensearch.test.OpenSearchTestCase.assertBusy(OpenSearchTestCase.java:1086)
	at org.opensearch.remotestore.RemoteStoreStatsIT.testDownloadStatsCorrectnessSinglePrimarySingleReplica(RemoteStoreStatsIT.java:294)
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
	at java.base/java.lang.reflect.Method.invoke(Method.java:578)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:938)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:974)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:988)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:48)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45)
	at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
	at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:947)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:832)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:883)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
	at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
	at org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.base/java.lang.Thread.run(Thread.java:1623)
	Suppressed: java.lang.AssertionError
		at org.junit.Assert.fail(Assert.java:87)
		at org.junit.Assert.assertTrue(Assert.java:42)
		at org.junit.Assert.assertTrue(Assert.java:53)
		at org.opensearch.remotestore.RemoteStoreStatsIT.lambda$testDownloadStatsCorrectnessSinglePrimarySingleReplica$6(RemoteStoreStatsIT.java:310)
		at org.opensearch.test.OpenSearchTestCase.assertBusy(OpenSearchTestCase.java:1074)
		... 39 more
	Suppressed: java.lang.AssertionError
		at org.junit.Assert.fail(Assert.java:87)
		at org.junit.Assert.assertTrue(Assert.java:42)
		at org.junit.Assert.assertTrue(Assert.java:53)
		at org.opensearch.remotestore.RemoteStoreStatsIT.lambda$testDownloadStatsCorrectnessSinglePrimarySingleReplica$6(RemoteStoreStatsIT.java:310)
		at org.opensearch.test.OpenSearchTestCase.assertBusy(OpenSearchTestCase.java:1074)
		... 39 more
	Suppressed: java.lang.AssertionError
		at org.junit.Assert.fail(Assert.java:87)
		at org.junit.Assert.assertTrue(Assert.java:42)
		at org.junit.Assert.assertTrue(Assert.java:53)
		at org.opensearch.remotestore.RemoteStoreStatsIT.lambda$testDownloadStatsCorrectnessSinglePrimarySingleReplica$6(RemoteStoreStatsIT.java:310)
		at org.opensearch.test.OpenSearchTestCase.assertBusy(OpenSearchTestCase.java:1074)
		... 39 more
	Suppressed: java.lang.AssertionError
		at org.junit.Assert.fail(Assert.java:87)
		at org.junit.Assert.assertTrue(Assert.java:42)
		at org.junit.Assert.assertTrue(Assert.java:53)
		at org.opensearch.remotestore.RemoteStoreStatsIT.lambda$testDownloadStatsCorrectnessSinglePrimarySingleReplica$6(RemoteStoreStatsIT.java:310)
		at org.opensearch.test.OpenSearchTestCase.assertBusy(OpenSearchTestCase.java:1074)
		... 39 more
	Suppressed: java.lang.AssertionError
		at org.junit.Assert.fail(Assert.java:87)
		at org.junit.Assert.assertTrue(Assert.java:42)
		at org.junit.Assert.assertTrue(Assert.java:53)
		at org.opensearch.remotestore.RemoteStoreStatsIT.lambda$testDownloadStatsCorrectnessSinglePrimarySingleReplica$6(RemoteStoreStatsIT.java:310)
		at org.opensearch.test.OpenSearchTestCase.assertBusy(OpenSearchTestCase.java:1074)
		... 39 more
	Suppressed: java.lang.AssertionError
		at org.junit.Assert.fail(Assert.java:87)
		at org.junit.Assert.assertTrue(Assert.java:42)
		at org.junit.Assert.assertTrue(Assert.java:53)
		at org.opensearch.remotestore.RemoteStoreStatsIT.lambda$testDownloadStatsCorrectnessSinglePrimarySingleReplica$6(RemoteStoreStatsIT.java:310)
		at org.opensearch.test.OpenSearchTestCase.assertBusy(OpenSearchTestCase.java:1074)
		... 39 more
	Suppressed: java.lang.AssertionError
		at org.junit.Assert.fail(Assert.java:87)
		at org.junit.Assert.assertTrue(Assert.java:42)
		at org.junit.Assert.assertTrue(Assert.java:53)
		at org.opensearch.remotestore.RemoteStoreStatsIT.lambda$testDownloadStatsCorrectnessSinglePrimarySingleReplica$6(RemoteStoreStatsIT.java:310)
		at org.opensearch.test.OpenSearchTestCase.assertBusy(OpenSearchTestCase.java:1074)
		... 39 more
	Suppressed: java.lang.AssertionError
		at org.junit.Assert.fail(Assert.java:87)
		at org.junit.Assert.assertTrue(Assert.java:42)
		at org.junit.Assert.assertTrue(Assert.java:53)
		at org.opensearch.remotestore.RemoteStoreStatsIT.lambda$testDownloadStatsCorrectnessSinglePrimarySingleReplica$6(RemoteStoreStatsIT.java:310)
		at org.opensearch.test.OpenSearchTestCase.assertBusy(OpenSearchTestCase.java:1074)
		... 39 more
	Suppressed: java.lang.AssertionError
		at org.junit.Assert.fail(Assert.java:87)
		at org.junit.Assert.assertTrue(Assert.java:42)
		at org.junit.Assert.assertTrue(Assert.java:53)
		at org.opensearch.remotestore.RemoteStoreStatsIT.lambda$testDownloadStatsCorrectnessSinglePrimarySingleReplica$6(RemoteStoreStatsIT.java:310)
		at org.opensearch.test.OpenSearchTestCase.assertBusy(OpenSearchTestCase.java:1074)
		... 39 more
	Suppressed: java.lang.AssertionError
		at org.junit.Assert.fail(Assert.java:87)
		at org.junit.Assert.assertTrue(Assert.java:42)
		at org.junit.Assert.assertTrue(Assert.java:53)
		at org.opensearch.remotestore.RemoteStoreStatsIT.lambda$testDownloadStatsCorrectnessSinglePrimarySingleReplica$6(RemoteStoreStatsIT.java:310)
		at org.opensearch.test.OpenSearchTestCase.assertBusy(OpenSearchTestCase.java:1074)
		... 39 more
	Suppressed: java.lang.AssertionError
		at org.junit.Assert.fail(Assert.java:87)
		at org.junit.Assert.assertTrue(Assert.java:42)
		at org.junit.Assert.assertTrue(Assert.java:53)
		at org.opensearch.remotestore.RemoteStoreStatsIT.lambda$testDownloadStatsCorrectnessSinglePrimarySingleReplica$6(RemoteStoreStatsIT.java:310)
		at org.opensearch.test.OpenSearchTestCase.assertBusy(OpenSearchTestCase.java:1074)
		... 39 more
	Suppressed: java.lang.AssertionError
		at org.junit.Assert.fail(Assert.java:87)
		at org.junit.Assert.assertTrue(Assert.java:42)
		at org.junit.Assert.assertTrue(Assert.java:53)
		at org.opensearch.remotestore.RemoteStoreStatsIT.lambda$testDownloadStatsCorrectnessSinglePrimarySingleReplica$6(RemoteStoreStatsIT.java:310)
		at org.opensearch.test.OpenSearchTestCase.assertBusy(OpenSearchTestCase.java:1074)
		... 39 more
	Suppressed: java.lang.AssertionError
		at org.junit.Assert.fail(Assert.java:87)
		at org.junit.Assert.assertTrue(Assert.java:42)
		at org.junit.Assert.assertTrue(Assert.java:53)
		at org.opensearch.remotestore.RemoteStoreStatsIT.lambda$testDownloadStatsCorrectnessSinglePrimarySingleReplica$6(RemoteStoreStatsIT.java:310)
		at org.opensearch.test.OpenSearchTestCase.assertBusy(OpenSearchTestCase.java:1074)
		... 39 more
	Suppressed: java.lang.AssertionError
		at org.junit.Assert.fail(Assert.java:87)
		at org.junit.Assert.assertTrue(Assert.java:42)
		at org.junit.Assert.assertTrue(Assert.java:53)
		at org.opensearch.remotestore.RemoteStoreStatsIT.lambda$testDownloadStatsCorrectnessSinglePrimarySingleReplica$6(RemoteStoreStatsIT.java:310)
		at org.opensearch.test.OpenSearchTestCase.assertBusy(OpenSearchTestCase.java:1074)
		... 39 more
	Suppressed: java.lang.AssertionError
		at org.junit.Assert.fail(Assert.java:87)
		at org.junit.Assert.assertTrue(Assert.java:42)
		at org.junit.Assert.assertTrue(Assert.java:53)
		at org.opensearch.remotestore.RemoteStoreStatsIT.lambda$testDownloadStatsCorrectnessSinglePrimarySingleReplica$6(RemoteStoreStatsIT.java:310)
		at org.opensearch.test.OpenSearchTestCase.assertBusy(OpenSearchTestCase.java:1074)
		... 39 more
	Suppressed: java.lang.AssertionError
		at org.junit.Assert.fail(Assert.java:87)
		at org.junit.Assert.assertTrue(Assert.java:42)
		at org.junit.Assert.assertTrue(Assert.java:53)
		at org.opensearch.remotestore.RemoteStoreStatsIT.lambda$testDownloadStatsCorrectnessSinglePrimarySingleReplica$6(RemoteStoreStatsIT.java:310)
		at org.opensearch.test.OpenSearchTestCase.assertBusy(OpenSearchTestCase.java:1074)
		... 39 more
		```

The text was updated successfully, but these errors were encountered:

ashking94 · 2023-11-02T09:41:32Z

#11021 (comment)

Poojita-Raj · 2023-12-26T18:06:32Z

Ran it successfully 1000 times - twice on local.
Tested the test suite 200 times, tried running the module.
Wasn't able to reproduce the failure even with any of the seeds on previously seen failures.

Poojita-Raj added bug Something isn't working untriaged labels Sep 22, 2023

Poojita-Raj mentioned this issue Sep 22, 2023

[AUTOCUT] Gradle Check Failure on push to main #10149

Closed

reta mentioned this issue Sep 27, 2023

[AUTOCUT] Gradle Check Failure on push to 2.x #10245

Closed

psychbot mentioned this issue Oct 3, 2023

[Remote Store] Add support to restrict creation & deletion if system repository and mutation of immutable settings of system repository #9839

Merged

6 tasks

ashking94 mentioned this issue Oct 11, 2023

Fix shard failure on flush during upload failures for remote indexes #10513

Merged

7 tasks

This was referenced Oct 22, 2023

Make index and global metadata upload timeout dynamic cluster settings #10814

Merged

[Backport 2.x] Make index and global metadata upload timeout dynamic cluster setting #10852

Merged

This was referenced Oct 30, 2023

Fix passing wrong parameter when calling newConfigurationException() in DotExpanderProcessor #10737

Merged

[Streaming Indexing] Introduce new experimental HTTP transport based on Netty 4 and Project Reactor (Reactor Netty) #9672

Merged

ashking94 mentioned this issue Nov 2, 2023

[Remote Store] Fix stuck segments upload issue #11021

Closed

8 tasks

neetikasinghal mentioned this issue Nov 10, 2023

update index random function to fix the bogus document deletion #11142

Merged

8 tasks

reta mentioned this issue Nov 22, 2023

Bump OpenTelemetry from 1.31.0 to 1.32.0 and OpenTelemetry Semconv from 1.21.0-alpha to 1.23.1-alpha #11305

Merged

8 tasks

anasalkouz added Other Indexing:Replication Issues and PRs related to core replication framework eg segrep Storage:Remote and removed Other untriaged labels Dec 14, 2023

Poojita-Raj self-assigned this Dec 20, 2023

Poojita-Raj mentioned this issue Jan 3, 2024

Add logging for tests in RemoteStoreStatsIT to catch assertion failure cause #11734

Merged

8 tasks

gaobinlong mentioned this issue Jan 10, 2024

[Backport 2.x] Enable must_exist parameter for update aliases API (#11210) #11833

Merged

8 tasks

reta mentioned this issue Jan 31, 2024

Add advance(int) for numeric values in order to allow point based optimization to kick in #12089

Merged

8 tasks

ajaymovva mentioned this issue Apr 1, 2024

[AUTOCUT] Gradle Check Failure on push to main #10811

Closed

neetikasinghal mentioned this issue May 24, 2024

Backport 2.x - Refactor writable warm feature flag to include tiering #13799

Merged

9 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[BUG]org.opensearch.remotestore.RemoteStoreStatsIT.testDownloadStatsCorrectnessSinglePrimarySingleReplica #10193

[BUG]org.opensearch.remotestore.RemoteStoreStatsIT.testDownloadStatsCorrectnessSinglePrimarySingleReplica #10193

Poojita-Raj commented Sep 22, 2023

ashking94 commented Nov 2, 2023

Poojita-Raj commented Dec 26, 2023

[BUG]org.opensearch.remotestore.RemoteStoreStatsIT.testDownloadStatsCorrectnessSinglePrimarySingleReplica #10193

[BUG]org.opensearch.remotestore.RemoteStoreStatsIT.testDownloadStatsCorrectnessSinglePrimarySingleReplica #10193

Comments

Poojita-Raj commented Sep 22, 2023

ashking94 commented Nov 2, 2023

Poojita-Raj commented Dec 26, 2023