Skip to content

Removing assertion in asynchronous operation to avoid flaky test#1

Open
mkl201 wants to merge 1 commit into
masterfrom
fix-flaky
Open

Removing assertion in asynchronous operation to avoid flaky test#1
mkl201 wants to merge 1 commit into
masterfrom
fix-flaky

Conversation

@mkl201
Copy link
Copy Markdown
Owner

@mkl201 mkl201 commented Jan 20, 2022

What is the purpose of this PR

  • This PR fixes a flaky test to avoid an assertion on an asynchronous operation uk.co.flax.luwak.TestCachePurging#testBackgroundPurges

Reproduce the test failure

  • The test fails non-deterministically when execution delays occur. To observe a test failure, the unmodified test can be re-executed many times. The likelihood of observing a test failure increases when the test executes on a slow machine. To deterministically reproduce the test failure, introduce an execution delay:
@Test
public void testBackgroundPurges() throws IOException, InterruptedException, UpdateException, Exception {

    QueryIndexConfiguration config = new QueryIndexConfiguration().setPurgeFrequency(1, TimeUnit.SECONDS);
    try (Monitor monitor = new Monitor(new LuceneQueryParser("field"), new MatchAllPresearcher(), config)) {

        assertThat(monitor.getQueryCacheStats().lastPurged).isEqualTo(-1);

        for (int i = 0; i < 100; i++) {
            monitor.update(newMonitorQuery(i));
        }
        monitor.deleteById("5");
        Thread.sleep(600L);
        assertThat(monitor.getQueryCacheStats().queries).isEqualTo(99);
        assertThat(monitor.getQueryCacheStats().cachedQueries).isEqualTo(100);

        TimeUnit.SECONDS.sleep(2);
        assertThat(monitor.getQueryCacheStats().queries).isEqualTo(99);
        assertThat(monitor.getQueryCacheStats().cachedQueries).isEqualTo(99);
        assertThat(monitor.getQueryCacheStats().lastPurged).isGreaterThan(0);
    }
}

Expected result:

  • The test should run successfully in all runs.

Actual result:

  • In 57 out of 100 repetitions of the test suite, the test fails with:
org.junit.ComparisonFailure: expected:<[100]> but was:<[99]>

Why the test fails

Fix

  • Remove the assertions on an asynchronous operation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant