Skip to content
This repository has been archived by the owner on Jan 31, 2024. It is now read-only.

Commit

Permalink
Tests: wait for bulk before closing the test
Browse files Browse the repository at this point in the history
When closing a test, our test infra removes created indices.

But as the river is still processing documents, the bulk processor contains remaining document which have not been indexed yet.

When the river stops, it fires a `close()` call on the bulk processor.
This call creates again the index. Test framework throw an exception as the mock directory wrapper can not close due to write operations in progress.

Logs:

```
  1> [2014-10-09 11:59:40,965][INFO ][cluster.metadata         ] [node_s0] [_river] deleting index
  1> [2014-10-09 11:59:40,982][INFO ][test.store               ] [node_s1] [_river][5] Shard state before potentially flushing is STARTED
  1> [2014-10-09 11:59:41,084][INFO ][test.store               ] [node_s3] [_river][0] Shard state before potentially flushing is STARTED
  1> [2014-10-09 11:59:41,115][INFO ][test.store               ] [node_s2] [_river][6] Shard state before potentially flushing is STARTED
  1> [2014-10-09 11:59:41,943][INFO ][river.wikipedia          ] [node_s1] [wikipedia][wikipedia] closing wikipedia river
  1> [2014-10-09 11:59:42,693][WARN ][river.wikipedia          ] [node_s1] [wikipedia][wikipedia] river was closing while processing wikipedia page [39]/[Aube (département)
  1>     ]. Operation skipped.
  1> [2014-10-09 11:59:42,768][INFO ][river.wikipedia          ] [WikipediaRiverTest#testWikipediaRiverFrench]: cleaning up after test
  1> [2014-10-09 11:59:42,769][INFO ][cluster.metadata         ] [node_s0] [wikipedia] deleting index
  1> [2014-10-09 11:59:43,587][WARN ][river.wikipedia          ] There was failures while executing bulk
  1> [2014-10-09 11:59:44,971][INFO ][cluster.metadata         ] [node_s0] [wikipedia] creating index, cause [auto(bulk api)], shards [9]/[2], mappings [_default_]
  1> [2014-10-09 11:59:45,805][INFO ][cluster.metadata         ] [node_s0] [wikipedia] update_mapping [page] (dynamic)
  1> [2014-10-09 12:00:00,466][INFO ][test.store               ] [node_s2] [wikipedia][0] MockDirectoryWrapper#close() threw exception
  1> java.lang.RuntimeException: MockDirectoryWrapper: cannot close: there are still open locks: {write.lock=java.lang.RuntimeException: lock "write.lock" was not released}
  1> 	at org.apache.lucene.store.MockDirectoryWrapper.close(MockDirectoryWrapper.java:709)
  1> 	at org.elasticsearch.test.store.MockDirectoryHelper$ElasticsearchMockDirectoryWrapper.close(MockDirectoryHelper.java:140)
  1> 	at org.elasticsearch.test.hamcrest.ElasticsearchAssertions.assertAllFilesClosed(ElasticsearchAssertions.java:667)
  1> 	at org.elasticsearch.test.TestCluster.assertAfterTest(TestCluster.java:76)
  1> 	at org.elasticsearch.test.ElasticsearchIntegrationTest.afterInternal(ElasticsearchIntegrationTest.java:603)
  1> 	at org.elasticsearch.test.ElasticsearchIntegrationTest.after(ElasticsearchIntegrationTest.java:1722)
```

Closes #37.
(cherry picked from commit d2f869c)
  • Loading branch information
dadoonet committed Oct 9, 2014
1 parent 4ea052f commit 04518ad
Showing 1 changed file with 2 additions and 7 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -68,18 +68,13 @@ public void createEmptyRiverIndex() {
public void deleteRiverAndWait() throws InterruptedException {
logger.info(" --> remove all wikipedia rivers");
client().admin().indices().prepareDelete("_river").get();

// We just wait a few to make sure that all bulks has been processed
awaitBusy(new Predicate<Object>() {
@Override
public boolean apply(Object o) {
try {
client().admin().indices().prepareGetIndex().addIndices("_river").get();
} catch (IndexMissingException e) {
return true;
}
return false;
}
}, 10, TimeUnit.SECONDS);
}, 2, TimeUnit.SECONDS);
}

@Test
Expand Down

0 comments on commit 04518ad

Please sign in to comment.