Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SOLR-16304 De-annotate some slow tests #951

Merged
merged 33 commits into from Aug 2, 2022
Merged
Show file tree
Hide file tree
Changes from 31 commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
510d877
SOLR-16304 De-annotate some slow tests
madrob Jul 22, 2022
ca3831c
revert accidental inclusion
madrob Jul 22, 2022
febd9e5
fix shards to speed up non-nightly test
madrob Jul 22, 2022
49907e2
more not really slow tests
madrob Jul 23, 2022
51862b9
modules
madrob Jul 25, 2022
99534d9
no slow in solrj
madrob Jul 25, 2022
7219436
tidy
madrob Jul 25, 2022
b346693
cluster can be shared in a test class
madrob Jul 25, 2022
c14d465
revert accidental extras
madrob Jul 25, 2022
5cfca07
tidy
madrob Jul 25, 2022
40dc68f
more tests
madrob Jul 25, 2022
18b7598
slow to nightly
madrob Jul 25, 2022
0818d65
update docs to remove slow
madrob Jul 25, 2022
8260f7f
fix failing test
madrob Jul 25, 2022
f761526
revert debugging
madrob Jul 26, 2022
ff3857f
chaos tests to nightly
madrob Jul 26, 2022
c8e22ea
move work to nightly
madrob Jul 28, 2022
5c899fe
./gradlew spotlessApply
cpoerschke Jul 29, 2022
f87fe8a
remove -Dtests.slow=true in comments
cpoerschke Jul 29, 2022
d6d05e5
de-annotate some slow tests that we should feel until we fix them
madrob Jul 28, 2022
9d2b4a9
share cluster
madrob Jul 28, 2022
be0e1b6
tidy
madrob Jul 28, 2022
123b9a2
speed up wait code
madrob Jul 28, 2022
4fbd147
clarify some code
madrob Jul 29, 2022
ad1326b
precommit
madrob Jul 29, 2022
19c85bf
review feedback
madrob Jul 29, 2022
e8a9acd
more work to nightlies
madrob Jul 29, 2022
68bcc04
tidy
madrob Aug 1, 2022
8116c21
randomize for less work
madrob Aug 1, 2022
1593b2c
the rest of Slow
madrob Aug 1, 2022
2515211
rewrite loops as zk.wait
madrob Aug 1, 2022
410c368
semicolon
madrob Aug 1, 2022
b2aec5a
CHANGES
madrob Aug 2, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
4 changes: 2 additions & 2 deletions dev-tools/scripts/reproduceJenkinsFailures.py
Expand Up @@ -38,8 +38,8 @@
reAntInvocation = re.compile(r'\bant(?:\.bat)?\s+.*(?:jenkins-(?:hourly|nightly)|nightly-smoke)')
reAntSysprops = re.compile(r'"-D[^"]+"|-D[^=]+="[^"]*"|-D\S+')

# Method example: NOTE: reproduce with: ant test -Dtestcase=ZkSolrClientTest -Dtests.method=testMultipleWatchesAsync -Dtests.seed=6EF5AB70F0032849 -Dtests.slow=true -Dtests.locale=he-IL -Dtests.timezone=NST -Dtests.asserts=true -Dtests.file.encoding=UTF-8
# Suite example: NOTE: reproduce with: ant test -Dtestcase=CloudSolrClientTest -Dtests.seed=DB2DF2D8228BAF27 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=es-AR -Dtests.timezone=America/Argentina/Cordoba -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
# Method example: NOTE: reproduce with: ant test -Dtestcase=ZkSolrClientTest -Dtests.method=testMultipleWatchesAsync -Dtests.seed=6EF5AB70F0032849 -Dtests.locale=he-IL -Dtests.timezone=NST -Dtests.asserts=true -Dtests.file.encoding=UTF-8
# Suite example: NOTE: reproduce with: ant test -Dtestcase=CloudSolrClientTest -Dtests.seed=DB2DF2D8228BAF27 -Dtests.multiplier=3 -Dtests.locale=es-AR -Dtests.timezone=America/Argentina/Cordoba -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
reReproLine = re.compile(r'NOTE:\s+reproduce\s+with:(\s+ant\s+test\s+-Dtestcase=(\S+)\s+(?:-Dtests.method=\S+\s+)?(.*))')
reTestsSeed = re.compile(r'-Dtests.seed=\S+\s*')

Expand Down
1 change: 0 additions & 1 deletion gradle/testing/randomization.gradle
Expand Up @@ -85,7 +85,6 @@ allprojects {
[propName: 'tests.timezone', value: "random", description: "Sets the default time zone tests should run with."],
// filtering
[propName: 'tests.filter', value: null, description: "Applies a test filter (see :helpTests)."],
[propName: 'tests.slow', value: true, description: "Enables or disables @Slow tests."],
madrob marked this conversation as resolved.
Show resolved Hide resolved
[propName: 'tests.nightly', value: false, description: "Enables or disables @Nightly tests."],
[propName: 'tests.weekly', value: false, description: "Enables or disables @Weekly tests."],
[propName: 'tests.monster', value: false, description: "Enables or disables @Monster tests."],
Expand Down
8 changes: 4 additions & 4 deletions help/tests.txt
Expand Up @@ -68,17 +68,17 @@ Test groups
-----------

Tests can be filtered by an annotation they're marked with.
Some test group annotations include: @AwaitsFix, @Nightly, @Slow
Some test group annotations include: @AwaitsFix, @Nightly, @Monster

This uses filtering infrastructure on the *runner* (randomizedtesting),
not gradle's built-in mechanisms (but it can be combined with "--tests").
For example, run all solr-core tests annotated as @Slow:
For example, run all solr-core tests annotated as @Nightly:

gradlew -p solr/core test -Ptests.filter=@Slow
gradlew -p solr/core test -Ptests.filter=@Nightly

Test group filters can be combined into Boolean expressions:

gradlew -p solr/core test "default and not(@awaitsfix or @slow)"
gradlew -p solr/core test -Ptests.filter="default and not(@awaitsfix)"


Reiteration ("beasting")
Expand Down
Expand Up @@ -238,7 +238,7 @@ private void _write(OutputStream os) throws IOException {
// obscure a condition to handle as part of this patch, if someone wants to pursue it can be
// reproduced with:
// ant test -Dtestcase=StreamingTest -Dtests.method=testAllValidExportTypes
// -Dtests.seed=10F13879D0D1D6AD -Dtests.slow=true -Dtests.locale=es-PA
// -Dtests.seed=10F13879D0D1D6AD -Dtests.locale=es-PA
// -Dtests.timezone=America/Bahia_Banderas -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
// You'll have to uncomment the if below to hit the null pointer exception.
// This is such an unusual case (i.e. an empty index) that catching this concdition here is
Expand Down
Expand Up @@ -18,7 +18,6 @@

import java.util.Arrays;
import java.util.List;
import org.apache.lucene.tests.util.LuceneTestCase.Slow;
import org.apache.solr.SolrTestCaseJ4.SuppressSSL;
import org.apache.solr.client.solrj.SolrQuery;
import org.apache.solr.client.solrj.response.IntervalFacet.Count;
Expand All @@ -27,7 +26,6 @@
import org.junit.BeforeClass;
import org.junit.Test;

@Slow
@SuppressSSL(bugUrl = "https://issues.apache.org/jira/browse/SOLR-9182 - causes OOM")
// See: https://issues.apache.org/jira/browse/SOLR-12028 Tests cannot remove files on Windows
// machines occasionally
Expand Down
Expand Up @@ -20,7 +20,6 @@

import java.io.IOException;
import java.util.List;
import org.apache.lucene.tests.util.LuceneTestCase.Slow;
import org.apache.solr.SolrTestCaseJ4.SuppressPointFields;
import org.apache.solr.client.solrj.SolrClient;
import org.apache.solr.client.solrj.SolrServerException;
Expand All @@ -39,7 +38,6 @@
*
* @since solr 4.0
*/
@Slow
@SuppressPointFields(bugUrl = "https://issues.apache.org/jira/browse/SOLR-10844")
public class TestDistributedGrouping extends BaseDistributedSearchTestCase {

Expand Down
Expand Up @@ -16,12 +16,10 @@
*/
package org.apache.solr;

import org.apache.lucene.tests.util.LuceneTestCase.Slow;
import org.apache.solr.client.solrj.response.QueryResponse;
import org.junit.Test;

/** Tests sortMissingFirst and sortMissingLast in distributed sort */
@Slow
public class TestDistributedMissingSort extends BaseDistributedSearchTestCase {

public TestDistributedMissingSort() {
Expand Down
2 changes: 0 additions & 2 deletions solr/core/src/test/org/apache/solr/TestDistributedSearch.java
Expand Up @@ -32,7 +32,6 @@
import java.util.concurrent.ExecutorCompletionService;
import java.util.concurrent.Future;
import org.apache.commons.lang3.StringUtils;
import org.apache.lucene.tests.util.LuceneTestCase.Slow;
import org.apache.solr.SolrTestCaseJ4.SuppressSSL;
import org.apache.solr.client.solrj.SolrClient;
import org.apache.solr.client.solrj.SolrQuery;
Expand Down Expand Up @@ -71,7 +70,6 @@
*
* @since solr 1.3
*/
@Slow
@SuppressSSL(bugUrl = "https://issues.apache.org/jira/browse/SOLR-9061")
public class TestDistributedSearch extends BaseDistributedSearchTestCase {

Expand Down
2 changes: 0 additions & 2 deletions solr/core/src/test/org/apache/solr/TestRandomDVFaceting.java
Expand Up @@ -22,7 +22,6 @@
import java.util.List;
import java.util.Map;
import java.util.Random;
import org.apache.lucene.tests.util.LuceneTestCase.Slow;
import org.apache.lucene.tests.util.TestUtil;
import org.apache.solr.common.params.ModifiableSolrParams;
import org.apache.solr.request.SolrQueryRequest;
Expand All @@ -39,7 +38,6 @@
* and compares the docvalues facet results to the indexed facet results as if it were just another
* faceting method.
*/
@Slow
@SolrTestCaseJ4.SuppressPointFields(
bugUrl = "Test explicitly compares Trie to Points, randomization defeats the point")
@SolrTestCaseJ4.SuppressSSL
Expand Down
2 changes: 0 additions & 2 deletions solr/core/src/test/org/apache/solr/TestRandomFaceting.java
Expand Up @@ -29,7 +29,6 @@
import java.util.Set;
import java.util.function.Consumer;
import java.util.regex.Pattern;
import org.apache.lucene.tests.util.LuceneTestCase.Slow;
import org.apache.lucene.tests.util.TestUtil;
import org.apache.solr.common.SolrException.ErrorCode;
import org.apache.solr.common.params.ModifiableSolrParams;
Expand All @@ -41,7 +40,6 @@
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

@Slow
public class TestRandomFaceting extends SolrTestCaseJ4 {

private static final Pattern trieFields = Pattern.compile(".*_t.");
Expand Down
Expand Up @@ -16,7 +16,6 @@
*/
package org.apache.solr.cloud;

import org.apache.lucene.tests.util.LuceneTestCase.Slow;
import org.apache.solr.SolrTestCaseJ4.SuppressSSL;
import org.junit.Test;

Expand All @@ -25,7 +24,6 @@
* expected. Implementation moved to AbstractBasicDistributedZkTestBase as it is used by many HDFS
* contrib tests.
*/
@Slow
@SuppressSSL(bugUrl = "https://issues.apache.org/jira/browse/SOLR-5776")
public class BasicDistributedZkTest extends AbstractBasicDistributedZkTestBase {

Expand Down
Expand Up @@ -16,13 +16,11 @@
*/
package org.apache.solr.cloud;

import org.apache.lucene.tests.util.LuceneTestCase.Slow;
import org.apache.solr.SolrTestCaseJ4.SuppressSSL;

/*
* Implementation moved to AbstractChaosMonkeyNothingIsSafeTestBase.java as it is also
* used by the HDFS contrib tests.
*/
@Slow
@SuppressSSL(bugUrl = "https://issues.apache.org/jira/browse/SOLR-5776")
public class ChaosMonkeyNothingIsSafeTest extends AbstractChaosMonkeyNothingIsSafeTestBase {}
Expand Up @@ -23,7 +23,7 @@
import java.util.List;
import java.util.Set;
import java.util.concurrent.TimeUnit;
import org.apache.lucene.tests.util.LuceneTestCase.Slow;
import org.apache.lucene.tests.util.LuceneTestCase;
import org.apache.solr.SolrTestCaseJ4.SuppressSSL;
import org.apache.solr.client.solrj.SolrQuery;
import org.apache.solr.client.solrj.impl.CloudLegacySolrClient;
Expand All @@ -42,7 +42,7 @@
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

@Slow
@LuceneTestCase.Nightly
@SuppressSSL(bugUrl = "https://issues.apache.org/jira/browse/SOLR-5776")
public class ChaosMonkeyNothingIsSafeWithPullReplicasTest extends AbstractFullDistribZkTestBase {
private static final int FAIL_TOLERANCE = 100;
Expand Down
Expand Up @@ -16,18 +16,12 @@
*/
package org.apache.solr.cloud;

import org.apache.lucene.tests.util.LuceneTestCase.Slow;

/*
* Implementation moved to AbstractChaosMonkeySafeLeaderTestBase as is used by HDFS contrib module test
*/
@Slow
public class ChaosMonkeySafeLeaderTest extends AbstractChaosMonkeySafeLeaderTestBase {

private static final String DIRECTORY_FACTORY = "solr.StandardDirectoryFactory";

@Override
protected String getDirectoryFactory() {
return DIRECTORY_FACTORY;
return "solr.StandardDirectoryFactory";
}
}
Expand Up @@ -21,7 +21,7 @@
import java.util.EnumSet;
import java.util.List;
import java.util.concurrent.TimeUnit;
import org.apache.lucene.tests.util.LuceneTestCase.Slow;
import org.apache.lucene.tests.util.LuceneTestCase;
import org.apache.solr.client.solrj.SolrQuery;
import org.apache.solr.client.solrj.SolrServerException;
import org.apache.solr.client.solrj.impl.CloudSolrClient;
Expand All @@ -40,7 +40,7 @@
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

@Slow
@LuceneTestCase.Nightly
public class ChaosMonkeySafeLeaderWithPullReplicasTest extends AbstractFullDistribZkTestBase {
private static final Logger log = LoggerFactory.getLogger(MethodHandles.lookup().lookupClass());

Expand Down
Expand Up @@ -23,12 +23,10 @@
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicInteger;
import org.apache.lucene.tests.util.LuceneTestCase.Slow;
import org.apache.solr.client.solrj.SolrClient;
import org.apache.solr.cloud.api.collections.ShardSplitTest;
import org.apache.solr.common.SolrInputDocument;
import org.apache.solr.common.cloud.ClusterState;
import org.apache.solr.common.cloud.DocCollection;
import org.apache.solr.common.cloud.DocRouter;
import org.apache.solr.common.cloud.Replica;
import org.apache.solr.common.cloud.Slice;
Expand All @@ -47,7 +45,6 @@
import org.slf4j.LoggerFactory;

/** Test split phase that occurs when a Collection API split call is made. */
@Slow
@Ignore("SOLR-4944")
public class ChaosMonkeyShardSplitTest extends ShardSplitTest {

Expand Down Expand Up @@ -216,28 +213,21 @@ public void run() {
}

private void waitTillRecovered() throws Exception {
for (int i = 0; i < 30; i++) {
Thread.sleep(3000);
ZkStateReader zkStateReader = ZkStateReader.from(cloudClient);
zkStateReader.forceUpdateCollection("collection1");
ClusterState clusterState = zkStateReader.getClusterState();
DocCollection collection1 = clusterState.getCollection("collection1");
Slice slice = collection1.getSlice("shard1");
Collection<Replica> replicas = slice.getReplicas();
boolean allActive = true;
for (Replica replica : replicas) {
if (!clusterState.liveNodesContain(replica.getNodeName())
|| replica.getState() != Replica.State.ACTIVE) {
allActive = false;
break;
}
}
if (allActive) {
return;
}
}
printLayout();
fail("timeout waiting to see recovered node");
ZkStateReader zkStateReader = ZkStateReader.from(cloudClient);
zkStateReader.waitForState(
"collection1",
90,
TimeUnit.SECONDS,
(n, c) -> {
Collection<Replica> replicas = c.getSlice("shard1").getReplicas();
for (Replica replica : replicas) {
if (n.contains(replica.getNodeName()) == false
|| replica.getState() != Replica.State.ACTIVE) {
return false;
}
}
return true;
});
}

// skip the randoms - they can deadlock...
Expand Down
Expand Up @@ -22,7 +22,6 @@
import java.util.Locale;
import java.util.concurrent.TimeUnit;
import org.apache.commons.io.FileUtils;
import org.apache.lucene.tests.util.LuceneTestCase;
import org.apache.solr.client.solrj.embedded.JettySolrRunner;
import org.apache.solr.client.solrj.request.CollectionAdminRequest;
import org.apache.solr.common.cloud.DocCollection;
Expand All @@ -33,7 +32,6 @@
import org.junit.BeforeClass;
import org.junit.Test;

@LuceneTestCase.Slow
public class CleanupOldIndexTest extends SolrCloudTestCase {

@BeforeClass
Expand Down Expand Up @@ -64,8 +62,7 @@ public void test() throws Exception {
// TODO make this configurable on StoppableIndexingThread
cluster.getSolrClient().setDefaultCollection(COLLECTION);

int[] maxDocList = new int[] {300, 500, 700};
int maxDoc = maxDocList[random().nextInt(maxDocList.length - 1)];
int maxDoc = atLeast(300);

StoppableIndexingThread indexThread =
new StoppableIndexingThread(null, cluster.getSolrClient(), "1", true, maxDoc, 1, true);
Expand Down Expand Up @@ -123,7 +120,7 @@ public void test() throws Exception {
TimeUnit.SECONDS,
(n, c) -> DocCollection.isFullyActive(n, c, 1, 2));

assertTrue(!oldIndexDir1.isDirectory());
assertTrue(!oldIndexDir2.isDirectory());
assertFalse(oldIndexDir1.isDirectory());
assertFalse(oldIndexDir2.isDirectory());
}
}
Expand Up @@ -18,7 +18,6 @@

import java.util.Map;
import java.util.Set;
import org.apache.lucene.tests.util.LuceneTestCase.Slow;
import org.apache.solr.client.solrj.embedded.JettySolrRunner;
import org.apache.solr.client.solrj.request.CollectionAdminRequest;
import org.apache.solr.common.cloud.ClusterState;
Expand All @@ -30,22 +29,17 @@
import org.junit.BeforeClass;
import org.junit.Test;

@Slow
public class ClusterStateUpdateTest extends SolrCloudTestCase {

@Override
public void setUp() throws Exception {
super.setUp();
configureCluster(3).addConfig("conf", configset("cloud-minimal")).configure();
}

@BeforeClass
public static void beforeClass() {
public static void beforeClass() throws Exception {
System.setProperty("solrcloud.skip.autorecovery", "true");
configureCluster(3).addConfig("conf", configset("cloud-minimal")).configure();
}

@AfterClass
public static void afterClass() {
public static void afterClass() throws Exception {
shutdownCluster();
System.clearProperty("solrcloud.skip.autorecovery");
System.clearProperty("genericCoreNodeNames");
}
Expand Down
Expand Up @@ -28,7 +28,6 @@
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.concurrent.atomic.AtomicReference;
import org.apache.lucene.tests.util.LuceneTestCase;
import org.apache.solr.SolrTestCaseJ4;
import org.apache.solr.client.solrj.request.CollectionAdminRequest;
import org.apache.solr.client.solrj.response.CollectionAdminResponse;
Expand All @@ -42,7 +41,6 @@
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

@LuceneTestCase.Slow
@SolrTestCaseJ4.SuppressSSL
public class CollectionPropsTest extends SolrCloudTestCase {
private static final int TIMEOUT = 5000;
Expand Down