re-enable test timeout, but factor in nightly, multiplier, clover [LUCENE-4301]

[asfimport]

Having a test timeout is great (for things like test hangs), but its going to cause a lot of failed builds until we factor in the various flags that can cause tests to run longer, or enable longer-running tests.

Its hard to say what the limits should be since jenkins history is pretty limited at the moment (only 2 nightly runs history available)

---

Migrated from LUCENE-4301 by Robert Muir (@rmuir), updated Aug 13 2012
Attachments: evil.jpg

[asfimport]

Robert Muir (@rmuir) (migrated from JIRA)

I think the best would be a sysprop here.

I imagine we might need different limits for e.g. Uwe's jenkins runs on windows
than the freebsd one at apache and what not.

[asfimport]

Uwe Schindler (@uschindler) (migrated from JIRA)

You can look into the Artifacts build history. Those were the nightly runs before. I just factored the test runs out to the nightly jobs.

[asfimport]

Dawid Weiss (@dweiss) (migrated from JIRA)

Let's take the maximum we know of and re-enable. Timeouts are meant to end hung tests that nobody looks at (jenkins). Even two hours per suite is fine as long as it is terminated at some point.

[asfimport]

Dawid Weiss (@dweiss) (migrated from JIRA)

I should probably add that there is no way to calculate the timeout value from the code because the decisions take place before the test code is started. It could be a system property override but then you'd have to set it properly based on your jenkins configuration, etc.

I'll re-enable for max. suite time = 2 hours? Alternatively override @TimeoutSuite on a particular test class that you know lasts longer than that.

[asfimport]

Robert Muir (@rmuir) (migrated from JIRA)

ok, 2 hours sounds good. and Test2BTerms should have integer.max_value :)

[asfimport]

Dawid Weiss (@dweiss) (migrated from JIRA)

Uh, really?! Damn. :) How long does it take? If it takes > 1 day I don't think it's a good test, really :)

[asfimport]

Uwe Schindler (@uschindler) (migrated from JIRA)

The problem with this test its that it must create an index with > 2 billion terms...

With some similar problems regarding overflow of document numbers, I used a trick in another test:

IndexReader small = ...
IndexReader[] arr1 = new IndexReader[count];
Arrays.fill(arr1, small);
MultiReader intermediate = new MultiReader(arr1);
IndexReader[] arr2 = new IndexReader[count];
Arrays.fill(arr2, intermediate);
MultiReader huge = new MultiReader(arr2);

Producing a composite reader with count * count * small.maxDoc() docs, but spending almost no RAM. Maybe we can do something similar here (wrapping this huge synthetic reader with SlowMultiReaderWrapper)?

[asfimport]

Robert Muir (@rmuir) (migrated from JIRA)

This test really needs 2 billion unique terms.

[asfimport]

Dawid Weiss (@dweiss) (migrated from JIRA)

:)

[asfimport]

Robert Muir (@rmuir) (migrated from JIRA)

hehehe. But uwe's trick might speed up Test2BPostings a bit, if we are careful.

it only cares about 2B term/doc pairs in a segment, so we could build a reader
tree like this from one real segment and forceMerge it: maybe it would speed it up some.