Server side nightly test results incorrect without-enable-llcbeta

[llvmbot]

Bugzilla Link	949
Version	trunk
OS	All
Reporter	LLVM Bugzilla Contributor

Extended Description

Currently the nightly tester's option -enable-llcbeta doesn't do anything. You
get the llcbeta test whether you use this option or not. While it is probably a
reasonable default to have llcbeta turned on by default, there should be a way
to turn it off.

I would prefer that the llcbeta was on by default and a -disable-llcbeta option
was provided, similar to -disable-jit, -disable-cbe, etc.

[lattner]

makes sense to me

[llvmbot]

The cause is not so much the -enable-llcbeta flag. That flag is being passed
correctly to the llvm-test suite when it is specified. The problem is the server
side display doesn't know about this switch.

When results are displayed from a run that has not used -enable-llcbeta, the
"Test Suite Failures" section claims that "LLC-BETA compile" and "LLC-BETA" have
failed for every test. This isn't correct. If -enable-llcbeta was not specified,
it should not count these as failures because the tests were never run.

See: http://llvm.org/nightlytest/test.php?machine=71&night=1459 for an example
of this.

The problem is that it makes it very difficult to see the trees for the forest.
With every test reporting something failing, you can only discern the actual
failures by reading the individual failure reports and discounting the ones that
only say "LLC-BETA compile" and "LLC-BETA".

There should be a fairly simple fix to this. Have the NewNightlyTest.pl script
report to the server side whether -enable-llcbeta was specified or not. On the
server side, if this flag was not set, do two things:

a) Don't report "LLC-BETA compile" and "LLC-BETA" as "failing"
b) Don't generate any LLC-BETA related columns in the detailed results.

tests have failed when they weren't even run.

[bwendling]

I'm guessing that this is no longer an issue given the current build farms we have.