//src/test/shell/bazel:bazel_test_test does not pass

[jasharpe]

Steps to reproduce:

cd $(mktemp -d) && \
git clone https://bazel.googlesource.com/bazel && \
cd bazel && \
bazel build //src:bazel && \
cp -f bazel-bin/src/bazel /tmp/test-bazel && \
/tmp/test-bazel test //src/test/shell/bazel:bazel_test_test --test_output=errors

Partial output (didn't copy the whole thing since it contains directory information):

test_flaky_test FAILED: Expected regexp PASS: //:flaky not found exactly once .
...
test_run_under_path FAILED: Expected failure .

Am I doing something wrong or are these tests broken at HEAD?

Platform: Ubuntu

[philwo]

I can repro this on my machine. Trying to find the culprit now.

[philwo]

I couldn't find a single revision where this test actually passes. Weird.

[aehlig]

@philwo 87cc92e is a good revision on FreeBSD (tested this several hundred times when trying to reproduce #3630).

[jasharpe]

I think I figured this out - the issue is that some .bazelrc file is leaking into the test. I happen to have on my machine a bazelrc in /etc/bazel.bazelrc that I believe causes the tests to fail because it enables the --experimental_ui flag (for test_flaky_test) and uses --test_env (I suspect for test_run_under_path).

The first-order fix is to just add --nomaster_bazelrc to some of the bazel invocations in test_run_under_path and test_flaky_test, which makes these tests work on my system again. Alternately, explicitly adding --noexperimental_ui also works for test_flaky_test.

Of course this ignores the issue that the tests won't be compatible with the new UI, which is another issue. And the deeper issue that tests should be more hermetic than this!

(I'm not entirely clear on the distinction between --master_bazelrc and --bazelrc, but only the master version is required to fix this test.)