testing: provide access to initial thread from tests

[niemeyer]

I spent some time on GopherCon trying to figure how to make the qml package work
smoothly on Mac OS. Apparently it was working fine out of luck, because the OS requires
some calls to be made from the initial thread. Now something unrelated was touched, and
the graphic goroutine is landing on a different thread and generating errors.

The solution is simple: an init function that locks the main thread, and a qml.Run
function that runs the main loop. Problem solved.

Except, that doesn't work with the testing package anymore, because the test logic
itself is run at the end of main, and tests are moved on their own goroutine.

This problem might be easily solved by changing the test template from:

  testing.Main(matchString, tests, benchmarks, examples)

to something like:

  go func() {
          testing.Main(matchString, tests, benchmarks, examples)
          close(done)
  }()
  if testing.RunFromMain != nil {
          testing.RunFromMain(done)
  }
  <-done

Then, for the qml package, I'd implement RunFromMain along the lines of:

  testing.RunFromMain = func(done chan struct{}) {
          stopWhenClosed(done)
          qml.Run(nil)
  }

Can we add such a testing.RunFromMain hook?

Another alternative, suggested by Dmitry Vyukov, is to run tests on the initial thread
instead of spawning them in goroutines. That would solve the problem too.

[dvyukov]

Comment 1:

Labels changed: added repo-main.

[dvyukov]

Comment 2:

---------- Forwarded message ----------
From: Russ Cox <rsc>
Date: Fri, May 2, 2014 at 5:05 PM
Subject: Re: [golang-dev] Main thread function for tests
You can't move test execution onto the main goroutine. t.FailNow uses runtime.Goexit.

[dvyukov]

Comment 3:

You are right.
This would require either starting separate processes (at least 1 process per adjacent
sequence of non-failing tests), or some help from runtime (something like: unlock the
current goroutine from the thread and atomically lock that other goroutine to this
thread).

[niemeyer]

Comment 4:

I also realized that just having tests in main would be sub-optimal for the problem at
hand either way. Ideally the event loop can run for the duration of multiple tests,
rather than starting and stopping it on every single test.
The original solution suggested, with a RunFromMain hook on testing, still sounds like
the best approach for the problem, and it's also very simple.

[dvyukov]

Comment 5:

>Ideally the event loop can run for the duration of multiple tests
How critical is it?
testing.RunFromMain interface looks quite involved to me. More in the spirit of Go's
approach would be:
func TestFoo(t *testing.T) {
  t.OnMainThread()
  ...
}
But this will involve subprocesses, and each test do own setup/teardown.

[dvyukov]

Comment 6:

On a similar note, recently somebody was asking the following feature:
func TestDie(t *testing.T) {
        output, err := t.DeathTest(func() { log.Fatal("Die") } )
        if err != nil || !regexp.Match("Die", output) {
                t.Fatal(...)
        }
}
Such "death tests" also must run in a subprocess. So probably we can figure out a single
testing package extension that covers both cases.

[niemeyer]

Comment 7:

Although I did manage to make the main event loop stop in the qml package, the main
event loop of many GUI toolkits are designed to run once and for the duration of the
application. I'd prefer to not fix the go test problem in a way that makes doing that
impossible, as it may uncover bugs that few people care about now or in the future.
The RunFromMain interface is both simple to implement, and simple to use. With it, I
will offer a function that may be called as:
func init() {
        qml.SetupTesting()
}
and then tests can just use the qml package as usual, for example.

[rsc]

Comment 8:

FWIW, I thought about this for a long time and still have no idea how to do it.

Labels changed: added release-go1.4.

Status changed to Accepted.

[niemeyer]

Comment 9:

I've been holding back the fix for qml a bit too long, so I'm releasing v1 of the API
with an in-memory code patching hack for the time being. It's exclusive to testing and
fairly isolated, so not too bad:
http://blog.labix.org/2014/08/13/announcing-qml-v1-for-go
Is there anything I can do to help refining the actual solution?

[rsc]

Comment 10:

Status changed to Duplicate.

Merged into issue #8202.