New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

testing: add (*T).Deadline #28135

Open
bcmills opened this Issue Oct 10, 2018 · 7 comments

Comments

Projects
None yet
7 participants
@bcmills
Member

bcmills commented Oct 10, 2018

Background

#16221 added a func (*T) Context() context.Context to the testing package, which was backed out due to concerns from @dsnet, @neild, @niemeyer, me, and perhaps others as well (#18199).

CL 134395 (errgroup propagation of Goexit, among other things) mitigates those concerns somewhat: it gives test authors a straightforward way to start a collection of goroutines and wait for them to finish, and cancels them if any of the goroutines calls t.FailNow or t.SkipNow (#15758 notwithstanding).

That leaves one use-case unaddressed: as @neild noted in the golang-dev thread and @dsnet noted in #18199 (comment), if a test times out (e.g. because of a deadlock in the package under test), it is often useful to shut it down cleanly and emit useful logs rather than wait for the testing package to dump all of the running goroutines.

@bradfitz suggested that test authors can add their own derived context, but a test in general has no idea how much time it has left: if we want to check the -test.timeout flag, we need to be able to subtract off the amount of time spent so far, and that implies the use of TestMain and the boilerplate that goes with it.


Proposal

I propose that we add back the (*testing.T).Context method, with the same signature as before but the following semantics:

  • When a test binary times out:
    • First, all tests that are still running are marked as having failed due to timing out.
    • Then, the contexts provided to tests are marked as Done, with Err returning context.DeadlineExceeded.
    • Finally, the test binary waits for some “reasonable” (but unspecified) grace period elapses, or until all tests that called their Context method (and their subtests) have returned, whichever occurs first.
    • If any tests are still running at that point, the test binary writes out the logs for any completed tests and then panics as before.
  • When a test is marked as Failed or Skipped:
    • The context provided to that test (if any) is marked as Done, with Err returning context.Canceled.
  • When a test function exits normally:
    • The context provided to that test is not marked as Done. That discourages the use of (*T).Context for normal cleanup of “background” goroutines, which should be accomplished by some other means (sync.WaitGroup, errgroup, tomb, or whatever alternative the test author prefers.)

@bcmills bcmills added this to the Proposal milestone Oct 10, 2018

@neild

This comment has been minimized.

Contributor

neild commented Oct 10, 2018

Another option would be to add a t.Deadline and leave context creation up to the test.

@bcmills

This comment has been minimized.

Member

bcmills commented Oct 10, 2018

t.Deadline would be somewhat more annoying to use, but acceptable for my use-cases.

@deanveloper

This comment has been minimized.

deanveloper commented Oct 15, 2018

The way I see it, current testing is very simple and I like it that way. Adding a t.Deadline I think is a good idea, however. But contexts are big constructs that can make things complicated quick, especially when looking at other people's code. Probably better to just have a t.Deadline() since that is much simpler.

@rsc

This comment has been minimized.

Contributor

rsc commented Oct 17, 2018

The last attempt at adding Context was trying to signal "this test is done, all you helpers can go away".
The new attempt is trying to signal - using exactly the same mechanism! - "this test is about to be killed, you might want to die with a more useful message".

I don't see why we should prefer one over the other. They both seem very indirect. The fact that there were these two very different plausible uses suggests that having plain t.Context will be confusing to people expecting one or the other (or a third equally plausible meaning).

If the goal is to expose timeout information, let's focus on an API that does that well, instead of shoehorning it into context.

@andybons

This comment has been minimized.

Member

andybons commented Oct 31, 2018

@rsc are you OK with t.Deadline?

Per @golang/proposal-review

@Capstan

This comment has been minimized.

Capstan commented Dec 11, 2018

This appears to be related, if not a resurrection of #18368 which was FrozenDueToAge. I'm adding this comment here to act as a trail of breadcrumbs, hoping that the link will show up in the older proposal.

@rsc

This comment has been minimized.

Contributor

rsc commented Dec 12, 2018

Sure, let's try adding t.Deadline for Go 1.13.

@rsc rsc changed the title from proposal: testing: add (*T).Context, canceled only on timeout, failure, or skip to proposal: testing: add (*T).Deadline Dec 12, 2018

@rsc rsc modified the milestones: Proposal, Go1.13 Dec 12, 2018

@ianlancetaylor ianlancetaylor changed the title from proposal: testing: add (*T).Deadline to testing: add (*T).Deadline Dec 12, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment