Treat OneTimeSetup and OneTimeTearDown as separate work items

[rprouse]

Issue #1062 and pr #1086 highlight a minor problem with the fact that the setup and tear down are treated very similarly, the output from the setup ends up showing up after the tests which is potentially confusing.

If this is fixed, the fix in pr #1086 should be updated to output the setup before the tests.

[CharliePoole]

I'll enlarge my comment in PR #1086 so that we know what we are talking about here. The problem of ordering of text output is minor but the design change I refer to is a major one, justified by much broader considerations that text output ordering.

Before executing tests, NUnit translates them into test commands. Each test generates a compound test command, chaining together several simple commands like context initialization, setup/teardown and test execution. When it comes time to execute the commands each one is attached to a WorkItem, which is the unit of scheduling for the internal dispatcher. In general, one work item corresponds to one command, which corresponds to one test.

In the case of CompositeWorkItems, this pattern is a bit different. One such item corresponds to two commands, a SetUpCommand and a TearDownCommand, which in turn are associated with the same test suite. Essentially, a CompositeWorkItem executes in two phases. In the first phase, it performs setup and launches all the individual test commands. In the second phase, after the tests have completed, it performs teardown and reports its result.

The change we are talking about means that we would create two work items for each test suite. One would be associated with the SetUpCommand and the other with the TearDownCommand. Each of them would have separate completion events, thereby resolving the output issue. More importantly, this would have much wider consequences, because it would replace the implicit dependency of the teardown on all the tests by an explicit dependency, tracked by the dispatcher.

This explicit dependency is important to us because it removes an obstacle to running test methods in parallel with one another. Additionally, the same mechanism can be used to implement general dependencies between tests themselves.

[oznetmaster]

Actually, wouldn't this imply three work items. OneTimeSetUp, execute Test items, and then OneTimeTearDown?

Same dependency process would be in effect All test items would depend on completion of the setup, and the teardown would depend on the completion of all test items.

[CharliePoole]

Yes, except that execute Test items is n work items.

More exactly:

1. SetUp/Test/Teardown is one item - call this a test item.
2. If there is a OneTimeSetUp item in the fixture, the test items all depend on it.
3. If there is a OneTimeTearDown in the fixture, it depends on all test items.
4. A OneTimeSetUp item may depend on higher level OneTimeSetUp items.
5. A OneTimeTearDown item may depend on lower level OneTimeTearDown items.

The nature of the above dependencies is one of two types:

• for 2 and 4, it the dependent item requires successful completion of the items depended on.
• for 3 and 5, the dependent item runs even if the item depended on throws an exception.

Issue #51 may create additional relations and "flavors of relationships.

[oznetmaster]

Exactly.