Support retiring test states

[hbenl]

I couldn't test this code (yet) because the latest insiders build doesn't implement the latest version of the testing API yet (and I don't know how to get an older vscode.proposed.d.ts).

[hbenl]

Fixes #8

[hbenl]

I forgot to mention: this assumes that TestItem.invalidateResults() works recursively (because the retire event may contain suite IDs).

[hbenl]

I managed to test it now, but VSCode doesn't seem to react to TestItem.invalidateResults().

[hbenl]

I have update this PR to the latest API. TestItem.invalidateResults() still doesn't seem to do anything.

[connor4312]

Thanks for the PR. I actually want to remove, or at least rethink, invalidation in the new APIs. The Test Explorer uses it as a UI hint and (or primarily?) to support autorun. However, in the native APIs I would rather delegate autorun to extensions themselves since most underlying runners already have a notion of autorun built-in (like mocha --watch for example). For test runners that don't, implementing autorun yourself is not much harder than implementing the invalidation which generic autorun requires. For this reason, invalidation was not one of the APIs that is going to stable for this release.

I want to tackle this next month and would be interested to hear your thoughts 🙂

[hbenl]

The only reason why autorun is handled by the Test Explorer extension is that it's possible to enable autorun only for some tests - the adapter doesn't have that information, so it sends an invalidation event to the Test Explorer and lets it decide which of the invalidated tests should be run. Without this feature (i.e. if autorun can only be enabled for all tests or none) it probably makes more sense to let the adapter handle it.
Invalidation is not only about autorun, however: the initial motivation for that feature was a use case that is very important to me: slow tests. If running all tests takes a minute or longer, the test results become more "valuable" since it takes some time to recreate them. So I don't want the information what the last test result was to just disappear. But it's also essential to show that a test result isn't valid anymore. Hence the need for invalidated test results in the UI.

[hbenl]

Apparently tests are now only retired/invalidated when I restart VS Code. Is that correct? Are there any plans to support retiring tests programmatically or in response to some events other than restarting VS Code?

[connor4312]

Unfortunately some other high priority work has come up that has delayed work on microsoft/vscode#134941 and, by proxy, this. Currently I don't plan to support additional notion of programmatic retirement, however that may change after initial implementation of autorun based on feedback I get from extension authors.