Yea, my logic for not finalizing it was that it would mostly be called by other extensions. But it's reasonable to want to call it yourself, I'll fix it up and move it for finalization.

I don't reproduce this issue: https://memes.peet.io/img/21-06-e8b99a9a-a862-4dfb-92b4-a057b9ed18eb.mp4

Can you add instructions?

Is it because the test item passed to this API must be the item in the test explorer?

Use the following code can repro the issue:

vscode.commands.registerCommand('test-provider-sample.runTests', async tests => {
      const root = WorkspaceTestRoot.create(vscode.workspace.workspaceFolders![0]);
      await vscode.test.runTests({ tests: [root], debug: false });
      vscode.window.showInformationMessage('Test run complete');
})

More background

In Java we have a explorer called Java Project explorer which provides the package view similar to other Java IDEs.

Meanwhile, there are some 'shortcut' commands registered to that view, like run tests

my original attempt is trying to create a new test item for the node in Java Project explorer then pass it to test.runTests()

Ah, I see. Currently this works only for tests that are "attached" to an existing item, returned from the TestController, and this is tricky to change:

• Given a free-floating TestItem, we don't know what Controller it's associated with. If your extension registered multiple TestControllers, we'd have no idea which one to invoke.
• runTests can be called from any extension. We guarantee to the TestController that we will only ask it to run tests that it created, which allows it to depend on the custom data in the test. Other extensions may not fulfill this contract. Even barring that, we don't require data to be serializable since it's never transferred to other extension hosts, but that would have to change to make this work.

However, I realize that there's a difficulty for you in this scenario: the test items are requested in a "pull" model from VS Code. If it happens that someone is using your project explorer but never opened the Test Explorer or any documents which would cause items to be requested, it's impossible for you to provide a valid test item to runTests. I'll have to think about how to fix this. Suggestions welcome as well.

Just an idea come up in my mind: make test request has an optional field that can associate with the test controller(s), like a id or something?

I think having an option for making test runs cancellable would work. The runs don't actually require "attached" test items, so you could just create a run manually and do what you need with it, and cancellation provides the necessary control for users.

I think having an option for making test runs cancellable would work. The runs don't actually require "attached" test items, so you could just create a run manually and do what you need with it, and cancellation provides the necessary control for users.

Sounds good. Does that mean the cancellation token will be provided when calling test.createTestRun?

There's now a cancellation token on the TestRun object.