@DanTup nice catch, thanks for filling this. I can reproduce.
I can also reproduce it with the current stable version and the previous stable version. So it seems like we always had this, just people did not hit it. Thus this is not a candidate.
Investigating

I can also reproduce it with the current stable version and the previous stable version. So it seems like we always had this, just people did not hit it.

It hasn't always been this way - it used to print a message in the console like "running tests from the command line is only possible when no instance of VS Code is open" (I always found the message amusing, since I wasn't running from the console - I guess it was triggered by not having the debugger attached - though honestly I don't understand why that limitation exists).

I agree it's not a priority though, the impact is low (though it is a little frustrating for those of us when we do hit it :-))

I can repro this all the time. However I am pretty clueless who is closing the window.
What I found is that it is not the debug service closing the window here

Trying to find out who calls electronService.closeWindow. Also produced no effect.

@weinand @bpasero might you have ideas who might close the first window when debugging EH tests with NoDebug flag?

I cannot reproduce, a new window opens with the extension being debugged. I tried insiders and out of sources, 😩

Oh this is running tests, yeah I see it...

@isidorn seems to be this code:

If you run code with --verbose you see more output:

...
CommandService#executeCommand setContext
  1 passing (10ms)
[main 2019-10-22T15:37:12.011Z] IPC#vscode:exit null
[main 2019-10-22T15:37:12.012Z] Lifecycle#kill()
[main 2019-10-22T15:37:12.013Z] Lifecycle#window.on('closed') - window ID 1
[main 2019-10-22T15:37:12.017Z] Lifecycle#window.on('closed') - window ID 3

The problem is the definition of the environment flag isExtensionDevTestFromCli:

This flag should not become true when launching an extension (or its tests) from the debugger (even if "Start without Debugging" is used).

But the condition environmentService.debugExtensionHost.break is not correct because when "Start without Debugging" is used, the value of "break" is false, but the extension is still launched from the debugger.

The correct condition is environmentService.debugExtensionHost.debugId

Thanks!