adapter: multiple concurrent sessions

[int19h]

The adapter process currently has a 1:1 mapping to DAP process, and the associated debug session. For #1706, we need to change this to 1:N.

DAP requires a separate connection for every process / debug session, so we will still have separate communication channels for each one. And for every connection from the IDE to the adapter, there's a corresponding connection from the adapter to the debug server inside the debugged process. Specific requirements for the IDE and the server will be described below.

[int19h]

Connections between the adapter and the IDE

Since stdin/stdout can only be used for one channel at a time, all the extra channels will have to be socket channels, utilizing client access tokens (#1711) for security. The stdio channel is not really necessary at that point, and we could do everything over sockets - but we already have it implemented, and DAP clients other than VS and VSCode might not support connection to the server via sockets. Also the DAP spec still defines stdio as the DAP transport. So it's best to keep it supported.

Thus, for "launch" scenarios, we will keep the current arrangement for single-proc : the IDE launches the adapter directly, and uses stdio for communication. This will also be the case for "launch" that involves multiple processes, but only for the first process; with the child processes, the adapter sends notifications (ptvsd_subprocess) about them to the IDE, and the IDE then starts an "attach" debug session for that process. This logic is already implemented in vscode-python.

For "attach", both single-proc and multiproc, there will be no local adapter at all. Instead, the adapter is spawned on the remote side (#1712), and the IDE will connect directly to it over a socket. This will require a small change to DebugAdapterFactory shim in vscode-python: it currently always spawns the adapter, but now it will need to route "attach" debug configs via DebugAdapterServer instead.

We'll build on the existing debugServer support in the adapter for socket connections. The existing implementation always listens on 127.0.0.1, and only accepts a single connection before closing the listener. This needs to be changed to listen on the specified host (but still default to 127.0.0.1), and to accept connections repeatedly. It also needs to add support for client access tokens for security reasons.

[int19h]

Connections between the adapter and the debug server (ptvsd/pydevd)

All of these connections are over sockets - the only difference is that instead of having zero or one connection, we need to support zero or more connections.

As before, the direction of the connection can be either adapter->server or server->adapter. However, different connections can have different directions. Furthermore, in multiproc mode (or perhaps always, for simplicity?) even if the initial connection is adapter->server, the adapter must still start a listener for future server->adapter connections from subprocesses.

Regardless of the direction, the adapter will need to pass the server access token, and to validate the client access token (#1710).

Per #1708, pydevd will now be sending information about the debug server listener port in the "initialize" event, when a new subprocess first connects to the adapter. The adapter will need to extract that information and preserve it for re-connection purposes.

[int19h]

Debug sessions

Every channel between the adapter and the IDE has a corresponding channel between the adapter and the debug server - the mapping is N:1:N. Every pair of channels defines a logical debug session, seen as a separate debugged process from within the IDE.

This means that some things that are currently treated as singletons in the adapter will now have multiple instances, one for each session:

1. State.
2. Channels.
3. IDEMessages and ServerMessages.
4. ProcessTracker (because individual processes can receive "terminate" requests that should not affect other processes).
5. The debuggee module (this will become a class).

We will likely need a new class, Session, that represents a debug session as a whole - this would be the one that creates instances of everything from the list above, and through which they communicate with each other.

The only global state in the adapter will then be a list of Session objects, and the state associated with the listener sockets on which it accepts incoming connections from the IDE and the servers.