Remote debugging support on artiq_master

[evilmav]

ARTIQ Feature Request

Problem this request addresses

As of now, when submitting to artiq_master, AFAIK the typical development workflow is limited to print-debugging. Although debugging using breakpoints is problematic in ARTIQ due to eating up slack inbetween kernel executions, being able to inspect the state might nevertheless be quite useful and the former is not an issue anywhere before break_realtime() or outside run() anyway.

It would be nice to have a supported and documented method of debugging the host side of the experiment, ideally including the required IDE setup.

Describe the solution you'd like

There seem to be two approaches worth investigation:

• add debugger mode args to artiq_client, including a port number, that it would send to artiq_master. Attach debugpy to the experiment once in execution and wait for the IDE to connect using .wait_for_client() before continuing
• allow the whole master to be run under debugpy (which is not possible out-of-the-box right now, for one due to artiq_master being a wrapper not understood by -m debugpy

where the first one seems to be preferable in multi-user environment, as it could provide at least some isolation between the users such that the IDE will not stop the whole master with a single click of the stop button, but I am not sure if this can be implemented with debugpy this way. In both cases, setting up the IDE to call artiq_client at execution and attaching to the IP of interest should not be an issue.

Additional context

In an ideal world, there would be a way to stop the rtio counter when a breakpoint is encountered on the host...

[sbourdeauducq]

• for one due to artiq_master being a wrapper not understood by -m debugpy

You can run the master (or any other ARTIQ tool) without the wrapper as artiq.frontend.artiq_master e.g. python -m artiq.frontend.artiq_master. Does it solve this problem?

The worker subprocesses won't be run through debugpy I suppose, though.