Add support for PDB debugging module

[slush0]

Hello,

I'd like to see support for debugger in Micropython. I know it has been discussed before, but I cannot find any conclusion on that topic from Micropython team. Is even possible to implement PDB support in current Micropython codebase?

If so, what is needed to make it happen? Do the project need more manpower for this task or is it matter of priority or money?

[dpgeorge]

Is even possible to implement PDB support in current Micropython codebase?

Everything is possible :) In the case of PDB I think it is difficult to implement it and requires significant changes/additions to the VM.

what is needed to make it happen

It's not a priority and to get it started would require someone (likely an additional person to the current team) to perform some initial research and analysis to propose ways to implement PDB support.

[slush0]

Well, it is getting priority for us in TREZOR; we're building Micropython-based framework for crypto devices and having a debugging features would be really helpful not only for us, but also for framework users later.

Considering it is some priority for us and I expect it would be also useful addition into Micropython environment, can you steer me what's the best choice to make it done? I see three general options - open a bounty for pull request, donate directly to Micropython team or implement it by ourselves (least feasible as we don't have free capacity now). What do you prefer? You mentioned some research, so maybe it won't be so straightforward as I imagine, so internal research would be necessary anyway?

[dpgeorge]

You mentioned some research, so maybe it won't be so straightforward as I imagine, so internal research would be necessary anyway?

Yes it requires some investigation first before implementation could proceed. Such an investigation can really only be done by someone who has deep knowledge of the internals of the VM and runtime.

I just took a quick look at CPython's pdb module. It basically requires 3 layers to implement a debugger:

1. addition of sys.settrace() function to register a Python callback to execute at each event (events are call, return, next line, exception, etc)
2. creation of low-level bdb module (written in Python) to implement core debugging capabilities, using sys.settrace
3. creation of high-level pdf module (written in Python) to provide a user-friendly interface to bdb

Each of these steps are non-trivial. Point 1 requires modest changes to the VM and runtime to intercept all the relevant events. It would make the VM less efficient (execution of each opcode needs a check if the trace function is set) so would need to be a compile time option that's disabled by default. It would also require extra work to expose the names of local variables for the debugger (currently, once the code is compiled, the local variable names in a function are lost).

Points 2 and 3 require a fair bit of work, and probably it's easiest to start with CPython's implementation and adapt that.

I have no doubt that pdb would be a useful feature. At this stage the main thing holding back an implementation is lack of time from someone familiar enough with the code.

[pfalcon]

def foo():
    a = 1
    loc = locals()
    print(loc)
    a = 2
    print(loc)

foo()

{'a': 1}
{'a': 1}

So, we're lucky that Python's locals() doesn't return a live view into local vars, but rather a current snapshot. So, implementing it in uPy would be easier.

[m-farahmand]

+1

[dpgeorge]

sys.settrace() is now implemented, see #5026

[kevinjwalters]

This is likely to become more of an issue in one of MicroPython's descendents, CircuitPython, as the boards are getting "bigger" now and this is allowing much larger applications to be written.

[mytechnotalent]

>>> help(sys)
object <module 'sys'> is of type module
  __name__ -- sys
  path -- ['', '/lib']
  argv -- []
  version -- 3.4.0
  version_info -- (3, 4, 0)
  implementation -- (name='micropython', version=(1, 13, 0), mpy=10757)
  platform -- esp32
  byteorder -- little
  maxsize -- 2147483647
  exit -- <function>
  stdin -- <io.FileIO 0>
  stdout -- <io.FileIO 1>
  stderr -- <io.FileIO 2>
  modules -- {'main': <module 'main' from 'main.py'>, 'flashbdev': <module 'flashbdev' from 'flashbdev.py'>}
  print_exception -- <function>
>>> sys.settrace()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'module' object has no attribute 'settrace'

@dpgeorge I do not see this in the current build was this removed? Any update on progress of adding PDB to the platform?

[int19h]

MicroPython must be built with MICROPY_PY_SYS_SETTRACE=1 for sys.settrace() to be available.

[mytechnotalent]

Thanks @int19h I will git that a try.

[Omid888]

Having (any) debugger in micropython is a "must" for me too. print statement is not good enough, and sometimes not helpful at all. I wish I could help!

[andrewleech]

I used to think the lack of debugger would be a deal-breaker, but in fact found the new ways of working that are possible changed the way I code. Things like Jupiter notebook live connected to device, and test driven development over mpremote mount have been great, but the thing that beats it all has been aiorepl running in my production app, so I can live interact with it while running.

That being said I still use gbd a lot while working on C extensions or internal C code updates and there are some things I'd love the python debugger in vscode to just work for, but have been very low on personal dev time to work in things like this.

[jonnor]

There is now a draft MR at micropython/micropython-lib#499