Evaluating expression of large objects from the debug console is extremely slow

[dobos]

I'm seeing an issue that apparently hasn't been reported yet. I have a fairly large dictionary in memory and when I want to get its length from the debug console it takes tens of seconds. Obviously, getting the repr of the dict takes that much time but I only want to get its length.

Code to reproduce:

from unittest import TestCase
import numpy as np
from types import SimpleNamespace

class MyTest(TestCase):
    def test_large_defaultdict(self):
        d = defaultdict(lambda: defaultdict(list))
        for k in 'abcdef':
            for l in 'ghijk':
                for i in range(10000):
                    d[k][l].append(SimpleNamespace(
                        key = k,
                        cycle = np.arange(20),
                        value = np.zeros(20)
                    ))

        pass

Set breakpoint on pass and try to evaluate len(d) from the debug console after the breakpoint is hit. This warning is printed in the console:

pydevd warning: Computing repr of d (defaultdict) was slow (took 42.36s)
Customize report timeout by setting the `PYDEVD_WARN_SLOW_RESOLVE_TIMEOUT` environment variable to a higher timeout (default is: 0.5s)

Yes, it is slow but why is repr called to print the number of items?

[dobos]

The culprit can be the variables window (auto-watch). I've tried hiding it but it didn't help.

[connor4312]

pydevd warning: Computing repr of d (defaultdict) was slow (took 42.36s)

This looks like it's on the Python debugger side @eleanorjboyd

If there is renderer issues: VS Code and DAP support paginating arrays via indexedVariables, I recommend the Python debugger adopt this if it hasn't already to avoid slowdowns.

[eleanorjboyd]

Thanks Connor for taking a look.

Ill investigate this @dobos, one question; if you don't run "len(d) from the debug console after the breakpoint is hit" do you still see the pydevd warning? Trying to understand if you are saying the calculation is done by the variables window or only once you take the action