Spyder 5.4.3 shows a 30x slowdown in debug mode on pandas functions

[henryeck]

Description

What steps will reproduce the problem?

import datetime
import pandas as pd

def pdwork():
df = pd.DataFrame({'a':[0,1,2,3,4,5]})
df['b'] = df['a'] + 1
df1 = df - 3
df2 = df1.combine_first(df1)
return

def spinpd(n):
for i in range(n):
pdwork()
return
start_time = datetime.datetime.now()

spinpd(1000)

end_time = datetime.datetime.now()
print(end_time - start_time)

run the above in debug/not using Spyder 5.4.3 vs 5.2.2

the latter shows a slowdown of about 4x, the former 30x

running python 3.10.6 vs 3.9.12 seems not to matter

nor does pandas 2.0.0 vs earlier; it's the diff versions of spyder that make the difference

Versions

• Spyder version: 5.2.2
• Python version: 3.10.6
• Qt version: 5.12.10
• PyQt5 version: 5.12.3
• Operating System: Linux 5.19.0-40-generic

Dependencies

# Mandatory:
atomicwrites >=1.2.0          :  1.4.1 (OK)
chardet >=2.0.0               :  5.1.0 (OK)
cloudpickle >=0.5.0           :  2.2.1 (OK)
cookiecutter >=1.6.0          :  2.1.1 (OK)
diff_match_patch >=20181111   :  20200713 (OK)
intervaltree >=3.0.2          :  3.1.0 (OK)
IPython >=7.6.0;<8.0.0        :  7.34.0 (OK)
jedi >=0.17.2;<0.19.0         :  0.18.2 (OK)
jellyfish >=0.7               :  0.11.2 (OK)
jsonschema >=3.2.0            :  4.17.3 (OK)
keyring >=17.0.0              :  23.13.1 (OK)
nbconvert >=4.0               :  7.3.1 (OK)
numpydoc >=0.6.0              :  1.5.0 (OK)
parso >=0.7.0;<0.9.0          :  0.8.3 (OK)
pexpect >=4.4.0               :  4.8.0 (OK)
pickleshare >=0.4             :  0.7.5 (OK)
psutil >=5.3                  :  5.9.5 (OK)
pygments >=2.0                :  2.15.1 (OK)
pylint >=2.5.0                :  2.17.2 (OK)
pyls_spyder >=0.4.0           :  0.4.0 (OK)
pylsp >=1.3.2;<1.4.0          :  1.3.3 (OK)
pylsp_black >=1.0.0           :  1.1.0 (OK)
qdarkstyle =3.0.2             :  3.0.2 (OK)
qstylizer >=0.1.10            :  0.2.2 (OK)
qtawesome >=1.0.2             :  1.2.3 (OK)
qtconsole >=5.2.1;<5.3.0      :  5.2.2 (OK)
qtpy >=1.5.0                  :  2.3.1 (OK)
rtree >=0.9.7                 :  1.0.1 (OK)
setuptools >=49.6.0           :  59.6.0 (OK)
sphinx >=0.6.6                :  6.2.0 (OK)
spyder_kernels >=2.2.1;<2.3.0 :  2.2.1 (OK)
textdistance >=4.2.0          :  4.5.0 (OK)
three_merge >=0.1.1           :  0.1.1 (OK)
watchdog >=0.10.3             :  3.0.0 (OK)
xdg >=0.26                    :  0.28 (OK)
zmq >=17                      :  25.0.2 (OK)

# Optional:
cython >=0.21                 :  0.29.34 (OK)
matplotlib >=2.0.0            :  3.7.1 (OK)
numpy >=1.7                   :  1.24.3 (OK)
pandas >=1.1.1                :  2.0.0 (OK)
scipy >=0.17.0                :  1.10.1 (OK)
sympy >=0.7.3                 :  1.11.1 (OK)

[Artyom-K]

I'm also facing this issue. For me the code above runs for 2 seconds in normal mode and 1.5 minutes in the debug mode. I have Spyder 5.4.3, pandas 1.5.3, Python 3.11.3.

[dalthviz]

Hi @henryeck and @Artyom-K thank you for the feedback! Could this be related with #20571 and specifically the ipython/ipython#13972 IPython issue that was detected?

[Artyom-K]

Hi @henryeck and @Artyom-K thank you for the feedback! Could this be related with #20571 and specifically the ipython/ipython#13972 IPython issue that was detected?

Probably, since in spyder 5.3.3 my own code runs through debugging a lot faster, same for the code from #20571
If needed I would be glad to make some tests if you tell me what to do.

[ccordoba12]

Actually, I think this is more related to issue #20639. Although the title is misleading, we found there that a serious slowdown in the debugger is caused by a new feature added in IPython 8 (which we started to support in Spyder 5.4).

So, for now the workaround is to downgrade to IPython 7.34. We'll provide a real fix for IPython 8 in our next version (5.4.4), to be released in a couple of weeks.

[ccordoba12]

I'm going to close this issue in favor of #20639 and #20571 to consolidate things.