Variable explorer does not work with pandas 1.4 and Spyder Windows installer

[rhkarls]

Issue Report Checklist

• Searched the issues page for similar reports
• Read the relevant sections of the Spyder Troubleshooting Guide and followed its advice
• Reproduced the issue after updating with conda update spyder (or pip, if not using Anaconda)
• Could not reproduce inside jupyter qtconsole (if console-related)
• Tried basic troubleshooting (if a bug/error)
  • Restarted Spyder
  • Reset preferences with spyder --reset
  • Reinstalled the latest version of Anaconda
  • Tried the other applicable steps from the Troubleshooting Guide
• Completed the Problem Description, Steps to Reproduce and Version sections below

Problem Description

The Windows standalone installer for version 5.2.2 comes with python 3.7 and and pandas 1.3.5. When connecting to an interpreter in an environment with pandas 1.4 the dataframe viewer in the variable explorer does not work. Doing the same with an environment with pandas 1.3.x does not produce any error. Might be the same issue with the Mac standalone installers, or any Spyder installation with pandas < 1.4 in the spyder environment.

This error message appears (apologies for doing exactly what it says not to do...but I understand this as an unwanted appearance of this error for dataframes):

Installing Spyder from conda-forge in an environment with python 3.8 and pandas 1.4 the dataframe viewer works, both with environments having pandas 1.4 and pandas 1.3.5. Note that pandas 1.4 requires python 3.8 or higher.

I've only tested this with a very simple dataframe
df = pd.DataFrame([1])

A possible simple solution without patching the variable explorer could be to package the future installers with python 3.8 and pandas 1.4. However, I suppose the issue would still be present with Spyder installed for example via pip or conda together with pandas 1.3.

What steps reproduce the problem?

1. Launch Windows standalone installed Spyder (5.2.2)

2. Connect to environment with pandas 1.4 installed

3. Launch a new interpreter

   import pandas as pd
   df = pd.DataFrame([1])
   

4. Double click df to open the dataframe in the variable explorer

What is the expected output? What do you see instead?

Expected dataframe viewer to open, instead error message appears.

Edit:
The exception from the debug logs appears to be:
2022-01-27 14:36:52,932 [DEBUG] [spyder_kernels.comms.commbase] -> Exception in cloudpickle.loads : Can't get attribute '_unpickle_block' on <module 'pandas._libs.internals' from 'C:\\Users\\user\\AppData\\Local\\Programs\\Spyder\\pkgs\\pandas\\_libs\\internals.cp37-win_amd64.pyd'>

More from the debug log:

2022-01-27 14:36:49,430 [DEBUG] [traitlets] -> execute_input: {'code': 'df=pd.DataFrame([1])', 'execution_count': 3}
2022-01-27 14:36:49,483 [DEBUG] [traitlets] -> execute_reply: {'status': 'ok', 'execution_count': 3, 'user_expressions': {}, 'payload': []}
2022-01-27 14:36:49,515 [DEBUG] [traitlets] -> handle_msg[1d2c16b67f7611ec8eb1ccf9e4c29407]({'header': {'msg_id': '9a0f779a-50c113d9226ca98cd1e0856c_14804_370', 'msg_type': 'comm_msg', 'username': 'username', 'session': '9a0f779a-50c113d9226ca98cd1e0856c', 'date': datetime.datetime(2022, 1, 27, 13, 36, 49, 515032, tzinfo=tzutc()), 'version': '5.3'}, 'msg_id': '9a0f779a-50c113d9226ca98cd1e0856c_14804_370', 'msg_type': 'comm_msg', 'parent_header': {'msg_id': '397a450d-18878a471ff98bae7ef6f17a_10996_127', 'msg_type': 'comm_msg', 'username': 'username', 'session': '397a450d-18878a471ff98bae7ef6f17a', 'date': datetime.datetime(2022, 1, 27, 13, 36, 49, 515032, tzinfo=tzutc()), 'version': '5.3'}, 'metadata': {}, 'content': {'data': {'spyder_msg_type': 'remote_call_reply', 'content': {'is_error': False, 'call_id': '8d8c28fd265041e7b904685592cef265', 'call_name': 'get_namespace_view'}, 'pickle_protocol': 4, 'python_version': '3.9.7 | packaged by conda-forge | (default, Sep 29 2021, 19:15:42) [MSC v.1916 64 bit (AMD64)]'}, 'comm_id': '1d2c16b67f7611ec8eb1ccf9e4c29407'}, 'buffers': [<memory at 0x000002B0B3786648>]})
2022-01-27 14:36:49,515 [DEBUG] [traitlets] -> handle_msg[1d2c16b67f7611ec8eb1ccf9e4c29407]({'header': {'msg_id': '9a0f779a-50c113d9226ca98cd1e0856c_14804_371', 'msg_type': 'comm_msg', 'username': 'username', 'session': '9a0f779a-50c113d9226ca98cd1e0856c', 'date': datetime.datetime(2022, 1, 27, 13, 36, 49, 515032, tzinfo=tzutc()), 'version': '5.3'}, 'msg_id': '9a0f779a-50c113d9226ca98cd1e0856c_14804_371', 'msg_type': 'comm_msg', 'parent_header': {'msg_id': '397a450d-18878a471ff98bae7ef6f17a_10996_127', 'msg_type': 'comm_msg', 'username': 'username', 'session': '397a450d-18878a471ff98bae7ef6f17a', 'date': datetime.datetime(2022, 1, 27, 13, 36, 49, 515032, tzinfo=tzutc()), 'version': '5.3'}, 'metadata': {}, 'content': {'data': {'spyder_msg_type': 'remote_call_reply', 'content': {'is_error': False, 'call_id': '52ae688f32c7403e89e187cb8f1614d9', 'call_name': 'get_var_properties'}, 'pickle_protocol': 4, 'python_version': '3.9.7 | packaged by conda-forge | (default, Sep 29 2021, 19:15:42) [MSC v.1916 64 bit (AMD64)]'}, 'comm_id': '1d2c16b67f7611ec8eb1ccf9e4c29407'}, 'buffers': [<memory at 0x000002B0B3786708>]})
2022-01-27 14:36:49,515 [DEBUG] [traitlets] -> handle_msg[1d2c16b67f7611ec8eb1ccf9e4c29407]({'header': {'msg_id': '9a0f779a-50c113d9226ca98cd1e0856c_14804_374', 'msg_type': 'comm_msg', 'username': 'username', 'session': '9a0f779a-50c113d9226ca98cd1e0856c', 'date': datetime.datetime(2022, 1, 27, 13, 36, 49, 515032, tzinfo=tzutc()), 'version': '5.3'}, 'msg_id': '9a0f779a-50c113d9226ca98cd1e0856c_14804_374', 'msg_type': 'comm_msg', 'parent_header': {'msg_id': '397a450d-18878a471ff98bae7ef6f17a_10996_130', 'msg_type': 'comm_msg', 'username': 'username', 'session': '397a450d-18878a471ff98bae7ef6f17a', 'date': datetime.datetime(2022, 1, 27, 13, 36, 49, 515032, tzinfo=tzutc()), 'version': '5.3'}, 'metadata': {}, 'content': {'data': {'spyder_msg_type': 'remote_call_reply', 'content': {'is_error': False, 'call_id': '80a145386c9f4c44b57e0a657c92877a', 'call_name': 'get_cwd'}, 'pickle_protocol': 4, 'python_version': '3.9.7 | packaged by conda-forge | (default, Sep 29 2021, 19:15:42) [MSC v.1916 64 bit (AMD64)]'}, 'comm_id': '1d2c16b67f7611ec8eb1ccf9e4c29407'}, 'buffers': [<memory at 0x000002B0B3786588>]})
2022-01-27 14:36:52,647 [DEBUG] [spyder.plugins.editor.utils.autosave] -> Autosave triggered
2022-01-27 14:36:52,932 [DEBUG] [traitlets] -> handle_msg[1d2c16b67f7611ec8eb1ccf9e4c29407]({'header': {'msg_id': '9a0f779a-50c113d9226ca98cd1e0856c_14804_382', 'msg_type': 'comm_msg', 'username': 'username', 'session': '9a0f779a-50c113d9226ca98cd1e0856c', 'date': datetime.datetime(2022, 1, 27, 13, 36, 52, 932129, tzinfo=tzutc()), 'version': '5.3'}, 'msg_id': '9a0f779a-50c113d9226ca98cd1e0856c_14804_382', 'msg_type': 'comm_msg', 'parent_header': {'msg_id': '397a450d-18878a471ff98bae7ef6f17a_10996_133', 'msg_type': 'comm_msg', 'username': 'username', 'session': '397a450d-18878a471ff98bae7ef6f17a', 'date': datetime.datetime(2022, 1, 27, 13, 36, 52, 161378, tzinfo=tzutc()), 'version': '5.3'}, 'metadata': {}, 'content': {'data': {'spyder_msg_type': 'remote_call_reply', 'content': {'is_error': False, 'call_id': '7c4f9c52add1402ebee70e5e938302cd', 'call_name': 'get_value'}, 'pickle_protocol': 4, 'python_version': '3.9.7 | packaged by conda-forge | (default, Sep 29 2021, 19:15:42) [MSC v.1916 64 bit (AMD64)]'}, 'comm_id': '1d2c16b67f7611ec8eb1ccf9e4c29407'}, 'buffers': [<memory at 0x000002B0B3786708>]})
2022-01-27 14:36:52,932 [DEBUG] [spyder_kernels.comms.commbase] -> Exception in cloudpickle.loads : Can't get attribute '_unpickle_block' on <module 'pandas._libs.internals' from 'C:\\Users\\user\\AppData\\Local\\Programs\\Spyder\\pkgs\\pandas\\_libs\\internals.cp37-win_amd64.pyd'>

Versions

Versions and dependencies from the standalone installed Spyder

• Spyder version: 5.2.2 None
• Python version: 3.7.9 64-bit
• Qt version: 5.12.10
• PyQt5 version: 5.12.3
• Operating System: Windows 10

Dependencies

# Mandatory:
atomicwrites >=1.2.0          :  1.4.0 (OK)
chardet >=2.0.0               :  4.0.0 (OK)
cloudpickle >=0.5.0           :  2.0.0 (OK)
cookiecutter >=1.6.0          :  1.7.3 (OK)
diff_match_patch >=20181111   :  20200713 (OK)
intervaltree                  :  None (OK)
IPython >=7.6.0;<8.0.0        :  7.31.1 (OK)
jedi >=0.17.2;<0.19.0         :  0.18.1 (OK)
jellyfish >=0.7               :  0.9.0 (OK)
jsonschema >=3.2.0            :  4.4.0 (OK)
keyring >=17.0.0              :  23.5.0 (OK)
nbconvert >=4.0               :  6.4.0 (OK)
numpydoc >=0.6.0              :  1.1.0 (OK)
paramiko >=2.4.0              :  2.9.2 (OK)
parso >=0.7.0;<0.9.0          :  0.8.3 (OK)
pexpect >=4.4.0               :  4.8.0 (OK)
pickleshare >=0.4             :  0.7.5 (OK)
psutil >=5.3                  :  5.9.0 (OK)
pygments >=2.0                :  2.11.2 (OK)
pylint >=2.5.0                :  2.12.2 (OK)
pyls_spyder >=0.4.0           :  0.4.0 (OK)
pylsp >=1.3.2;<1.4.0          :  1.3.3 (OK)
pylsp_black >=1.0.0           :  1.0.1 (OK)
qdarkstyle =3.0.2             :  3.0.2 (OK)
qstylizer >=0.1.10            :  0.2.1 (OK)
qtawesome >=1.0.2             :  1.1.1 (OK)
qtconsole >=5.2.1;<5.3.0      :  5.2.2 (OK)
qtpy >=1.5.0                  :  2.0.0 (OK)
rtree >=0.9.7                 :  0.9.7 (OK)
setuptools >=49.6.0           :  60.5.0 (OK)
sphinx >=0.6.6                :  4.3.2 (OK)
spyder_kernels >=2.2.1;<2.3.0 :  2.2.1 (OK)
textdistance >=4.2.0          :  4.2.2 (OK)
three_merge >=0.1.1           :  0.1.1 (OK)
watchdog                      :  2.1.6 (OK)
zmq >=17                      :  22.3.0 (OK)

# Optional:
cython >=0.21                 :  0.29.26 (OK)
matplotlib >=2.0.0            :  3.5.1 (OK)
numpy >=1.7                   :  1.19.3 (OK)
pandas >=1.1.1                :  1.3.5 (OK)
scipy >=0.17.0                :  1.7.3 (OK)
sympy >=0.7.3                 :  1.9 (OK)

[ccordoba12]

Hey @rhkarls, thanks for reporting. The only way to solve this problem is to update our Windows installer to use Python 3.8 instead of 3.7. We'll try to do that in our next version (5.3.0), to be released in a couple of months.

Please keep using Pandas 1.3.5 in the meantime.