MacApp: Jedi Completion Not Entirely Working from Non-conda environment

[mrclary]

Issue Report Checklist

• Searched the issues page for similar reports
• Read the relevant sections of the Spyder Troubleshooting Guide and followed its advice
• Reproduced the issue after updating with conda update spyder (or pip, if not using Anaconda)
• Could not reproduce inside jupyter qtconsole (if console-related)
• Tried basic troubleshooting (if a bug/error)
  • Restarted Spyder
  • Reset preferences with spyder --reset
  • Reinstalled the latest version of Anaconda
  • Tried the other applicable steps from the Troubleshooting Guide
• Completed the Problem Description, Steps to Reproduce and Version sections below

Problem Description

Jedi command completion does not seem to work completely when Spyder is started from non-conda environments.

What steps reproduce the problem?

1. create pyenv virtual environment and install required packages
2. create a conda environment with spyder-kernels and some other package not in the pyenv environment, e.g. xarray
3. start spyder in bootstrap from the pyenv environment
4. change the python interpreter to the conda environment (if not already selected)
5. in the editor type import x and <tab>
6. observe command completion options do not include xarray

What is the expected output? What do you see instead?

command completion options should include packages in the external python interpreter

Versions

• Spyder version: 4.2.0.dev0
• Python version: 3.7.7
• Qt version: NA
• PyQt version: NA
• Operating System name/version: Mac OS 10.14.6

Dependencies

Dependencies

# Mandatory:
applaunchservices >=0.1.7      :  0.2.1 (OK)
atomicwrites >=1.2.0           :  1.3.0 (OK)
chardet >=2.0.0                :  3.0.4 (OK)
cloudpickle >=0.5.0            :  1.3.0 (OK)
diff_match_patch >=20181111    :  20181111 (OK)
intervaltree                   :  None (OK)
IPython >=4.0                  :  7.13.0 (OK)
jedi =0.15.2                   :  0.15.2 (OK)
nbconvert >=4.0                :  5.6.1 (OK)
numpydoc >=0.6.0               :  0.9.2 (OK)
parso =0.5.2                   :  0.5.2 (OK)
pexpect >=4.4.0                :  4.8.0 (OK)
pickleshare >=0.4              :  0.7.5 (OK)
psutil >=5.3                   :  5.7.0 (OK)
pygments >=2.0                 :  2.6.1 (OK)
pylint >=0.25                  :  2.4.4 (OK)
pyls >=0.31.9;<0.32.0          :  0.31.9 (OK)
qdarkstyle >=2.8               :  2.8.1 (OK)
qtawesome >=0.5.7              :  0.7.0 (OK)
qtconsole >=4.6.0              :  4.7.2 (OK)
qtpy >=1.5.0                   :  1.9.0 (OK)
sphinx >=0.6.6                 :  3.0.0 (OK)
spyder_kernels >=1.9.0;<1.10.0 :  1.10.0.dev0 (OK)
watchdog                       :  None (OK)
zmq >=17                       :  19.0.0 (OK)

# Optional:
cython >=0.21                  :  0.29.16 (OK)
matplotlib >=2.0.0             :  3.2.1 (OK)
numpy >=1.7                    :  1.18.2 (OK)
pandas >=0.13.1                :  1.0.3 (OK)
scipy >=0.17.0                 :  1.4.1 (OK)
sympy >=0.7.3                  :  1.5.1 (OK)

Conda launch environment

# packages in environment at /Users/rclary/anaconda3/envs/spy-dev:
#
# Name                    Version                   Build  Channel
alabaster                 0.7.12                   py37_0  
applaunchservices         0.2.1                      py_0    spyder-ide
appnope                   0.1.0                    py37_0  
argh                      0.26.2                   py37_0  
asn1crypto                1.3.0                    py37_0  
astroid                   2.3.3                    py37_0  
atomicwrites              1.3.0                    py37_1  
attrs                     19.3.0                     py_0  
autopep8                  1.4.4                      py_0  
babel                     2.8.0                      py_0  
backcall                  0.1.0                    py37_0  
bcrypt                    3.1.7            py37h1de35cc_0  
blas                      1.0                         mkl  
bleach                    3.1.0                    py37_0  
ca-certificates           2020.1.1                      0  
certifi                   2019.11.28               py37_1  
cffi                      1.14.0           py37hb5b8e2f_0  
chardet                   3.0.4                 py37_1003  
cloudpickle               1.3.0                      py_0  
coverage                  4.5.4            py37h1de35cc_0  
cryptography              2.8              py37ha12b0ac_0  
cycler                    0.10.0                   py37_0  
cython                    0.29.15          py37h0a44026_0  
dbus                      1.13.12              h90a0687_0  
decorator                 4.4.2                      py_0  
defusedxml                0.6.0                      py_0  
diff-match-patch          20181111                   py_0  
docutils                  0.16                     py37_0  
entrypoints               0.3                      py37_0  
expat                     2.2.6                h0a44026_0  
fastcache                 1.1.0            py37h1de35cc_0  
flake8                    3.7.9                    py37_0  
flaky                     3.6.1                      py_0  
freetype                  2.9.1                hb4e5f40_0  
future                    0.18.2                   py37_0  
gettext                   0.19.8.1             h15daf44_3  
glib                      2.63.1               hd977a24_0  
gmp                       6.1.2                hb37e062_1  
gmpy2                     2.0.8            py37h6ef4df4_2  
icu                       58.2                 h4b95b61_1  
idna                      2.9                        py_1  
imagesize                 1.2.0                      py_0  
importlib_metadata        1.5.0                    py37_0  
intel-openmp              2019.4                      233  
intervaltree              3.0.2                      py_0  
ipykernel                 5.1.4            py37h39e3cac_0  
ipython                   7.13.0           py37h5ca1d4c_0  
ipython_genutils          0.2.0                    py37_0  
isort                     4.3.21                   py37_0  
jedi                      0.15.2                   py37_0  
jinja2                    2.11.1                     py_0  
jpeg                      9b                   he5867d9_2  
jsonschema                3.2.0                    py37_0  
jupyter_client            6.1.2                      py_0  
jupyter_core              4.6.3                    py37_0  
keyring                   21.1.1                   py37_2  
kiwisolver                1.1.0            py37h0a44026_0  
lazy-object-proxy         1.4.3            py37h1de35cc_0  
libcxx                    4.0.1                hcfea43d_1  
libcxxabi                 4.0.1                hcfea43d_1  
libedit                   3.1.20181209         hb402a30_0  
libffi                    3.2.1                h475c297_4  
libgfortran               3.0.1                h93005f0_2  
libiconv                  1.15                 hdd342a3_7  
libpng                    1.6.37               ha441bb4_0  
libsodium                 1.0.16               h3efe00b_0  
libspatialindex           1.9.3                h0a44026_0  
libtiff                   4.1.0                hcb84e12_0  
markupsafe                1.1.1            py37h1de35cc_0  
matplotlib                3.1.3                    py37_0  
matplotlib-base           3.1.3            py37h9aa3819_0  
mccabe                    0.6.1                    py37_1  
mistune                   0.8.4            py37h1de35cc_0  
mkl                       2019.4                      233  
mkl-service               2.3.0            py37hfbe908c_0  
mkl_fft                   1.0.15           py37h5e564d8_0  
mkl_random                1.1.0            py37ha771720_0  
mock                      4.0.1                      py_0  
more-itertools            8.2.0                      py_0  
mpc                       1.1.0                h6ef4df4_1  
mpfr                      4.0.1                h3018a27_3  
mpmath                    1.1.0                    py37_0  
nbconvert                 5.6.1                    py37_0  
nbformat                  5.0.4                      py_0  
ncurses                   6.2                  h0a44026_0  
numpy                     1.18.1           py37h7241aed_0  
numpy-base                1.18.1           py37h6575580_1  
numpydoc                  0.9.2                      py_0  
olefile                   0.46                     py37_0  
openssl                   1.1.1f               h1de35cc_0  
packaging                 20.3                       py_0  
pandas                    1.0.3            py37h6c726b0_0  
pandoc                    2.2.3.2                       0  
pandocfilters             1.4.2                    py37_1  
paramiko                  2.7.1                      py_0  
parso                     0.5.2                      py_0  
pathtools                 0.1.2                      py_1  
pcre                      8.43                 h0a44026_0  
pexpect                   4.8.0                    py37_0  
pickleshare               0.7.5                    py37_0  
pillow                    7.0.0            py37h4655f20_0  
pip                       20.0.2                   py37_1  
pluggy                    0.13.1                   py37_0  
prompt-toolkit            3.0.4                      py_0  
prompt_toolkit            3.0.4                         0  
psutil                    5.7.0            py37h1de35cc_0  
ptyprocess                0.6.0                    py37_0  
py                        1.8.1                      py_0  
pycodestyle               2.5.0                    py37_0  
pycparser                 2.20                       py_0  
pydocstyle                4.0.1                      py_0  
pyflakes                  2.1.1                    py37_0  
pygments                  2.6.1                      py_0  
pylint                    2.4.4                    py37_0  
pynacl                    1.3.0            py37h1de35cc_0  
pyopenssl                 19.1.0                   py37_0  
pyparsing                 2.4.6                      py_0  
pyqt                      5.9.2            py37h655552a_2  
pyrsistent                0.16.0           py37h1de35cc_0  
pysocks                   1.7.1                    py37_0  
pytest                    4.6.2                    py37_0  
pytest-cov                2.8.1                      py_0  
pytest-faulthandler       1.6.0                    py37_0    spyder-ide
pytest-lazy-fixture       0.5.2                      py_0    spyder-ide
pytest-mock               2.0.0                      py_0  
pytest-ordering           0.6                        py_0    spyder-ide
pytest-qt                 3.2.1                      py_0    spyder-ide
python                    3.7.7           hc70fcce_0_cpython  
python-dateutil           2.8.1                      py_0  
python-jsonrpc-server     0.3.4                      py_0  
python-language-server    0.31.9                   py37_0  
pytz                      2019.3                     py_0  
pyxdg                     0.26                       py_0  
pyyaml                    5.3.1            py37h1de35cc_0  
pyzmq                     18.1.1           py37h0a44026_0  
qdarkstyle                2.8                        py_0  
qt                        5.9.7                h468cd18_1  
qtawesome                 0.7.0                      py_0  
qtconsole                 4.7.2                      py_0  
qtpy                      1.9.0                      py_0  
readline                  8.0                  h1de35cc_0  
requests                  2.23.0                   py37_0  
rope                      0.16.0                     py_0  
rtree                     0.9.3                    py37_0  
scipy                     1.4.1            py37h9fa6033_0  
setuptools                46.1.3                   py37_0  
sip                       4.19.8           py37h0a44026_0  
six                       1.14.0                   py37_0  
snowballstemmer           2.0.0                      py_0  
sortedcontainers          2.1.0                    py37_0  
sphinx                    2.4.4                      py_0  
sphinxcontrib-applehelp   1.0.2                      py_0  
sphinxcontrib-devhelp     1.0.2                      py_0  
sphinxcontrib-htmlhelp    1.0.3                      py_0  
sphinxcontrib-jsmath      1.0.1                      py_0  
sphinxcontrib-qthelp      1.0.3                      py_0  
sphinxcontrib-serializinghtml 1.1.4                      py_0  
spyder-kernels            1.8.0                    py37_0    spyder-ide
sqlite                    3.31.1               ha441bb4_0  
sympy                     1.5.1                    py37_0  
testpath                  0.4.4                      py_0  
tk                        8.6.8                ha441bb4_0  
tornado                   6.0.4            py37h1de35cc_1  
traitlets                 4.3.3                    py37_0  
ujson                     1.35             py37h1de35cc_0  
urllib3                   1.25.8                   py37_0  
watchdog                  0.10.2           py37h1de35cc_0  
wcwidth                   0.1.9                      py_0  
webencodings              0.5.1                    py37_1  
wheel                     0.34.2                   py37_0  
wrapt                     1.12.1           py37h1de35cc_1  
wurlitzer                 2.0.0                    py37_0  
xz                        5.2.4                h1de35cc_4  
yaml                      0.1.7                hc338f04_2  
yapf                      0.28.0                     py_0  
zeromq                    4.3.1                h0a44026_3  
zipp                      2.2.0                      py_0  
zlib                      1.2.11               h1de35cc_3  
zstd                      1.3.7                h5bba6e5_0  

Pyenv launch environment

Package                                 Version   
--------------------------------------- ----------
alabaster                               0.7.12    
altgraph                                0.17      
applaunchservices                       0.2.1     
appnope                                 0.1.0     
astroid                                 2.3.3     
atomicwrites                            1.3.0     
attrs                                   19.3.0    
autopep8                                1.5.1     
Babel                                   2.8.0     
backcall                                0.1.0     
bcrypt                                  3.1.7     
biplist                                 1.0.3     
bleach                                  3.1.4     
certifi                                 2020.4.5.1
cffi                                    1.14.0    
chardet                                 3.0.4     
cloudpickle                             1.3.0     
cryptography                            2.9       
cycler                                  0.10.0    
Cython                                  0.29.16   
decorator                               4.4.2     
defusedxml                              0.6.0     
diff-match-patch                        20181111  
dmgbuild                                1.3.3     
docutils                                0.16      
ds-store                                1.1.2     
entrypoints                             0.3       
flake8                                  3.7.9     
helpdev                                 0.6.10    
idna                                    2.9       
imagesize                               1.2.0     
importlib-metadata                      1.6.0     
intervaltree                            3.0.2     
ipykernel                               5.2.0     
ipython                                 7.13.0    
ipython-genutils                        0.2.0     
isort                                   4.3.21    
jedi                                    0.15.2    
Jinja2                                  2.11.1    
jsonschema                              3.2.0     
jupyter-client                          6.1.2     
jupyter-core                            4.6.3     
keyring                                 21.2.0    
kiwisolver                              1.2.0     
lazy-object-proxy                       1.4.3     
mac-alias                               2.0.7     
macholib                                1.14      
MarkupSafe                              1.1.1     
matplotlib                              3.2.1     
mccabe                                  0.6.1     
mistune                                 0.8.4     
modulegraph                             0.18      
mpmath                                  1.1.0     
nbconvert                               5.6.1     
nbformat                                5.0.5     
numpy                                   1.18.2    
numpydoc                                0.9.2     
packaging                               20.3      
pandas                                  1.0.3     
pandocfilters                           1.4.2     
paramiko                                2.7.1     
parso                                   0.5.2     
pathtools                               0.1.2     
pexpect                                 4.8.0     
pickleshare                             0.7.5     
pip                                     19.2.3    
pluggy                                  0.13.1    
prompt-toolkit                          3.0.5     
psutil                                  5.7.0     
ptyprocess                              0.6.0     
py2app                                  0.21      
pycodestyle                             2.5.0     
pycparser                               2.20      
pydocstyle                              5.0.2     
pyflakes                                2.1.1     
Pygments                                2.6.1     
pylint                                  2.4.4     
PyNaCl                                  1.3.0     
pyobjc                                  6.2       
pyobjc-core                             6.2       
pyobjc-framework-Accounts               6.2       
pyobjc-framework-AddressBook            6.2       
pyobjc-framework-AdSupport              6.2       
pyobjc-framework-AppleScriptKit         6.2       
pyobjc-framework-AppleScriptObjC        6.2       
pyobjc-framework-ApplicationServices    6.2       
pyobjc-framework-Automator              6.2       
pyobjc-framework-AVFoundation           6.2       
pyobjc-framework-AVKit                  6.2       
pyobjc-framework-BusinessChat           6.2       
pyobjc-framework-CalendarStore          6.2       
pyobjc-framework-CFNetwork              6.2       
pyobjc-framework-CloudKit               6.2       
pyobjc-framework-Cocoa                  6.2       
pyobjc-framework-Collaboration          6.2       
pyobjc-framework-ColorSync              6.2       
pyobjc-framework-Contacts               6.2       
pyobjc-framework-ContactsUI             6.2       
pyobjc-framework-CoreAudio              6.2       
pyobjc-framework-CoreAudioKit           6.2       
pyobjc-framework-CoreBluetooth          6.2       
pyobjc-framework-CoreData               6.2       
pyobjc-framework-CoreLocation           6.2       
pyobjc-framework-CoreMedia              6.2       
pyobjc-framework-CoreMediaIO            6.2       
pyobjc-framework-CoreML                 6.2       
pyobjc-framework-CoreServices           6.2       
pyobjc-framework-CoreSpotlight          6.2       
pyobjc-framework-CoreText               6.2       
pyobjc-framework-CoreWLAN               6.2       
pyobjc-framework-CryptoTokenKit         6.2       
pyobjc-framework-DictionaryServices     6.2       
pyobjc-framework-DiscRecording          6.2       
pyobjc-framework-DiscRecordingUI        6.2       
pyobjc-framework-DiskArbitration        6.2       
pyobjc-framework-DVDPlayback            6.2       
pyobjc-framework-EventKit               6.2       
pyobjc-framework-ExceptionHandling      6.2       
pyobjc-framework-ExternalAccessory      6.2       
pyobjc-framework-FinderSync             6.2       
pyobjc-framework-FSEvents               6.2       
pyobjc-framework-GameCenter             6.2       
pyobjc-framework-GameController         6.2       
pyobjc-framework-GameKit                6.2       
pyobjc-framework-GameplayKit            6.2       
pyobjc-framework-ImageCaptureCore       6.2       
pyobjc-framework-IMServicePlugIn        6.2       
pyobjc-framework-InputMethodKit         6.2       
pyobjc-framework-InstallerPlugins       6.2       
pyobjc-framework-InstantMessage         6.2       
pyobjc-framework-Intents                6.2       
pyobjc-framework-IOSurface              6.2       
pyobjc-framework-iTunesLibrary          6.2       
pyobjc-framework-LatentSemanticMapping  6.2       
pyobjc-framework-LaunchServices         6.2       
pyobjc-framework-libdispatch            6.2       
pyobjc-framework-LocalAuthentication    6.2       
pyobjc-framework-MapKit                 6.2       
pyobjc-framework-MediaAccessibility     6.2       
pyobjc-framework-MediaLibrary           6.2       
pyobjc-framework-MediaPlayer            6.2       
pyobjc-framework-MediaToolbox           6.2       
pyobjc-framework-Metal                  6.2       
pyobjc-framework-MetalKit               6.2       
pyobjc-framework-ModelIO                6.2       
pyobjc-framework-MultipeerConnectivity  6.2       
pyobjc-framework-NaturalLanguage        6.2       
pyobjc-framework-NetFS                  6.2       
pyobjc-framework-Network                6.2       
pyobjc-framework-NetworkExtension       6.2       
pyobjc-framework-NotificationCenter     6.2       
pyobjc-framework-OpenDirectory          6.2       
pyobjc-framework-OSAKit                 6.2       
pyobjc-framework-Photos                 6.2       
pyobjc-framework-PhotosUI               6.2       
pyobjc-framework-PreferencePanes        6.2       
pyobjc-framework-PubSub                 6.2       
pyobjc-framework-QTKit                  6.2       
pyobjc-framework-Quartz                 6.2       
pyobjc-framework-SafariServices         6.2       
pyobjc-framework-SceneKit               6.2       
pyobjc-framework-ScreenSaver            6.2       
pyobjc-framework-ScriptingBridge        6.2       
pyobjc-framework-SearchKit              6.2       
pyobjc-framework-Security               6.2       
pyobjc-framework-SecurityFoundation     6.2       
pyobjc-framework-SecurityInterface      6.2       
pyobjc-framework-ServiceManagement      6.2       
pyobjc-framework-Social                 6.2       
pyobjc-framework-SpriteKit              6.2       
pyobjc-framework-StoreKit               6.2       
pyobjc-framework-SyncServices           6.2       
pyobjc-framework-SystemConfiguration    6.2       
pyobjc-framework-UserNotifications      6.2       
pyobjc-framework-VideoSubscriberAccount 6.2       
pyobjc-framework-VideoToolbox           6.2       
pyobjc-framework-Vision                 6.2       
pyobjc-framework-WebKit                 6.2       
pyparsing                               2.4.7     
PyQt5                                   5.12.3    
PyQt5-sip                               12.7.2    
PyQtWebEngine                           5.12.1    
pyrsistent                              0.16.0    
python-dateutil                         2.8.1     
python-jsonrpc-server                   0.3.4     
python-language-server                  0.31.9    
pytz                                    2019.3    
pyxdg                                   0.26      
pyzmq                                   19.0.0    
QDarkStyle                              2.8.1     
QtAwesome                               0.7.0     
qtconsole                               4.7.2     
QtPy                                    1.9.0     
requests                                2.23.0    
rope                                    0.16.0    
scipy                                   1.4.1     
setuptools                              41.2.0    
six                                     1.14.0    
snowballstemmer                         2.0.0     
sortedcontainers                        2.1.0     
Sphinx                                  3.0.0     
sphinxcontrib-applehelp                 1.0.2     
sphinxcontrib-devhelp                   1.0.2     
sphinxcontrib-htmlhelp                  1.0.3     
sphinxcontrib-jsmath                    1.0.1     
sphinxcontrib-qthelp                    1.0.3     
sphinxcontrib-serializinghtml           1.1.4     
sympy                                   1.5.1     
testpath                                0.4.4     
tornado                                 6.0.4     
traitlets                               4.3.3     
typed-ast                               1.4.1     
ujson                                   1.35      
urllib3                                 1.25.8    
watchdog                                0.10.2    
wcwidth                                 0.1.9     
webencodings                            0.5.1     
wrapt                                   1.11.2    
yapf                                    0.29.0    
zipp                                    3.1.0

External Conda Environment

# packages in environment at /Users/rclary/anaconda3/envs/c2w_37_spy:
#
# Name                    Version                   Build  Channel
apipkg                    1.5                      py37_0  
appnope                   0.1.0                    py37_0  
asn1crypto                1.3.0                    py37_0  
asteval                   0.9.18                   pypi_0    pypi
astroid                   2.3.3                    py37_0  
atomicwrites              1.3.0                    py37_1  
attrs                     19.3.0                     py_0  
autopep8                  1.4.4                      py_0  
backcall                  0.1.0                    py37_0  
bcrypt                    3.1.7            py37h1de35cc_0  
blas                      1.0                         mkl  
ca-certificates           2020.1.1                      0  
certifi                   2019.11.28               py37_1  
cffi                      1.14.0           py37hb5b8e2f_0  
chardet                   3.0.4                 py37_1003  
cloudpickle               1.3.0                      py_0  
cryptography              2.8              py37ha12b0ac_0  
cycler                    0.10.0                   py37_0  
dbus                      1.13.12              h90a0687_0  
decorator                 4.4.1                      py_0  
diskcache                 4.1.0                    pypi_0    pypi
execnet                   1.7.1                      py_0  
expat                     2.2.6                h0a44026_0  
freetds                   1.00.97              ha12b0ac_0  
freetype                  2.9.1                hb4e5f40_0  
future                    0.18.2                   py37_0  
gettext                   0.19.8.1             h15daf44_3  
gitdb2                    2.0.6                      py_0  
gitpython                 3.0.5                      py_0  
glib                      2.63.1               hd977a24_0  
h5py                      2.9.0            py37h3134771_0  
hdf5                      1.10.4               hfa1e0ec_0  
icu                       58.2                 h4b95b61_1  
idna                      2.8                      py37_0  
importlib_metadata        1.5.0                    py37_0  
influxdb                  5.2.3                    pypi_0    pypi
intel-openmp              2019.4                      233  
ipykernel                 5.1.4            py37h39e3cac_0  
ipython                   7.12.0           py37h5ca1d4c_0  
ipython_genutils          0.2.0                    py37_0  
isort                     4.3.21                   py37_0  
jedi                      0.16.0                   py37_0  
joblib                    0.14.1                     py_0  
jpeg                      9b                   he5867d9_2  
jupyter_client            5.3.4                    py37_0  
jupyter_core              4.6.1                    py37_0  
kiwisolver                1.1.0            py37h0a44026_0  
krb5                      1.16.4               hddcf347_0  
lazy-object-proxy         1.4.3            py37h1de35cc_0  
libcxx                    4.0.1                hcfea43d_1  
libcxxabi                 4.0.1                hcfea43d_1  
libedit                   3.1.20181209         hb402a30_0  
libffi                    3.2.1                h475c297_4  
libgfortran               3.0.1                h93005f0_2  
libiconv                  1.15                 hdd342a3_7  
libpng                    1.6.37               ha441bb4_0  
libpq                     11.2                 h051b688_0  
libsodium                 1.0.16               h3efe00b_0  
llvm-openmp               4.0.1                hcfea43d_1  
llvmlite                  0.29.0           py37h98b8051_0  
lmfit                     1.0.0                    pypi_0    pypi
matplotlib                3.1.0            py37h54f8f79_0  
mccabe                    0.6.1                    py37_1  
mkl                       2019.4                      233  
mkl-service               2.3.0            py37hfbe908c_0  
mkl_fft                   1.0.15           py37h5e564d8_0  
mkl_random                1.1.0            py37ha771720_0  
more-itertools            8.2.0                      py_0  
ncurses                   6.2                  h0a44026_0  
numba                     0.45.1           py37h6440ff4_0  
numpy                     1.16.4           py37hacdab7b_0  
numpy-base                1.16.4           py37h6575580_0  
numpy-quaternion          2019.12.11.22.25.52          pypi_0    pypi
opencv-python             4.2.0.32                 pypi_0    pypi
openssl                   1.1.1f               h1de35cc_0  
packaging                 20.1                       py_0  
pandas                    0.25.1           py37h0a44026_0  
paramiko                  2.7.1                      py_0  
parso                     0.6.1                      py_0  
pcre                      8.43                 h0a44026_0  
pexpect                   4.8.0                    py37_0  
pickleshare               0.7.5                    py37_0  
pint                      0.9                      pypi_0    pypi
pip                       20.0.2                   py37_1  
pluggy                    0.13.1                   py37_0  
prompt_toolkit            3.0.3                      py_0  
psutil                    5.6.7            py37h1de35cc_0  
psycopg2                  2.7.6.1          py37ha12b0ac_0  
ptyprocess                0.6.0                    py37_0  
py                        1.8.1                      py_0  
pycodestyle               2.5.0                    py37_0  
pycparser                 2.19                     py37_0  
pydocstyle                4.0.1                      py_0  
pyfftw                    0.11.1                   pypi_0    pypi
pyflakes                  2.1.1                    py37_0  
pygments                  2.5.2                      py_0  
pylint                    2.4.4                    py37_0  
pymssql                   2.1.4            py37h1de35cc_0  
pymysql                   0.9.3                    py37_0  
pynacl                    1.3.0            py37h1de35cc_0  
pyopenssl                 19.1.0                   py37_0  
pyparsing                 2.4.6                      py_0  
pyqt                      5.9.2            py37h655552a_2  
pysocks                   1.7.1                    py37_0  
pytest                    5.0.1                    py37_0  
pytest-forked             1.1.3                      py_0  
pytest-xdist              1.28.0                     py_0  
python                    3.7.6                h359304d_2  
python-dateutil           2.8.1                      py_0  
python-jsonrpc-server     0.3.4                      py_0  
python-language-server    0.26.1                   py37_0    conda-forge
pytz                      2019.3                     py_0  
pyzmq                     18.1.1           py37h0a44026_0  
qt                        5.9.7                h468cd18_1  
readline                  7.0                  h1de35cc_5  
requests                  2.22.0                   py37_1  
rope                      0.16.0                     py_0  
scikit-learn              0.21.2           py37h27c97d8_0  
scikit-tensor-py3         0.4.1                    pypi_0    pypi
scipy                     1.3.1            py37h1410ff5_0  
setuptools                45.2.0                   py37_0  
sip                       4.19.8           py37h0a44026_0  
six                       1.14.0                   py37_0  
smmap2                    2.0.5                    py37_0  
snowballstemmer           2.0.0                      py_0  
spyder-kernels            1.9.0                    py37_0  
sqlite                    3.31.1               ha441bb4_0  
tbb                       2020.0               h04f5b5a_0  
termcolor                 1.1.0                    py37_1  
tk                        8.6.8                ha441bb4_0  
tornado                   6.0.3            py37h1de35cc_3  
tqdm                      4.42.1                     py_0  
traitlets                 4.3.3                    py37_0  
ujson                     1.35             py37h1de35cc_0  
uncertainties             3.1.2                    pypi_0    pypi
unixodbc                  2.3.7                h1de35cc_0  
urllib3                   1.25.8                   py37_0  
wcwidth                   0.1.8                      py_0  
wheel                     0.34.2                   py37_0  
wrapt                     1.11.2           py37h1de35cc_0  
wurlitzer                 2.0.0                    py37_0  
xarray                    0.12.1                     py_0  
xz                        5.2.4                h1de35cc_4  
yapf                      0.28.0                     py_0  
zeromq                    4.3.1                h0a44026_3  
zipp                      2.2.0                      py_0  
zlib                      1.2.11               h1de35cc_3

[mrclary]

This issue does not manifest when launching spyder (bootstrap) from the conda launch environment (listed above).

While Spyder does not officially support non-conda environments, this is relevant for the stand-alone Mac application.

I've attached two language server logs, one from the successful operation and one from the failed operation. Of particular note is near the end of the logs, parso.cache loads relevant modules from both the Spyder environment and the external conda environment. This is true for both the pyenv and conda launch cases. But in the pyenv launch case, xarray and xdist are not loaded. I'm not sure why this is the case.

server_python_conda.log
server_python_pyenv.log

Pyenv launch case:

2020-04-09 18:23:26,344 UTC - DEBUG - parso.cache - pickle loaded: /Users/rclary/.pyenv/versions/3.7.7/envs/spy377/lib/python3.7/site-packages/xdg/__init__.py
2020-04-09 18:23:26,383 UTC - DEBUG - parso.cache - pickle loaded: /Users/rclary/anaconda3/envs/c2w_37_spy/lib/python3.7/xdrlib.py
2020-04-09 18:23:26,387 UTC - DEBUG - parso.cache - pickle loaded: /Users/rclary/.pyenv/versions/3.7.7/envs/spy377/lib/python3.7/site-packages/jedi/third_party/typeshed/stdlib/2and3/xdrlib.pyi
2020-04-09 18:23:26,390 UTC - DEBUG - parso.cache - pickle loaded: /Users/rclary/anaconda3/envs/c2w_37_spy/lib/python3.7/xml/__init__.py
2020-04-09 18:23:26,392 UTC - DEBUG - parso.cache - pickle loaded: /Users/rclary/.pyenv/versions/3.7.7/envs/spy377/lib/python3.7/site-packages/jedi/third_party/typeshed/stdlib/2and3/xml/__init__.pyi
2020-04-09 18:23:26,395 UTC - DEBUG - parso.cache - pickle loaded: /Users/rclary/anaconda3/envs/c2w_37_spy/lib/python3.7/xmlrpc/__init__.py
2020-04-09 18:23:26,402 UTC - DEBUG - pyls.config.config -   finish pyls_completions --> [[{'label': 'xdg', 'kind': 9, 'detail': 'xdg', 'documentation': '', 'sortText': 'axdg', 'insertText': 'xdg'}, {'label': 'xdrlib', 'kind': 9, 'detail': 'xdrlib', 'documentation': '', 'sortText': 'axdrlib', 'insertText': 'xdrlib'}, {'label': 'xml', 'kind': 9, 'detail': 'xml', 'documentation': '', 'sortText': 'axml', 'insertText': 'xml'}, {'label': 'xmlrpc', 'kind': 9, 'detail': 'xmlrpc', 'documentation': '', 'sortText': 'axmlrpc', 'insertText': 'xmlrpc'}, {'label': 'xxlimited', 'kind': 9, 'detail': 'xxlimited', 'documentation': '', 'sortText': 'axxlimited', 'insertText': 'xxlimited'}, {'label': 'xxsubtype', 'kind': 9, 'detail': 'xxsubtype', 'documentation': '', 'sortText': 'axxsubtype', 'insertText': 'xxsubtype'}]] [hook]

Conda launch case:

2020-04-09 17:51:40,188 UTC - DEBUG - parso.cache - pickle loaded: /Users/rclary/anaconda3/envs/c2w_37_spy/lib/python3.7/site-packages/xarray/__init__.py
2020-04-09 17:51:40,207 UTC - DEBUG - parso.cache - pickle loaded: /Users/rclary/anaconda3/envs/c2w_37_spy/lib/python3.7/site-packages/xdist/__init__.py
2020-04-09 17:51:40,212 UTC - DEBUG - parso.cache - pickle loaded: /Users/rclary/anaconda3/envs/c2w_37_spy/lib/python3.7/xdrlib.py
2020-04-09 17:51:40,214 UTC - DEBUG - parso.cache - pickle loaded: /Users/rclary/anaconda3/envs/spy-dev/lib/python3.7/site-packages/jedi/third_party/typeshed/stdlib/2and3/xdrlib.pyi
2020-04-09 17:51:40,216 UTC - DEBUG - parso.cache - pickle loaded: /Users/rclary/anaconda3/envs/c2w_37_spy/lib/python3.7/xml/__init__.py
2020-04-09 17:51:40,218 UTC - DEBUG - parso.cache - pickle loaded: /Users/rclary/anaconda3/envs/spy-dev/lib/python3.7/site-packages/jedi/third_party/typeshed/stdlib/2and3/xml/__init__.pyi
2020-04-09 17:51:40,219 UTC - DEBUG - parso.cache - pickle loaded: /Users/rclary/anaconda3/envs/c2w_37_spy/lib/python3.7/xmlrpc/__init__.py
2020-04-09 17:51:40,223 UTC - DEBUG - pyls.config.config -   finish pyls_completions --> [[{'label': 'xarray', 'kind': 9, 'detail': 'xarray', 'documentation': '', 'sortText': 'axarray', 'insertText': 'xarray'}, {'label': 'xdist', 'kind': 9, 'detail': 'xdist', 'documentation': '', 'sortText': 'axdist', 'insertText': 'xdist'}, {'label': 'xdrlib', 'kind': 9, 'detail': 'xdrlib', 'documentation': '', 'sortText': 'axdrlib', 'insertText': 'xdrlib'}, {'label': 'xml', 'kind': 9, 'detail': 'xml', 'documentation': '', 'sortText': 'axml', 'insertText': 'xml'}, {'label': 'xmlrpc', 'kind': 9, 'detail': 'xmlrpc', 'documentation': '', 'sortText': 'axmlrpc', 'insertText': 'xmlrpc'}, {'label': 'xxlimited', 'kind': 9, 'detail': 'xxlimited', 'documentation': '', 'sortText': 'axxlimited', 'insertText': 'xxlimited'}, {'label': 'xxsubtype', 'kind': 9, 'detail': 'xxsubtype', 'documentation': '', 'sortText': 'axxsubtype', 'insertText': 'xxsubtype'}]] [hook]

Here you can see that command completions for xml, xmlrpc, and xdrlib show up in both launch cases, and come from the external conda envrionment (c2w_37_spy). The conda launch case is the only one with xarray and xdist, but the pyenv launch case has the unique xdg offering. xmlrpc is exclusive to the external environment in both cases, so I know that jedi is engaging the external environment in the pyenv case. The two questions are:

1. Why should anything be picked up from the launch environment when jedi's environment is explicitly set to the external environment?
2. Why would some things be picked up in the external environment (xmlrpc) but not others (xarray, xdist)?

[mrclary]

Okay, I think this may be a bug with jedi. I submitted an issue: davidhalter/jedi#1540.
As a work-around, I think we can just add the required site-package path to the extra-paths configuration value.

[mrclary]

FYI: This issue may be resolved with a pending update to jedi (>0.17.0). See davidhalter/jedi#1546.
Not sure what this implies for #12271...

[ccordoba12]

FYI: This issue may be resolved with a pending update to jedi (>0.17.0)

Great to know!

Not sure what this implies for #12271...

We can use that PR until we're able to support the Jedi versions that come with that fix in the Python language server.

[mrclary]

Let's leave this issue open after merging #12271 in order to track the adoption of jedi>0.17.0. Many of the changes in #12271 can be reverted at that time. I will also create an issue for python-language-server.

[mrclary]

issue already exists: palantir/python-language-server#744

[mrclary]

There is yet another bug when the target environment python libraries are not in a standard location. See davidhalter/jedi#1617, davidhalter/jedi#1619, and palantir/python-language-server#822.
I've got the fix and I'll submit PR after PR is merged in pyls.

[ccordoba12]

Thanks @mrclary! Let's hope Jedi's 0.17.2 is released soon so we can also release a new version of the PyLS with your fix.

[ccordoba12]

@mrclary, just by supporting Jedi 0.17.2 this bug is solved?

[mrclary]

@mrclary, just by supporting Jedi 0.17.2 this bug is solved?

Along with palantir/python-language-server#822 (which you just merged), yes, adoption of Jedi 0.17.2 is all that is required.

[ccordoba12]

Great! Thanks for letting me know about it.

[ccordoba12]

This was fixed by #13839.