The ouline explorer doesn't populate for some files with latest 4.x #13928

jnsebgosselin · 2020-10-05T21:03:07Z

Problem Description

The outline explorer just doesn't want to populate at all for some files on my system. Moreover, the code style and error analysis do not seem to work for these files too.

Here are some lsp logs that I have created by trying to open a very simple file for which the outline explorer doesn't work.

transport_python_1716.log
server_python_1716.log

The content of the file was simply:

# -*- coding: utf-8 -*-


def test_outline_explorer():
    a = 3
    c = b
    return a

Versions

Spyder version: 4.2.0.dev0 6e06663
Python version: 3.7.6 64-bit
Qt version: 5.12.5
PyQt5 version: 5.12.3
Operating System: Windows 10

Dependencies


# Mandatory:
atomicwrites >=1.2.0           :  1.4.0 (OK)
chardet >=2.0.0                :  3.0.4 (OK)
cloudpickle >=0.5.0            :  1.5.0 (OK)
diff_match_patch >=20181111    :  20200713 (OK)
intervaltree                   :  None (OK)
IPython >=4.0                  :  7.16.1 (OK)
jedi =0.17.2                   :  0.17.2 (OK)
nbconvert >=4.0                :  5.6.1 (OK)
numpydoc >=0.6.0               :  1.1.0 (OK)
paramiko >=2.4.0               :  2.7.1 (OK)
parso =0.7.0                   :  0.7.0 (OK)
pexpect >=4.4.0                :  4.8.0 (OK)
pickleshare >=0.4              :  0.7.5 (OK)
psutil >=5.3                   :  5.7.2 (OK)
pygments >=2.0                 :  2.6.1 (OK)
pylint >=1.0                   :  2.5.3 (OK)
pyls >=0.35.0;<1.0.0           :  0.35.1+1.gd81c7ba (OK)
qdarkstyle >=2.8               :  2.8.1 (OK)
qtawesome >=0.5.7              :  0.8.0.dev0 (OK)
qtconsole >=4.7.7              :  4.8.0.dev0 (OK)
qtpy >=1.5.0                   :  1.9.0 (OK)
rtree >=0.8.3                  :  0.9.4 (OK)
sphinx >=0.6.6                 :  3.1.2 (OK)
spyder_kernels >=1.9.4;<1.10.0 :  1.10.0.dev0 (OK)
watchdog                       :  None (OK)
zmq >=17                       :  19.0.1 (OK)

# Optional:
cython >=0.21                  :  0.29.21 (OK)
matplotlib >=2.0.0             :  3.2.2 (OK)
numpy >=1.7                    :  1.19.1 (OK)
pandas >=1.1.1                 :  1.0.5 (NOK)
scipy >=0.17.0                 :  1.5.0 (OK)
sympy >=0.7.3                  :  None (OK)

The text was updated successfully, but these errors were encountered:

jnsebgosselin · 2020-10-05T21:05:04Z

Opening the same file at commit 66e98a5, right before PR #13109 was merged, is working flawlessly.

steff456 · 2020-10-05T22:28:47Z

@jnsebgosselin can you please check if the error persists if you update the PyLS repo after this PR palantir/python-language-server#866 ?

jnsebgosselin · 2020-10-05T22:38:35Z

Thanks for the followup @steff456.

The issue is still happening with latest python-language-server on the develop branch.

I don't know if this is related, but there is an error that is reported in the log files.

2020-10-05 18:35:19,363 UTC - ERROR - pyls_jsonrpc.endpoint - Failed to handle notification textDocument/didOpen: {'textDocument': {'uri': 'file:///D:/test_outline_explorer.py', 'languageId': 'Python', 'version': 1, 'text': '# -*- coding: utf-8 -*-\n\n\ndef test_outline_explorer():\n    a = 3\n    c = b\n    return a\n'}}
Traceback (most recent call last):
  File "C:\Anaconda3\envs\python37\lib\site-packages\pyls_jsonrpc\endpoint.py", line 142, in _handle_notification
    handler_result = handler(params)
  File "C:\Anaconda3\envs\python37\lib\site-packages\pyls_jsonrpc\dispatchers.py", line 23, in handler
    return method(**(params or {}))
  File "C:\Users\User\python-language-server\pyls\python_ls.py", line 298, in m_text_document__did_open
    workspace.put_document(textDocument['uri'], textDocument['text'], version=textDocument.get('version'))
AttributeError: 'NoneType' object has no attribute 'put_document'

This error doesn't occur if I go before PR #13109 was merged on the 4.x branch.

steff456 · 2020-10-05T22:40:14Z

What version of json-rpc do you have in the environment?

jnsebgosselin · 2020-10-05T22:55:43Z

It is version 0.4.0.

ccordoba12 · 2020-10-06T04:33:14Z

Thanks a lot @jnsebgosselin for your help with tracking this bug!

I don't know if this is related, but there is an error that is reported in the log files.

I think this is related to an error while initializing the PyLS because (according to your traceback) the document is trying to be assigned to a non-existing workspace.

Could you try with PR #13828? That's ready for review and it should solve those issues.

jnsebgosselin · 2020-10-06T11:32:20Z

Thanks @ccordoba12 for your help with this. I tried with PR #13828 and this doesn't solve the problem with the outline explorer not working for some files, but this solve the broken code analysis for those files and there isn't any traceback in the log files anymore...

What is very strange is that this seems to be related to the location of the files. I have folder that if I put any file in it, then Spyder is unable to build the outline explorer tree. If I take these files out and put them in another location, then everything seems to be working fine... this is really strange.

jnsebgosselin · 2020-10-06T12:05:44Z

If I put the file test_outline_explorer.py in C:\Users\User\gwhat\gwhat\meteo it is working, but if I put it in C:\Users\User\gwhat\gwhat\meteo\tests it is not...

I will continue to investigate and report back if I find anything useful.

jnsebgosselin · 2020-10-06T13:07:40Z

ok, I don't know exactly what I did, but after restarting Spyder multiple times and deleting, renaming, copying, moving some files around, changing the root directory, the working directy, closing and opening my Spyder project, etc.... the problem seems to be fixed when running Spyder with PR #13828.

So maybe PR #13828 did fixed the problem after all and the problem was still occurring because of temp files or something? Anyway, I don't know. I suggest not to close this yet, I'll go through my day and I'll report back if the problem is indeed completely resolved or not.

ccordoba12 · 2020-10-06T14:10:25Z

tried with PR #13828 and ... this solve the broken code analysis for those files and there isn't any traceback in the log files anymore...

This is great news! Thanks for the confirmation @jnsebgosselin!

Anyway, I don't know. I suggest not to close this yet, I'll go through my day and I'll report back if the problem is indeed completely resolved or not.

I was investigating this problem yesterday and I think I found the cause. I'm preparing a PR now and I'll ping you when it's up.

jnsebgosselin · 2020-10-06T21:02:10Z

@ccordoba12 after working in Spyder all day using PR #13828, I cannot reproduce this bug anymore. So I guess PR #13828 did indeed fix this issue.

The reason why I was still experiencing this issue after switching to PR #13828 and restarting Spyder might be because of some caching mechanism that eventually got resolved after I restarted Spyder several times and deleted/renamed/moved/restored the problematic folders?

In all cases, I cannot reproduce this problem anymore, so I think this issue should be closed now, or after PR #13828 is merged. It is as you wish.

ccordoba12 · 2020-10-06T21:12:04Z

This is going to be fixed by PR #13938 because even after the fixes in #13828, it's true that the Outline doesn't populate for files in your previous session (except the one that has focus when Spyder starts).

jnsebgosselin · 2020-10-06T21:19:09Z

Here I was talking about files for which the outline explorer did not work at all, even after triggering a document change manually and even if the file was focused on startup.

Maybe I was not clear in my opening post, but it was not related to the issue where the outline tree was not populated automatically for the files that were opened in the previous session. This particular issue, I reported in #13897.

jnsebgosselin · 2020-10-06T21:28:39Z

Just to clarify my last post. I was experiencing two clearly distinct bugs with the outline explorer.

The first one was that the outline explorer was not populating the outline tree automatically for the files that were opened in the previous session. That I reported in Issue #13897, which I think is going to be fixed by PR #13938.

The second bug, which I reported here, was that no matter what I did, for some files, I was not able to get the outline explorer to work. This specific issue I cannot reproduce anymore after working all day with PR #13828.

jnsebgosselin · 2020-10-07T02:17:06Z

OK, I can still reproduce the issue even when using PR #13828, but I have new information that could help to pinpoint the issue.

The issue seems to happen only when a Spyder project is active. When it is the case, then the outline explorer just cannot populate the tree for some files. The wheel is spinning endlessly.

If I close the project, then the outline explorer is able to populate the tree for these same files without any problem.

ccordoba12 · 2020-10-07T04:08:16Z

The first one was that the outline explorer was not populating the outline tree automatically for the files that were opened in the previous session. That I reported in Issue #13897, which I think is going to be fixed by PR #13938.

Sorry, I thought this issue was about that problem. And yes, it's going to be fixed by PR #13938.

The issue seems to happen only when a Spyder project is active. When it is the case, then the outline explorer just cannot populate the tree for some files. The wheel is spinning endlessly.

This should be solved by the combined effect of PRs #13828 and #13938.

So do you want to keep this one open until those PRs are merged?

jnsebgosselin · 2020-10-07T14:35:06Z

Yes I would like to wait before closing this. I'm not yet convinced this is going to be fixed by PRs #13828 and #13938.

jnsebgosselin · 2020-10-07T14:36:37Z

Here is a reproducible case with two Spyder files. You need to have a spyder project for the repo Spyder active for the bug to happen.

ccordoba12 · 2020-10-07T14:45:15Z

And that's even with both my PRs applied?

jnsebgosselin · 2020-10-07T14:49:05Z

And that's even with both my PRs applied?

Yes. The only difference is that the wheel stop spinning after a second or two, but the tree is never populated.

ccordoba12 · 2020-10-07T14:58:30Z

Ok, I'll take a look at that once my PRs are merged.

jnsebgosselin · 2020-10-07T16:07:15Z

@ccordoba12 for the Spyder reproducible case, if I had a __init__.py file in the folder spyder/tools, then the outline explorer is able to build the outline tree for the files create_app.py and rm_whitespace.py. A restart is required after you add or remove the __init__.py file.

jnsebgosselin · 2020-10-07T16:21:31Z

I am able to produce a very simple example for reproducing the issue.

Basically, all you need to do is:

Create a new Spyder project
Add a file with some functions in the newly created project folder.
Create a new sub-folder.
Add a new file in the subfolder.
Open both files in Spyder.

The outline tree for the file that is in the subfolder is not going to be populated unless you add a __init__.py in the subfolder and restart spyder.

ccordoba12 · 2020-10-07T16:42:01Z

The outline tree for the file that is in the subfolder is not going yo be populated unless you add a init.py in the subfolder and restart spyder.

This looks like an issue with Jedi or the PyLS, so not an easy fix. But thanks for the reproducible example.

jnsebgosselin · 2020-10-07T19:55:28Z

This looks like an issue with Jedi or the PyLS, so not an easy fix. But thanks for the reproducible example.

Ok , this is not good news :/

Let me know if I can help further with this issue. I've tried to look more into it, but unfortunately, all that pyls stuff is a little bit over my head...

steff456 self-assigned this Oct 5, 2020

steff456 added the status:Awaiting Followup label Oct 5, 2020

ccordoba12 changed the title ~~The ouline explorer doesn't populate for some files with latest 4.x and code analysis seems broken for these files~~ The ouline explorer doesn't populate for some files with latest 4.x Oct 6, 2020

ccordoba12 added component:Outline Explorer type:Bug and removed status:Awaiting Followup labels Oct 6, 2020

ccordoba12 assigned ccordoba12 and unassigned steff456 Oct 6, 2020

ccordoba12 added this to the v4.2.0 milestone Oct 6, 2020

This was referenced Oct 6, 2020

PR: Don't send requests to the PyLS until it's been properly initialized #13828

Merged

PR: Update outline for files in previous session after Spyder started #13938

Merged

This was referenced Nov 1, 2020

Pass document path to jedi_names when a file is not placed in a module palantir/python-language-server#882

Merged

PR: Check that we can get symbols in a file not part of a Python module #14119

Merged

ccordoba12 closed this as completed in ba69d67 Nov 4, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

The ouline explorer doesn't populate for some files with latest 4.x #13928

The ouline explorer doesn't populate for some files with latest 4.x #13928

jnsebgosselin commented Oct 5, 2020

jnsebgosselin commented Oct 5, 2020 •

edited

Loading

steff456 commented Oct 5, 2020

jnsebgosselin commented Oct 5, 2020

steff456 commented Oct 5, 2020

jnsebgosselin commented Oct 5, 2020

ccordoba12 commented Oct 6, 2020 •

edited

Loading

jnsebgosselin commented Oct 6, 2020 •

edited

Loading

jnsebgosselin commented Oct 6, 2020 •

edited

Loading

jnsebgosselin commented Oct 6, 2020

ccordoba12 commented Oct 6, 2020 •

edited

Loading

jnsebgosselin commented Oct 6, 2020

ccordoba12 commented Oct 6, 2020 •

edited

Loading

jnsebgosselin commented Oct 6, 2020 •

edited

Loading

jnsebgosselin commented Oct 6, 2020 •

edited

Loading

jnsebgosselin commented Oct 7, 2020 •

edited

Loading

ccordoba12 commented Oct 7, 2020

jnsebgosselin commented Oct 7, 2020

jnsebgosselin commented Oct 7, 2020 •

edited

Loading

ccordoba12 commented Oct 7, 2020

jnsebgosselin commented Oct 7, 2020

ccordoba12 commented Oct 7, 2020

jnsebgosselin commented Oct 7, 2020

jnsebgosselin commented Oct 7, 2020 •

edited

Loading

ccordoba12 commented Oct 7, 2020

jnsebgosselin commented Oct 7, 2020

The ouline explorer doesn't populate for some files with latest 4.x #13928

The ouline explorer doesn't populate for some files with latest 4.x #13928

Comments

jnsebgosselin commented Oct 5, 2020

Problem Description

Versions

Dependencies

jnsebgosselin commented Oct 5, 2020 • edited Loading

steff456 commented Oct 5, 2020

jnsebgosselin commented Oct 5, 2020

steff456 commented Oct 5, 2020

jnsebgosselin commented Oct 5, 2020

ccordoba12 commented Oct 6, 2020 • edited Loading

jnsebgosselin commented Oct 6, 2020 • edited Loading

jnsebgosselin commented Oct 6, 2020 • edited Loading

jnsebgosselin commented Oct 6, 2020

ccordoba12 commented Oct 6, 2020 • edited Loading

jnsebgosselin commented Oct 6, 2020

ccordoba12 commented Oct 6, 2020 • edited Loading

jnsebgosselin commented Oct 6, 2020 • edited Loading

jnsebgosselin commented Oct 6, 2020 • edited Loading

jnsebgosselin commented Oct 7, 2020 • edited Loading

ccordoba12 commented Oct 7, 2020

jnsebgosselin commented Oct 7, 2020

jnsebgosselin commented Oct 7, 2020 • edited Loading

ccordoba12 commented Oct 7, 2020

jnsebgosselin commented Oct 7, 2020

ccordoba12 commented Oct 7, 2020

jnsebgosselin commented Oct 7, 2020

jnsebgosselin commented Oct 7, 2020 • edited Loading

ccordoba12 commented Oct 7, 2020

jnsebgosselin commented Oct 7, 2020

jnsebgosselin commented Oct 5, 2020 •

edited

Loading

ccordoba12 commented Oct 6, 2020 •

edited

Loading

jnsebgosselin commented Oct 6, 2020 •

edited

Loading

jnsebgosselin commented Oct 6, 2020 •

edited

Loading

ccordoba12 commented Oct 6, 2020 •

edited

Loading

ccordoba12 commented Oct 6, 2020 •

edited

Loading

jnsebgosselin commented Oct 6, 2020 •

edited

Loading

jnsebgosselin commented Oct 6, 2020 •

edited

Loading

jnsebgosselin commented Oct 7, 2020 •

edited

Loading

jnsebgosselin commented Oct 7, 2020 •

edited

Loading

jnsebgosselin commented Oct 7, 2020 •

edited

Loading