Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The ouline explorer doesn't populate for some files with latest 4.x #13928

Closed
jnsebgosselin opened this issue Oct 5, 2020 · 25 comments
Closed

Comments

@jnsebgosselin
Copy link
Member

Problem Description

The outline explorer just doesn't want to populate at all for some files on my system. Moreover, the code style and error analysis do not seem to work for these files too.

Here are some lsp logs that I have created by trying to open a very simple file for which the outline explorer doesn't work.

transport_python_1716.log
server_python_1716.log

The content of the file was simply:

# -*- coding: utf-8 -*-


def test_outline_explorer():
    a = 3
    c = b
    return a

Versions

  • Spyder version: 4.2.0.dev0 6e06663
  • Python version: 3.7.6 64-bit
  • Qt version: 5.12.5
  • PyQt5 version: 5.12.3
  • Operating System: Windows 10

Dependencies


# Mandatory:
atomicwrites >=1.2.0           :  1.4.0 (OK)
chardet >=2.0.0                :  3.0.4 (OK)
cloudpickle >=0.5.0            :  1.5.0 (OK)
diff_match_patch >=20181111    :  20200713 (OK)
intervaltree                   :  None (OK)
IPython >=4.0                  :  7.16.1 (OK)
jedi =0.17.2                   :  0.17.2 (OK)
nbconvert >=4.0                :  5.6.1 (OK)
numpydoc >=0.6.0               :  1.1.0 (OK)
paramiko >=2.4.0               :  2.7.1 (OK)
parso =0.7.0                   :  0.7.0 (OK)
pexpect >=4.4.0                :  4.8.0 (OK)
pickleshare >=0.4              :  0.7.5 (OK)
psutil >=5.3                   :  5.7.2 (OK)
pygments >=2.0                 :  2.6.1 (OK)
pylint >=1.0                   :  2.5.3 (OK)
pyls >=0.35.0;<1.0.0           :  0.35.1+1.gd81c7ba (OK)
qdarkstyle >=2.8               :  2.8.1 (OK)
qtawesome >=0.5.7              :  0.8.0.dev0 (OK)
qtconsole >=4.7.7              :  4.8.0.dev0 (OK)
qtpy >=1.5.0                   :  1.9.0 (OK)
rtree >=0.8.3                  :  0.9.4 (OK)
sphinx >=0.6.6                 :  3.1.2 (OK)
spyder_kernels >=1.9.4;<1.10.0 :  1.10.0.dev0 (OK)
watchdog                       :  None (OK)
zmq >=17                       :  19.0.1 (OK)

# Optional:
cython >=0.21                  :  0.29.21 (OK)
matplotlib >=2.0.0             :  3.2.2 (OK)
numpy >=1.7                    :  1.19.1 (OK)
pandas >=1.1.1                 :  1.0.5 (NOK)
scipy >=0.17.0                 :  1.5.0 (OK)
sympy >=0.7.3                  :  None (OK)
@jnsebgosselin
Copy link
Member Author

jnsebgosselin commented Oct 5, 2020

Opening the same file at commit 66e98a5, right before PR #13109 was merged, is working flawlessly.

@steff456
Copy link
Member

steff456 commented Oct 5, 2020

@jnsebgosselin can you please check if the error persists if you update the PyLS repo after this PR palantir/python-language-server#866 ?

@jnsebgosselin
Copy link
Member Author

Thanks for the followup @steff456.

The issue is still happening with latest python-language-server on the develop branch.

I don't know if this is related, but there is an error that is reported in the log files.

2020-10-05 18:35:19,363 UTC - ERROR - pyls_jsonrpc.endpoint - Failed to handle notification textDocument/didOpen: {'textDocument': {'uri': 'file:///D:/test_outline_explorer.py', 'languageId': 'Python', 'version': 1, 'text': '# -*- coding: utf-8 -*-\n\n\ndef test_outline_explorer():\n    a = 3\n    c = b\n    return a\n'}}
Traceback (most recent call last):
  File "C:\Anaconda3\envs\python37\lib\site-packages\pyls_jsonrpc\endpoint.py", line 142, in _handle_notification
    handler_result = handler(params)
  File "C:\Anaconda3\envs\python37\lib\site-packages\pyls_jsonrpc\dispatchers.py", line 23, in handler
    return method(**(params or {}))
  File "C:\Users\User\python-language-server\pyls\python_ls.py", line 298, in m_text_document__did_open
    workspace.put_document(textDocument['uri'], textDocument['text'], version=textDocument.get('version'))
AttributeError: 'NoneType' object has no attribute 'put_document'

This error doesn't occur if I go before PR #13109 was merged on the 4.x branch.

@steff456
Copy link
Member

steff456 commented Oct 5, 2020

What version of json-rpc do you have in the environment?

@jnsebgosselin
Copy link
Member Author

It is version 0.4.0.

image

@ccordoba12
Copy link
Member

ccordoba12 commented Oct 6, 2020

Thanks a lot @jnsebgosselin for your help with tracking this bug!

I don't know if this is related, but there is an error that is reported in the log files.

I think this is related to an error while initializing the PyLS because (according to your traceback) the document is trying to be assigned to a non-existing workspace.

Could you try with PR #13828? That's ready for review and it should solve those issues.

@jnsebgosselin
Copy link
Member Author

jnsebgosselin commented Oct 6, 2020

Thanks @ccordoba12 for your help with this. I tried with PR #13828 and this doesn't solve the problem with the outline explorer not working for some files, but this solve the broken code analysis for those files and there isn't any traceback in the log files anymore...

What is very strange is that this seems to be related to the location of the files. I have folder that if I put any file in it, then Spyder is unable to build the outline explorer tree. If I take these files out and put them in another location, then everything seems to be working fine... this is really strange.

@jnsebgosselin
Copy link
Member Author

jnsebgosselin commented Oct 6, 2020

If I put the file test_outline_explorer.py in C:\Users\User\gwhat\gwhat\meteo it is working, but if I put it in C:\Users\User\gwhat\gwhat\meteo\tests it is not...

I will continue to investigate and report back if I find anything useful.

image

@jnsebgosselin
Copy link
Member Author

ok, I don't know exactly what I did, but after restarting Spyder multiple times and deleting, renaming, copying, moving some files around, changing the root directory, the working directy, closing and opening my Spyder project, etc.... the problem seems to be fixed when running Spyder with PR #13828.

So maybe PR #13828 did fixed the problem after all and the problem was still occurring because of temp files or something? Anyway, I don't know. I suggest not to close this yet, I'll go through my day and I'll report back if the problem is indeed completely resolved or not.

@ccordoba12 ccordoba12 changed the title The ouline explorer doesn't populate for some files with latest 4.x and code analysis seems broken for these files The ouline explorer doesn't populate for some files with latest 4.x Oct 6, 2020
@ccordoba12 ccordoba12 assigned ccordoba12 and unassigned steff456 Oct 6, 2020
@ccordoba12 ccordoba12 added this to the v4.2.0 milestone Oct 6, 2020
@ccordoba12
Copy link
Member

ccordoba12 commented Oct 6, 2020

tried with PR #13828 and ... this solve the broken code analysis for those files and there isn't any traceback in the log files anymore...

This is great news! Thanks for the confirmation @jnsebgosselin!

Anyway, I don't know. I suggest not to close this yet, I'll go through my day and I'll report back if the problem is indeed completely resolved or not.

I was investigating this problem yesterday and I think I found the cause. I'm preparing a PR now and I'll ping you when it's up.

@jnsebgosselin
Copy link
Member Author

@ccordoba12 after working in Spyder all day using PR #13828, I cannot reproduce this bug anymore. So I guess PR #13828 did indeed fix this issue.

The reason why I was still experiencing this issue after switching to PR #13828 and restarting Spyder might be because of some caching mechanism that eventually got resolved after I restarted Spyder several times and deleted/renamed/moved/restored the problematic folders?

In all cases, I cannot reproduce this problem anymore, so I think this issue should be closed now, or after PR #13828 is merged. It is as you wish.

@ccordoba12
Copy link
Member

ccordoba12 commented Oct 6, 2020

This is going to be fixed by PR #13938 because even after the fixes in #13828, it's true that the Outline doesn't populate for files in your previous session (except the one that has focus when Spyder starts).

@jnsebgosselin
Copy link
Member Author

jnsebgosselin commented Oct 6, 2020

Here I was talking about files for which the outline explorer did not work at all, even after triggering a document change manually and even if the file was focused on startup.

Maybe I was not clear in my opening post, but it was not related to the issue where the outline tree was not populated automatically for the files that were opened in the previous session. This particular issue, I reported in #13897.

@jnsebgosselin
Copy link
Member Author

jnsebgosselin commented Oct 6, 2020

Just to clarify my last post. I was experiencing two clearly distinct bugs with the outline explorer.

The first one was that the outline explorer was not populating the outline tree automatically for the files that were opened in the previous session. That I reported in Issue #13897, which I think is going to be fixed by PR #13938.

The second bug, which I reported here, was that no matter what I did, for some files, I was not able to get the outline explorer to work. This specific issue I cannot reproduce anymore after working all day with PR #13828.

@jnsebgosselin
Copy link
Member Author

jnsebgosselin commented Oct 7, 2020

OK, I can still reproduce the issue even when using PR #13828, but I have new information that could help to pinpoint the issue.

The issue seems to happen only when a Spyder project is active. When it is the case, then the outline explorer just cannot populate the tree for some files. The wheel is spinning endlessly.

If I close the project, then the outline explorer is able to populate the tree for these same files without any problem.

outline_explorer_not_working_in_project

@ccordoba12
Copy link
Member

The first one was that the outline explorer was not populating the outline tree automatically for the files that were opened in the previous session. That I reported in Issue #13897, which I think is going to be fixed by PR #13938.

Sorry, I thought this issue was about that problem. And yes, it's going to be fixed by PR #13938.

The issue seems to happen only when a Spyder project is active. When it is the case, then the outline explorer just cannot populate the tree for some files. The wheel is spinning endlessly.

This should be solved by the combined effect of PRs #13828 and #13938.

So do you want to keep this one open until those PRs are merged?

@jnsebgosselin
Copy link
Member Author

Yes I would like to wait before closing this. I'm not yet convinced this is going to be fixed by PRs #13828 and #13938.

@jnsebgosselin
Copy link
Member Author

jnsebgosselin commented Oct 7, 2020

Here is a reproducible case with two Spyder files. You need to have a spyder project for the repo Spyder active for the bug to happen.

image

@ccordoba12
Copy link
Member

And that's even with both my PRs applied?

@jnsebgosselin
Copy link
Member Author

And that's even with both my PRs applied?

Yes. The only difference is that the wheel stop spinning after a second or two, but the tree is never populated.

@ccordoba12
Copy link
Member

Ok, I'll take a look at that once my PRs are merged.

@jnsebgosselin
Copy link
Member Author

@ccordoba12 for the Spyder reproducible case, if I had a __init__.py file in the folder spyder/tools, then the outline explorer is able to build the outline tree for the files create_app.py and rm_whitespace.py. A restart is required after you add or remove the __init__.py file.

@jnsebgosselin
Copy link
Member Author

jnsebgosselin commented Oct 7, 2020

I am able to produce a very simple example for reproducing the issue.

Basically, all you need to do is:

  1. Create a new Spyder project
  2. Add a file with some functions in the newly created project folder.
  3. Create a new sub-folder.
  4. Add a new file in the subfolder.
  5. Open both files in Spyder.

The outline tree for the file that is in the subfolder is not going to be populated unless you add a __init__.py in the subfolder and restart spyder.

image

@ccordoba12
Copy link
Member

The outline tree for the file that is in the subfolder is not going yo be populated unless you add a init.py in the subfolder and restart spyder.

This looks like an issue with Jedi or the PyLS, so not an easy fix. But thanks for the reproducible example.

@jnsebgosselin
Copy link
Member Author

This looks like an issue with Jedi or the PyLS, so not an easy fix. But thanks for the reproducible example.

Ok , this is not good news :/

Let me know if I can help further with this issue. I've tried to look more into it, but unfortunately, all that pyls stuff is a little bit over my head...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants