Support linting inside current Virtualenv

[Routhinator]

Is your feature request related to a problem? Please describe.
Linter does not have a setting to support virtualenvs. PyLint ends up reporting valid imports as errors.

Describe the solution you'd like
A setting to tell PyLint plugin to use the configured virtualenv, similar to https://github.com/perses76/pylint-pycharm.git so we can get proper reports from the linter.

Describe alternatives you've considered
I can workaround this by not using this plugin and simply configuring an external tool with https://github.com/perses76/pylint-pycharm.git for now (setup is here for any curious: https://stackoverflow.com/a/48549144/2262288)

[bivald]

Amazing plugin!

I've worked around it locally by creating a pylint2 file which I point to and in it I add:

#!/path/to/bin/python3.4

# -*- coding: utf-8 -*-
import re
import sys
sys.path.append('/path/to/code')
from pylint import run_pylint

if __name__ == '__main__':
    sys.argv[0] = re.sub(r'(-script\.pyw?|\.exe)?$', '', sys.argv[0])
    sys.exit(run_pylint())

But in reality I would love to see that it adds all content roots which are marked as sources to pylint, or ability to select which source roots to use.

[bivald]

That said, can't you simply install pylint in the virtualenv and point to this binary? This will still give you import errors for your own code, but for things installed in your venv it should work. Perhaps I'm missing something :)

[Routhinator]

I actully tried this and uninstalling pylint from the host system and installing pylint in the virtualenv does work so I am using this plugin now, however it feels a bit... dirty? to include a linter in my project requirements. However I suppose I can simply change my thinking here and have a requirements.txt and a requirements-prod.txt and maintain both files.

[leinardi]

Hey guys, I am trying to find a solution but I am a Java developer that started to mess with Python only recently, so I need help to fully understand the problem.

Currently I am able to get the path of the current environment used on the PyCharm project:
ProjectRootManager.getInstance(project).getProjectSdk().getHomePath() will return the path to my venv, if I am using one.
But how can I tell pylint to use this path?

The python tool you linked is sourcing the content of the venv path + "activate": https://github.com/tgamauf/pylint-pycharm/blob/ff16d80b894275623503bd0e6fe7a8a2903bb125/pylint_pycharm/converter.py#L132

Is this the only way?

[leinardi]

Ok, after some investigation it seems that the activate file is just changing 3 environmental variables:

1. Adding VIRTUAL_ENV containing the virtualenv path
2. unsetting PYTHONHOME if set
3. PATH="$VIRTUAL_ENV/bin:$PATH"

I should be able to do those things directly from the plugin before running Pylint.

I will provide a test build soon and I would love is someone of you guys could test it.

[bivald]

I'm not sure if you need all that, or if you just can supply the binary to the virtualenv since that is the correct environment it (according to me) should run in. What would be nice would be if it can use your projects python interpreter (since that's often the virtualenv)

[leinardi]

OK, I think I fixed it.

@Routhinator @bivald @kwiersma, @switchtrue please test this build an let me know if is fixed also for you:
pylint-plugin-0.9.0.zip

EDIT:
I just find out that my fix only works if you specify the absolute path to pylint: if you test please go to the plugin settings and add the absolute path.
EDIT2:
Never mind, I have implemented some basic path auto-detection for *nix systems. Just download the plugin again to be sure to have the latest version.

[switchtrue]

This works perfectly!

[leinardi]

Hi all, unfortunately I just find out that the fix only works if you install Pylint in your virtualenv (@switchtrue can you please confirm this?).

I just noticed that, after setting up a venv from scratch, the plugin was not recognizing Pylint executable, even after providing the full path:

Traceback (most recent call last):
  File "/usr/local/bin/pylint", line 7, in <module>
    from pylint import run_pylint
ModuleNotFoundError: No module named 'pylint'

Any idea on how to fix this?

[leinardi]

Ok, my current plan is to make the plugin install Pylint inside the virtualenv if is not already installed. But I have no idea how to do it. I have asked for help here. Perhaps @vlasovskikh will have a look at it :)

[switchtrue]

Yes, I can confirm this!

Interesting. I always install pylint inside my virtualenvs as I use different versions for different project. As long as the auto install only installs if it's not already found then I should still be able to install whatever version I want outside of the plugin and have the plugin use it.
Ideally, if its auto installed we should make it clear that it's happened and at which version.

[leinardi]

OK, I have a new fix for this issue. I implemented a better auto-detect for the Pylint executable and I can now install Pylint inside the current project interpreter if is missing.

Please test this new version and let me know if works fine (if possible test multiple scenarios like a fresh project with no venv set, then one with a venv and another with the system env).

pylint-plugin-0.9.0.zip

[leinardi]

Has anyone tested the new fix? Should I release it?

[switchtrue]

Yes, apologies! I did and I started drafting a comment here but got distracted.

It seems to work really well. The "Auto-detected: ..." message in the settings is a really nice touch.

One issue I've noticed though it that it is linting files I have ignored in my pylintrc file. In pylintrc you can add an ignore like like this:

# Add files or directories to the blacklist. They should be base names, not
# paths.
ignore=CVS,migrations,local_settings.py,manage.py,wsgi.py

If I run pylint manually these do not get linted, but if I run via this plugin they do get linted and in my setup they fail which is a bit of an issue. All other pylintrc settings seem to get respected so I'm guessing this is an artifact of how the files are passed to pylint. Do you want me to raise another issue for this?

[leinardi]

It seems to work really well.

Nice! Then I procede with a new release 👍

One issue I've noticed though it that it is linting files I have ignored in my pylintrc file.

Mmm this could be hard to fix: the plugin is giving pylint a list of all the files it has to scan and I guess pylint is not respecting the pylintrc ignore if you ask to scan explicitly an ignored file...

I will open a separate bug to keep track of this issue but I don't see any easy solution...

[cpchen]

Hello, I have a issue with using a hardcoded pylint path that's in a virtualenv.

I'm using Intellij IDEA Ultimate 2018.2.4 on a Mac. The Pylint plugin is v0.10.2

I see:

Pylint Plugin (Unable to run Pylint) Pylint is installed inside the project environment but the plugin is not able to run it. If you just installed it try to restart your IDE, if the problem persists you may need to manually enter the path to the Pylint executable inside the Plugin settings. 

Pylint Plugin: Failure: executable "/Users/penc/virtualenvs/data-manager/bin/pylint" not found

in the log, even though the file is there and I can execute it directly. (I've also restarted the IDE many times.)

Any ideas?

[leinardi]

Hi @cpchen, could you please attach the idea.log file (https://intellij-support.jetbrains.com/hc/en-us/articles/207241085-Locating-IDE-log-files) after you get that error message?

[cpchen]

2018-10-12 09:26:00,023 [  69537]  ERROR - harm.pylint.plapi.PylintRunner - Error while checking Pylint path 
com.intellij.execution.process.ProcessNotCreatedException: Cannot run program "/Library/Java/JavaVirtualMachines/jdk-10.0.2.jdk/Contents/Home": error=13, Permission denied
	at com.intellij.execution.configurations.GeneralCommandLine.createProcess(GeneralCommandLine.java:408)
	at com.leinardi.pycharm.pylint.plapi.PylintRunner.isPylintPathValid(PylintRunner.java:92)
	at com.leinardi.pycharm.pylint.plapi.PylintRunner.checkPylintAvailable(PylintRunner.java:176)
	at com.leinardi.pycharm.pylint.plapi.PylintRunner.checkPylintAvailable(PylintRunner.java:150)
	at com.leinardi.pycharm.pylint.PylintInspection.inspectFile(PylintInspection.java:76)
	at com.leinardi.pycharm.pylint.PylintInspection.lambda$checkFile$0(PylintInspection.java:65)
	at com.intellij.openapi.application.impl.ApplicationImpl$2.call(ApplicationImpl.java:337)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: Cannot run program "/Library/Java/JavaVirtualMachines/jdk-10.0.2.jdk/Contents/Home": error=13, Permission denied
	at java.lang.ProcessBuilder.start(ProcessBuilder.java:1048)
	at com.intellij.execution.configurations.GeneralCommandLine.startProcess(GeneralCommandLine.java:436)
	at com.intellij.execution.configurations.GeneralCommandLine.createProcess(GeneralCommandLine.java:404)
	... 10 more
Caused by: java.io.IOException: error=13, Permission denied
	at java.lang.UNIXProcess.forkAndExec(Native Method)
	at java.lang.UNIXProcess.<init>(UNIXProcess.java:247)
	at java.lang.ProcessImpl.start(ProcessImpl.java:134)
	at java.lang.ProcessBuilder.start(ProcessBuilder.java:1029)
	... 12 more

[leinardi]

Mmm this looks strange, can you check if the Python interpreter is properly set in your PyCharm project? Can you try to switch to another one (e.g. local venv) to test if you get the same error?

[cpchen]

The Python interpreter is set to the one inside the venv the Pylint bin is in. I can try another venv, if you think that would make a difference?

[leinardi]

If you can post the entire idea.log here (you can use the attach feature), there is a problem with the path "/Library/Java/JavaVirtualMachines/jdk-10.0.2.jdk/Contents/Home" and I don't understand from where is coming. I think the plugin is recognizing that as the python interpreter for some reason.

[cpchen]

idea.log

[leinardi]

Yep, for some reason you interpreter configuration is returning your Home as the path of the interpreter.

Please try to remove the interpreter configuration from your project and re-add it making sure that the base interpreter is pointing to a python bin:

[cpchen]

I'm using IntelliJ and the project is a Java project with a Python module. I don't see a global project interpreter I can set here, but this is my configuration (the 10 folder is the Java JDK):

[leinardi]

I am sorry but I fear this configuration is not currently supported. It looks like the python core plugin is giving back the wring path for the interpreter.

[cpchen]

Argh. So you think there's no configuration change I can make on my end? I thought the Python plugin needs to respect what bin I pass it.

…

On Fri, Oct 12, 2018, 10:08 AM Roberto Leinardi ***@***.***> wrote: I am sorry but I fear this configuration is not currently supported. It looks like the python core plugin is giving back the wring path for the interpreter. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#7 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAho2yFuL2xeiEmzlmEzaHoXjMdzcz3zks5ukMyZgaJpZM4WYFnY> .

[vlasovskikh]

I guess the plugin tries the wrong SDK. In IntelliJ there may be many SDKs for different programming languages. On the screenshot I see that the project has a Java 10 SDK and a Python 2.7 virutalenv SDK set up.

@leinardi SDKs can be associated with a) the Project and b) with Modules of the project. If you run your code in the context of some file opened in the editor, you can use PythonSdkType#findPythonSdk(PsiElement): Sdk? to get the correct SDK. See PsiManager#findFile(VirtualFile): PsiFile? to get a PSI file from your VirtualFile and pass it to findPtyhonSdk.

[vlasovskikh]

@cpchen I'm not sure that your issue is related to the original issue. It's clearly about running the plugin in IntelliJ with several SDKs set up.

[leinardi]

@vlasovskikh thank you for your comment, that information might be very useful in the future, but for now I have to find the interpreter without the need of a VirtualFile (I need to be able to run pylint also before starting a scan of a file).
Is there a way to get a list of all the environments that I can filter to pick the Python one? Currently this is how I am getting the SDK: Sdk projectSdk = ProjectRootManager.getInstance(project).getProjectSdk();

[cpchen]

Looking at a comment in the top post in https://stackoverflow.com/questions/24769117/how-do-i-configure-a-python-interpreter-in-intellij-idea-with-the-pycharm-plugin

it looks like I can create a Java project with Python facet for the same module. would you be able to grab the interpreter that way?

[cpchen]

let's move this discussion to #31