PYTHONPATH handling in neovim breaks YouCompleteMe

[cstrahan]

Issue description

Here's an strace of nvim where the YouCompleteMe plugin (running python3 as the plugin host in nvim, presumably; see trace below) attempts to start the ycmd server (a python2 program):

215 stat("/nix/store/xxwxhdpasj6mcv3mvxdd4n3hyp13izd0-python3-3.7.2-env/lib/python27.zip", 0x7fffe1f0c3b0) = -1 ENOENT (No such file or directory)                                                                                                                                                                                                                                
216 stat("/nix/store/xxwxhdpasj6mcv3mvxdd4n3hyp13izd0-python3-3.7.2-env/lib", {st_mode=S_IFDIR|0555, st_size=134, ...}) = 0                                                                                                                                                                                                                                                       
217 stat("/nix/store/xxwxhdpasj6mcv3mvxdd4n3hyp13izd0-python3-3.7.2-env/lib/python27.zip", 0x7fffe1f0f430) = -1 ENOENT (No such file or directory)                                                                                                                                                                                                                                
218 stat("/nix/store/xxwxhdpasj6mcv3mvxdd4n3hyp13izd0-python3-3.7.2-env/lib/python2.7", 0x7fffe1f0c3b0) = -1 ENOENT (No such file or directory)                                                                                                                                                                                                                                   
219 stat("/nix/store/xxwxhdpasj6mcv3mvxdd4n3hyp13izd0-python3-3.7.2-env/lib", {st_mode=S_IFDIR|0555, st_size=134, ...}) = 0                                                                                                                                                                                                                                                       
220 stat("/nix/store/xxwxhdpasj6mcv3mvxdd4n3hyp13izd0-python3-3.7.2-env/lib/python2.7", 0x7fffe1f0f430) = -1 ENOENT (No such file or directory)                                                                                                                                                                                                                                   
221 stat("/nix/store/xxwxhdpasj6mcv3mvxdd4n3hyp13izd0-python3-3.7.2-env/lib/python2.7/plat-linux2", 0x7fffe1f0c3b0) = -1 ENOENT (No such file or directory)                                                                                                                                                                                                                       
222 stat("/nix/store/xxwxhdpasj6mcv3mvxdd4n3hyp13izd0-python3-3.7.2-env/lib/python2.7", 0x7fffe1f0c3b0) = -1 ENOENT (No such file or directory)                                                                                                                                                                                                                                   
223 stat("/nix/store/xxwxhdpasj6mcv3mvxdd4n3hyp13izd0-python3-3.7.2-env/lib", {st_mode=S_IFDIR|0555, st_size=134, ...}) = 0                                                                                                                                                                                                                                                       
224 stat("/nix/store/xxwxhdpasj6mcv3mvxdd4n3hyp13izd0-python3-3.7.2-env/lib/python2.7/plat-linux2", 0x7fffe1f0f430) = -1 ENOENT (No such file or directory)                                                                                                                                                                                                                       
225 stat("/nix/store/xxwxhdpasj6mcv3mvxdd4n3hyp13izd0-python3-3.7.2-env/lib/python2.7/lib-tk", 0x7fffe1f0c3b0) = -1 ENOENT (No such file or directory)                                                                                                                                                                                                                            
226 stat("/nix/store/xxwxhdpasj6mcv3mvxdd4n3hyp13izd0-python3-3.7.2-env/lib/python2.7", 0x7fffe1f0c3b0) = -1 ENOENT (No such file or directory)                                                                                                                                                                                                                                   
227 stat("/nix/store/xxwxhdpasj6mcv3mvxdd4n3hyp13izd0-python3-3.7.2-env/lib", {st_mode=S_IFDIR|0555, st_size=134, ...}) = 0                                                                                                                                                                                                                                                       
228 stat("/nix/store/xxwxhdpasj6mcv3mvxdd4n3hyp13izd0-python3-3.7.2-env/lib/python2.7/lib-tk", 0x7fffe1f0f430) = -1 ENOENT (No such file or directory)                                                                                                                                                                                                                            
229 stat("/nix/store/xxwxhdpasj6mcv3mvxdd4n3hyp13izd0-python3-3.7.2-env/lib/python2.7/lib-old", 0x7fffe1f0c3b0) = -1 ENOENT (No such file or directory)                                                                                                                                                                                                                           
230 stat("/nix/store/xxwxhdpasj6mcv3mvxdd4n3hyp13izd0-python3-3.7.2-env/lib/python2.7", 0x7fffe1f0c3b0) = -1 ENOENT (No such file or directory)                                                                                                                                                                                                                                   
231 stat("/nix/store/xxwxhdpasj6mcv3mvxdd4n3hyp13izd0-python3-3.7.2-env/lib", {st_mode=S_IFDIR|0555, st_size=134, ...}) = 0                                                                                                                                                                                                                                                       
232 stat("/nix/store/xxwxhdpasj6mcv3mvxdd4n3hyp13izd0-python3-3.7.2-env/lib/python2.7/lib-old", 0x7fffe1f0f430) = -1 ENOENT (No such file or directory)                                                                                                                                                                                                                           
233 stat("/nix/store/xxwxhdpasj6mcv3mvxdd4n3hyp13izd0-python3-3.7.2-env/lib/python2.7/lib-dynload", 0x7fffe1f0c3b0) = -1 ENOENT (No such file or directory)                                                                                                                                                                                                                       
234 stat("/nix/store/xxwxhdpasj6mcv3mvxdd4n3hyp13izd0-python3-3.7.2-env/lib/python2.7", 0x7fffe1f0c3b0) = -1 ENOENT (No such file or directory)                                                                                                                                                                                                                                   
235 stat("/nix/store/xxwxhdpasj6mcv3mvxdd4n3hyp13izd0-python3-3.7.2-env/lib", {st_mode=S_IFDIR|0555, st_size=134, ...}) = 0                                                                                                                                                                                                                                                       
236 stat("/nix/store/xxwxhdpasj6mcv3mvxdd4n3hyp13izd0-python3-3.7.2-env/lib/python2.7/lib-dynload", 0x7fffe1f0f430) = -1 ENOENT (No such file or directory)                                                                                                                                                                                                                       
237 write(2, "ImportError", 11)             = 11                                                                                                                                                                                                                                                                                                                                  
238 write(2, ": ", 2)                       = 2                                                                                                                                                                                                                                                                                                                                   
239 write(2, "No module named site", 20)    = 20                                                                                                                                                                                                                                                                                                                                  
240 write(2, "\n", 1)                       = 1                                                                                                                                                                                                                                                                                                                                   

It would seem that the neovim package does something such that when a python3 neovim plugin starts a python2 program, the python2 program's idea of PYTHONHOME is set to the environment created for the remote plugin host.

This can be confirmed running this in nvim:

:py import os; print(os.environ['PYTHONHOME'])                                                                                                                                                                                                                                                                                                                                    

That will print the /nix/store/xxwxhdpasj6mcv3mvxdd4n3hyp13izd0-python3-3.7.2-env path from above.

Steps to reproduce

$ strace -o process_dump -ff nvim

Note that the ycm plugin complains about the server shutting down. Look through the traces to see the aforementioned "ImportError: No module named site" message.

Technical details

• system: "x86_64-linux"
• host os: Linux 5.0.0-17-generic, Ubuntu, 19.04 (Disco Dingo)
• multi-user?: yes
• sandbox: no
• version: nix-env (Nix) 2.1.3
• channels(root): "nixpkgs"
• channels(cstrahan): "nixpkgs"
• nixpkgs: /nix/var/nix/profiles/per-user/cstrahan/channels/nixpkgs

[cstrahan]

There are a couple resolutions that come to mind:

• Build ycmd with python3, so that the import site works... though only if the ycmd executable uses the exact same version of python3 as the plugin host, and that may not necessarily be the case.
• Patch YouCompleteMe to unset the PYTHONHOME environment variable before starting ycmd. That would fix this case, but that does nothing to solve the general case of a nvim python plugin executing a python script with another python interpreter.

[jonringer]

I would like to update the ycmd nix expression, they don't really do releases, so I'm not sure what point would be a good point to do a checkout.

For now, that PR should resolve this issue.

[cstrahan]

This still has some remaining issues. When opening a python source file, the following message is printed:

InternalError: The subprocess /usr/bin/python2.7 has crashed (EOFError('Ran out of input'), stderr=b'ImportError: No module named site\n').

(I'm on Ubuntu right now, so I'm fairly sure a python2.7 from the store would have been resolved if I were on NixOS; I think that can be ignored as an orthogonal issue for this report)

ycmd's support for python code completion will still spawn a python2.7 interpreter, which will then crash due to PYTHONEHOME pointing at a python3 prefix.

Here's where ycmd figures out which python to use:

https://github.com/ycm-core/ycmd/blob/9df8728a90c67191198c37a8b823010bee3143e4/ycmd/completers/python/python_completer.py#L93-L126

See also: https://github.com/ycm-core/YouCompleteMe#configuring-through-vim-options

I think an easy fix will be patching YouCompleteMe to remove the PYTHONHOME environment variable before launching ycmd.

Again, other plugins and neovim scripts will be subject to similar issues whenever the plugin host launches a python interpreter that doesn't match. Maybe a reasonable solution would be to somehow monkey patch the python subprocess apis in the two python plugin hosts so that any spawned subprocesses don't inherit the plugin environment (the odds that they would utilize any of the packages would seem slim to me). Maybe that would fix more than it would break.

[cstrahan]

Removing the environment variable (and perhaps restoring it after) here should do the trick: https://github.com/ycm-core/YouCompleteMe/blob/04c3505129cd80b92f1b6177dca8aecc55cb0760/python/ycm/youcompleteme.py#L187

[cstrahan]

It would be nice if there was a PYTHON2HOME vs a PYTHON3HOME (or really, it would be nice if we could all just move to python3 and scrub any mention of python2 from existence, but I suppose that's being unreasonable). Given this class of defect is bound to be pervasive, we may want to consider carrying patches to our python interpreters such that they first look for PYTHONHOME, and failing that, look for PYTHON${majorVersion}HOME after.

I'm not convinced that's a perfect solution, but right now wherever we use a python environment via PYTHONHOME, we completely break any python subprocesses from the other major version.

@FRidh, as maintainer of the CPython interpreters, what are your thoughts?

[jonringer]

I'll take a look when i get home. I mean, python2 is being phased out soon, so if it's just compatible upgrades among the packages in the ycmd closure, then it shouldn't be too much of an issue.

I was hoping that bumping the ycmd to python3 would be enough.

[FRidh]

Leaking of PYTHONHOME is a problem. The solution is in my opinion #25985. It also uses an environment variable but then unsets it. Of course, this will break cases where subprocesses do need to invoke Python with these site-packages but instead point to the unwrapped versions. Wrapping sucks.

[FRidh]

#64634 is now in staging.

[siriobalmelli]

in case it helps anyone, my current workaround is:

{ nixpkgs, python }:  # the python passed here is 'python38Full'

with nixpkgs;
with vimPlugins;

nixpkgs.neovim.override {

  vimAlias = true;

  configure = {
    packages.YouCompleteMe.start = [ YouCompleteMe ];  # autocompletion
    # ... other packages elided for clarity ...

    customRC = (nixpkgs.lib.concatStringsSep "\n"
      [ (builtins.readFile ./vimrc)
        ''
        let g:ycm_python_binary_path = '${python}/bin/python'
        ''
      ]
    );
  };
}

[stale]

I marked this as stale due to inactivity. → More info

[jonringer]

sounds like this was resolved, but this issue was forgotten.