Skip to content
This repository has been archived by the owner on Jun 14, 2024. It is now read-only.

Could there be an option which controls the version of python to be used. #18

Closed
d0u9 opened this issue Dec 9, 2015 · 32 comments
Closed

Comments

@d0u9
Copy link

d0u9 commented Dec 9, 2015

I have set the homebrew installed python as the default python interpreter in my MAC OSX. However, lldb.vim seems uncompatible with it because of that lldb python module can only be used with the python which is shipped with system.

I hope there could be an option that allows me to select which version python to run. In that way, the vim runtime python can be separated from the python lldb.nvim works with.

@critiqjo
Copy link
Collaborator

critiqjo commented Dec 9, 2015

As I said earlier (in #16), did you try setting g:python_host_prog?

let g:python_host_prog = '/usr/bin/python2'

@d0u9
Copy link
Author

d0u9 commented Dec 10, 2015

I encountered a new problem, if I set let g:python_host_prog = '/usr/bin/python2' another plugin die. different plugins need different versions of python...

Majority plugins can cooperate with any version of python. However, lldb module only works fine with the default. That is why I open a new issue to post my will of lldb.nvim.

@critiqjo
Copy link
Collaborator

different plugins need different versions of python...

LLDB's python module can work with any version of python2 (I think -- greater than 2.6). Making it available with the version installed through homebrew might be tricky.

In my system, LLDB's python files are in this directory: /usr/lib/python2.7/site-packages/lldb. If I had a different (newer) version of python installed in /usr/local/, I would create a symlink like this:

ln -s /usr/lib/python2.7/site-packages/lldb /usr/local/lib/python2.7/site-packages/lldb

which should probably make it work...

@d0u9
Copy link
Author

d0u9 commented Dec 10, 2015

Sorry, I forgot to note that the lldb module in my system is shipped with Xcode. On Linux system, lldb works properly. Maybe, Apple's lldb is different.

The lldb library locates at /Applications/Xcode.app/Contents/SharedFrameworks/LLDB.framework/Resources/Python.

After making symbol link, when I import lldb in python which is installed through homebrew it reports:

Fatal Python error: PyThreadState_Get: no current thread
[1]    87589 abort      python

and a window pops up with Python quit unexpectedly. Click Reopen to open the application again. Click Report to see more detailed information and send a report to Apple. filled in.

@critiqjo
Copy link
Collaborator

A bit of searching around landed me here: https://github.com/Homebrew/homebrew/blob/master/share/doc/homebrew/Common-Issues.md#python-segmentation-fault-11-on-import-some_python_module

So symlinking will not work (sorry). I guess the only path left is to install LLDB from homebrew (I think it comes with LLVM). And while at it, try to install the latest version, 3.7.

@critiqjo
Copy link
Collaborator

@d0u9 May be doing this instead of symlinking will work.

(If only I could get my hands on an Apple! I would go on a testing spree!)

@LastSecondsToLive
Copy link

@d0u9 How did you finally fixed the problem? I'm stuck in the exact same situation at the moment (I need to link the Xcode lldb to the brew python.).

@d0u9
Copy link
Author

d0u9 commented Dec 20, 2015

@LastSecondsToLive, Sorry, I tried a lot but failed to make Xcode lldb compatible with brew python.

@LastSecondsToLive
Copy link

@d0u9 So I guess I'll need to install neovim together with the neovim python modules via the system python? Is there any way I can use the lldb.nvim?

@critiqjo
Copy link
Collaborator

It's probably impossible to make Xcode's LLDB compatible with brew python (as explained here). If you simply have to use brewed python with Neovim, you'll have to brew LLDB too, packaged with LLVM (?):

brew install llvm --with-python --with-lldb

But if you can make do with system python in just Neovim (leaving PATH intact), set the global variable g:python_host_prog to system python path (/usr/bin/python2) in init.vim, and do /usr/bin/pip2 install neovim so that system python gets the Neovim client module. In this case, you can probably also set PYTHONPATH appropriately in init.vim without messing with your bash/zsh environment:

let $PYTHONPATH = "/Applications/Xcode.app/Contents/SharedFrameworks/LLDB.framework/Resources/Python:" . $PYTHONPATH

@LastSecondsToLive
Copy link

Ok, I'll probably use the system python then. I just noticed, that there is no pip2 in /usr/bin/. What can I do about this? And I should probably uninstall neovim from my current pip and pip3 (since these pip's are from homebrew, right?

@critiqjo
Copy link
Collaborator

You don't need to uninstall neovim from pip, it will just stay there unused. pip3 neovim installation will be used by Neovim's python3 plugins, if you have any (eg. deoplete).

I used /usr/bin/pip2 to emphasize that neovim module should be installed from system's python2 pip (not 3); it is probably named /usr/bin/pip then...

@LastSecondsToLive
Copy link

Hm. No pip in /usr/bin/ neither. Do I need to install a new pip/pip2?

@LastSecondsToLive
Copy link

Ok, I'm really confused right now. I installed pip with easy_install in /Library/Python/2.7/site-packages/. When I write which pip I still get /usr/local/bin/pip- the homebrew pip. pip --version shows pip 7.1.2 from /Library/Python/2.7/site-packages/pip-7.1.2-py2.7.egg (python 2.7). Is /Library/Python/2.7/.../ the homebrew pipor the system pip? Sorry, I just spend to much time figuring this out and I think I'm just confused right now...

@critiqjo
Copy link
Collaborator

@LastSecondsToLive I'm not sure! I've never used a Mac, sorry!
@JayBusch Help! Do you have /usr/bin/pip? If yes, how did you get it (Did you ever [re]install pip itself)? If not, what do you get for which pip (with your modified PATH)?

@LastSecondsToLive
Copy link

@critiqjo I'm very glad you tried to help. You don't own a mac, but still post your ideas here - one day you'll own a mac as well and can go on a testing spree ;) For know I'll probably just wait for @JayBusch to jump in here, since I'm really excited for some LLDB debugging inside of neovim.

@d0u9
Copy link
Author

d0u9 commented Dec 21, 2015

@LastSecondsToLive if you really want to use lldb.nim, you need to set the python environment of neovim to the system default python.

let g:python_host_prog = '/usr/bin/python2'

However, due to the SIP of OSX 10.11, pip install neovim won't work for the naive python, so you have to disable SIP in order to install.

@d0u9
Copy link
Author

d0u9 commented Dec 21, 2015

@LastSecondsToLive
Also, there is a compromise, you can create a file mymodule.pth in python2 dist-packages path, i.e. /usr/lib/python2.7/dist-packages, append the library path of brew python to mymodule.pth. Now your system naive python can share the library of your brew python.

Note: Creating file in /usr/lib/python2.7/dist-packages needs SIP disabled as well.

For example, suppose that the library of brew python locates at /usr/local/lib/python2.7/site-packages/ in which the neovim python library is included.

  1. Disable SIP
  2. Create mymodule.pth in /usr/lib/python2.7/dist-packages.
  3. Append /usr/local/lib/python2.7/site-packages/ to mymodule.pth.
  4. Disable SIP, if you want.
  5. Set let g:python_host_prog = '/usr/bin/python2' in your init.vim.

@LastSecondsToLive
Copy link

@d0u9 Wow, that's crazy. 'SIP' seems like a pretty harsh restriction for developers - I just read about it. What do I have to do after I disabled SIP? I don't think pip install neovim will work (even with SIP disabled, since I couldn't find a preinstalled pip... Thank you really much for your help!

@LastSecondsToLive
Copy link

Wow, I'll try this! Tomorrow I think - It's 3:57AM in Germany - I'll report tomorrow.

@LastSecondsToLive
Copy link

@d0u9 I don't need to do the previous steps if I go with the compromise, right?

@d0u9
Copy link
Author

d0u9 commented Dec 21, 2015

@LastSecondsToLive, if you have brew python installed, before you operating naive python, maybe you have to execute brew unlink python before and brew link python after. I don't remember if the naive python shipped with pip, if not, you need to install it manually.

@d0u9
Copy link
Author

d0u9 commented Dec 21, 2015

@LastSecondsToLive Yes, if you don't want to mess up your library of naive python, follow my second reply.

@LastSecondsToLive
Copy link

@d0u9 I just did a factory reset (I messed something) up. So I'm good to go I think. Are this the steps I should do (I have nothing else than the default software at the moment)? I really don't want to mess something up again...

  1. Install brew
  2. Install neovim, python (including pip) and python3 (including pip3) via brew
  3. Disable SIP and create mymodule.pth in /usr/lib/python2.7/dist-packages.
  4. Append /usr/local/lib/python2.7/site-packages/ (the path to the brewed python) to mymodule.pth. mymodule.pth will only contain this line of text right?
  5. Enable SIP again (in general I'm PRO-Protection)
  6. Install neovim (the python module) with the brewed python/pip - not the brewed python3
  7. Set let g:python_host_prog = '/usr/bin/python2' in my init.nvim.

Now neovim should use the native python which has access to LLDB and has access to the shared library of the brewed python which in turn has access to the 'neovim' module. This seems like something easy to mess up, but I'll try my best.

Questions:
I don't need to link/unlink brew python in this order, right?
I don't need access to a native pip (OS X doesn't ship with pip), right - I use the brewed python/pip once to install the neovim module?
And is it save to not install python3 (including pip3) via brew - since we don't need it for this?

@d0u9 Thank you so much already for your effort!

@d0u9
Copy link
Author

d0u9 commented Dec 21, 2015

mymodule.pth will only contain this line of text right?

Yes, only one line in this file. If you want to test whether it works properly, you can import any libraries which are installed by brew python's pip into the naive python.

I don't need to link/unlink brew python in this order, right?

I am not very sure if this is a prerequisite step. Maybe, this step is only necessary for installing YouCompleteMe.

I don't need access to a native pip (OS X doesn't ship with pip), right - I use the brewed python/pip once to install the neovim module?

Yes, if you want to use brew python instead of naive python, you do not oblige to install pip for naive python at all.

And is it save to not install python3 (including pip3) via brew - since we don't need it for this?

Python3 is unnecessary I think. I haven't try any thing about python3.

BTW, It seems that brew python now is updated to 2.7.11 compare to the naive python 2.7.10. I am not certain about the effect which may be brought by the difference of these two versions of python.

@critiqjo
Copy link
Collaborator

BTW, It seems that brew python now is updated to 2.7.11 compare to the naive python 2.7.10. I am not certain about the effect which may be brought by the difference of these two versions of python.

Since neovim module is completely written in python (as opposed to one which loads a shared library such as lldb module), it's much more probable to just work across versions.

And yeah, python3 is not necessary for this plugin.

Thanks @d0u9

@JayBusch
Copy link

Sorry for the late reply; not totally caught up either, but here's what I think you were asking.

I do not have /usr/bin/pip

My pip according to "which pip" is in /usr/local/bin

It should be noted though that although I installed Python via homebrew, I installed NeoVim via source from GitHub.

I am also getting a Python crash now that I have installed YouCompleteMe, and have not had time to look into it. I imagine it is a result of YouCompleteMe is linked to the Python binary not on my path. I'll look into it further this week as time allows.

@LastSecondsToLive
Copy link

Ok, I finally got it (I didn't even need to disable SIP or do something special).

This is how to get lldb.nvim on OS X:

  1. Install homebrew.

  2. Install neovim via homebrew (See this - Since I installed it via homebrew, I can't be sure if it'll work with another neovim build).

  3. Install python via homebrew (brew install python).
    The just brewed python will be located under /usr/local/bin/python.
    Homebrew will install pip as well when installing python. This brewed pip will be located under /usr/local/bin/pip2.

  4. Be sure that the default pip/pip2 is the brewed pip (which pip and which pip2 should return /usr/local/bin/pip respectively /usr/local/bin/pip2 - otherwise modify your $PATH). Check the same for the python.

  5. Install neovim python module with the just brewed pip2 (pip2 install neovim).
    Pythonshould now be able to import neovim (Optional: Test in Python Interactive Mode).

  6. The naive python (located under /usr/bin/python is not able to import LLDB. To change this we add this to our .bash_profile:

    export PYTHONPATH="/Applications/Xcode.app/Contents/SharedFrameworks/LLDB.framework/Resources/Python:${PYTHONPATH}"
    

    NOTE: Only the naive python located under usr/bin/python will be able to import the LLDB python module (with import lldb) - our default python is the brewed python located under /usr/local/bin/python.

  7. The naive python still doesn't have access to the brewed neovim module. We change this by easily adding:

    export PYTHONPATH="${PYTHONPATH}:/usr/local/lib/python2.7/site-packages"
    

    to our .bash_profile. Now the naive python is able to access the LLDB module and is able to access the neovim module (which was installed with a brewed pip). Our default python is still the brewed python. That's fine. But neovim should use the naive python with all it's possibilities to import LLDB & neovim modules.

  8. To let neovim use the naive python. Add this to your .config/nvim/init.vim:

    let g:python_host_prog = '/usr/bin/python2.7' 
    

DONE.

This took me quite a while now and I really hope this post will save some people time in the future :) Thanks to @d0u9, @critiqjo and @JayBusch for the tips and tricks, which finally brought me here.

@d0u9
Copy link
Author

d0u9 commented Dec 23, 2015

@LastSecondsToLive PYTHONPATH environment can be accessed both by python3 and python2. So, if you set this environment only to a python2 library path, python3 will report error when you import libraries due to that python3 also accesses python2's library.

I encountered this problem before, and asked THIS question. Creating a file and fill it with path of brew python's library seems a safe way to restrict such impact.

@LastSecondsToLive
Copy link

@d0u9 That makes sense!
It didn't work at first, but I created the mymodule.pth in the wrong folder. It has to be located in /Library/Python/2.7/site-packages (not /usr/lib/python2.7/dist-packages). mymodule.pth contains the line /usr/local/lib/python2.7/site-packages and it's working!

Thank you really much. This will prevent me from messing up python3 in the future.

@critiqjo
Copy link
Collaborator

@LastSecondsToLive Step 6 should be unnecessary due to this (where the plugin itself tries to locate the module and add it to sys.path). Try executing the test script (without step 6).

critiqjo pushed a commit that referenced this issue Mar 11, 2016
@revolter
Copy link

@LastSecondsToLive, Step 7 doesn't work, still get the Fatal Python error: PyThreadState_Get: no current thread error.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants