Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Include installation instructions for python3 #721

Closed
1 task done
nottrobin opened this issue Dec 9, 2018 · 46 comments
Closed
1 task done

Include installation instructions for python3 #721

nottrobin opened this issue Dec 9, 2018 · 46 comments

Comments

@nottrobin
Copy link

nottrobin commented Dec 9, 2018

  • I have searched the issues of this repo and believe that this is not a duplicate.

Issue

The README states as the ideal way to install Poetry:

curl -sSL https://raw.githubusercontent.com/sdispater/poetry/master/get-poetry.py | python

But, on an Ubuntu Xenial system with only Python 3 installed, this leads to the error bash: python: command not found. If I simply pipe the curl command through python3 instead then it succeds, but then I get:

$ poetry install
/usr/bin/env: 'python': No such file or directory

This is because the top of the installed file at $HOME/.poetry/bin/poetry contains #! /usr/bin/env python.

If, instead, I install poetry with pip3 install poetry, then head -n 1 $(which poetry) shows #!/usr/bin/python3 at the top of the file, which works perfectly.

Ubuntu, from 18.04 Xenial onwards, includes only Python 3 by default. And the minimal version of Xenial doesn't come with Python out of the box at all.

If you install Python 3 on Ubuntu, the binary that's installed is python3. python doesn't exist unless you've also installed Python 2. This deliberately follows PEP 394:

  • python2 will refer to some version of Python 2.x.
  • python3 will refer to some version of Python 3.x.
  • for the time being, all distributions should ensure that python refers to the same target as python2.
  • however, end users should be aware that python refers to python3 on at least Arch Linux (that change is what prompted the creation of this PEP), so python should be used in the shebang line only for scripts that are source compatible with both Python 2 and 3.
  • in preparation for an eventual change in the default version of Python, Python 2 only scripts should either be updated to be source compatible with Python 3 or else to use python2 in the shebang line.

Although the PEP states "python should be used in the shebang line only for scripts that are source compatible with both Python 2 and 3" I'm not sure that this is compatible with the statement "all distributions should ensure that python refers to the same target as python2", because this means that if a system doesn't have Python 2 then the python binary won't exist. I think scripts have to do their best to use python3 if they know it's supported, and most I've encountered do.

I think maybe it would be good if the suggested install command was:

curl -sSL https://raw.githubusercontent.com/sdispater/poetry/master/get-poetry.py | ( python3 || python )

Or maybe you could provide a bash script (if you can rely on bash being available on all relevant systems) so that you could hide the logic of determining the Python version from the user:

curl -sSL https://raw.githubusercontent.com/sdispater/poetry/master/get-poetry.sh | bash

Or simply say in the README:

One of:

``` bash
curl -sSL https://raw.githubusercontent.com/sdispater/poetry/master/get-poetry.py | python3
# or
curl -sSL https://raw.githubusercontent.com/sdispater/poetry/master/get-poetry.py | python

And that the script would set up the poetry binary with either #! /usr/bin/env python3 for Python 3, or #! /usr/bin/env python for Python 2 (which would then allow for the possibility of python changing to Python 3 at some point).

@lqmanh
Copy link

lqmanh commented Dec 11, 2018

I installed poetry on Fedora 29 with

curl -sSL https://raw.githubusercontent.com/sdispater/poetry/master/get-poetry.py | python3

but it's no use.

@nottrobin
Copy link
Author

@lqmanh if this is actively blocking you, just use pip3 install poetry, which works fine.

@mbello
Copy link

mbello commented Jan 11, 2019

This gets worse.

I am on Ubuntu 18.04, and piped the get-poetry.py script to python3.
However, I also have python (=python2) installed in my system. Despite installing it with python3 it seems I have poetry activated with python2

I did not realize it until:

poetry new myproject
<edit pyproject.toml to have python = ">=3.5">
poetry add <package>
                                                                               
[RuntimeError]                                                  
The current Python version (2.7.15) is not supported by the project (>=3.5)  
Please activate a compatible Python version.

So despite having piped it to python3 it still installed under python2. How to change that behaviour?

@sdispater
Copy link
Member

@mbello This has already been discussed extensively in other issues and PR. This is also mentionned in the documentation that Poetry will always use the currently activated Python executable to create the virtualenv.

This has nothing to do with which python version you used to install Poetry.

So I suggest using pyenv and use it to explicitly activate a Python version or wait for the 1.0.0 release which will have the new env command, see #731

@mbello
Copy link

mbello commented Jan 13, 2019 via email

@nottrobin
Copy link
Author

nottrobin commented Jan 14, 2019

Just keep in mind that there is no such thing as 'activated python version'
in debian and ubuntu where python=python2 and python3=python3 always and
both are always 'activated'.

And that PEP 394 defines this as the standard way to name Python system binaries.

Thanks for poetry, love it.

Me too :)

@jgirardet
Copy link
Contributor

Sorry @sdispater but the new version doesn't fix it.
Here are some tests starting from a fresh install :

python3 get-poetry.py

the I update to the new one

$ poetry self update --preview
/home/jimmy/.poetry/lib/poetry/_vendor/py2.7/requests/__init__.py:83: RequestsDependencyWarning: Old version of cryptography ([1, 2, 3]) may cause slowdown.
  warnings.warn(warning, RequestsDependencyWarning)
Updating to 1.0.0a2
 - Downloading poetry-1.0.0a2-linux.tar.gz 100%

I get the this waning from python2

$ poetry env use python3.5
/home/jimmy/.poetry/lib/poetry/_vendor/py2.7/requests/__init__.py:83: RequestsDependencyWarning: Old version of cryptography ([1, 2, 3]) may cause slowdown.
  warnings.warn(warning, RequestsDependencyWarning)
Using virtualenv: /home/jimmy/.cache/pypoetry/virtualenvs/essai_poetry-Wufh11Oj-py3.5
$ poetry env info
/home/jimmy/.poetry/lib/poetry/_vendor/py2.7/requests/__init__.py:83: RequestsDependencyWarning: Old version of cryptography ([1, 2, 3]) may cause slowdown.
  warnings.warn(warning, RequestsDependencyWarning)

Virtualenv
Python:         3.5.2
Implementation: CPython
Path:           /home/jimmy/.cache/pypoetry/virtualenvs/essai_poetry-Wufh11Oj-py3.5
Valid:          True

System
Platform: linux2
OS:       posix
Python:   /usr

I don't know what is this warning but for sure I shouldn't appear if poetry only uses the selected virtualenv.

Finallay after monfying the sheebang line in $HOME/.poetry/bin/poetry to #!/usr/bin/env python3

poetry env info

Virtualenv
Python:         3.5.2
Implementation: CPython
Path:           /home/jimmy/.cache/pypoetry/virtualenvs/essai_poetry-Wufh11Oj-py3.5
Valid:          True

System
Platform: linux
OS:       posix
Python:   /usr

so clearly poetry is not always using the python version of the virtualenv.

@jeking3
Copy link

jeking3 commented Mar 7, 2019

Modifying the poetry script to use python3 seemed to work for me. It defaults to "python" but if your project requires 3.x, poetry fails:

jking@dvm:~/$ poetry show

[RuntimeError]
The current Python version (2.7.15) is not supported by the project (^3.7)
Please activate a compatible Python version.

It's unfortunate to run into something like this within a few minutes of downloading... if the project requires 3.7, could poetry re-run itself under python3 with the same arguments?

@dimaqq
Copy link
Contributor

dimaqq commented Mar 8, 2019

This is also a problem on "out-of-the-box" macOS, where python is 2.7.15.

I ran curl -sSL https://raw.githubusercontent.com/sdispater/poetry/master/get-poetry.py | python3.7

Which produced following ~/.poetry/bin/poetry that starts with:

#!/usr/bin/env python
import glob
import sys
import os

That is poetry still uses Python2.

Which is pretty bad because trying to poetry install on a modern project fails with:

[RuntimeError]
The current Python version (2.7.15) is not supported by the project (^3.7)
Please activate a compatible Python version.

@jeking3
Copy link

jeking3 commented Mar 8, 2019

I abandoned use of it for now. When it doesn't fail less than 5 minutes into use, I'll be back.

@festinuz
Copy link

#721 (comment)

Poetry uses your current python in order to create a matching virtualenv. If you want to use poetry NOW (pre-1.0.0 versions) then you should probably use something like pyenv (as recommended by @sdispater #721 (comment)).

pyenv install 3.7.2  # Install python 3.7.2 to your machine
# Go to project root
pyenv local 3.7.2  # Use python 3.7.2 as default python in current open directory
poetry install  # Will use default 'python', which is set to python3.7.2

@harrybiddle
Copy link

harrybiddle commented Mar 16, 2019

This is also a problem on "out-of-the-box" macOS, where python is 2.7.15.
...
Which is pretty bad because trying to poetry install on a modern project fails with:

[RuntimeError]
The current Python version (2.7.15) is not supported by the project (^3.7)
Please activate a compatible Python version.

As a first-time poetry user I ran into exactly this! When you search the documentation and past issues, the answer you always recieve is "just use pyenv".

But I think this places an unnecessary restriction: it basically means you can't use poetry for a modern Python project unless you also commit to (a) using pyenv, (b) manually changing the shebang of poetry yourself, or (c) running poetry within a Python3 virtual environment. Pipenv behaves better here: if Python 3 is required for the project it'll find it if it's installed and use it.

Sure, we could argue that this is out of the scope of poetry, but it's one of the first issues I hit and I'm sure others will too.

@nottrobin
Copy link
Author

It also feels kind of odd that Poetry, the most modern of package management tools, appears (to those first installing it on their system) be preferring python2 over python3, when python3 has been officially preferred for years, and is finally reaching something of a critical mass of adoption.

@mbello
Copy link

mbello commented Mar 27, 2019

I agree, if they insist on the current behavior they should at least default to using python3 when python2 and python3 are both available.

Nowadays however you can run "poetry env use /usr/bin/python3".

@cmckulka
Copy link

cmckulka commented Apr 19, 2019

Still have the same set of issues with a clean install on Ubuntu 18.04. No "python" by default ... just python3. Even if I change .poetry/bin/poetry -> .poetry/bin/poetry3 I get error because distutils.util is missing when I try running "poetry env use /usr/bin/python3" as suggested above.

And changing .poetry/bin/poetry makes me nervous because I imagine the when poetry updates itself in the future it will overwrite that.

Bummer. Seemed really promising. Looking forward to returning once it matures a bit more.

@mattburgess
Copy link

I've just hit this as well on a Fedora 30 system. They won't have migrated python to point at a Python-3 version until at least F31 but it sounds like they're also waiting until PEP-394 gets finalized to make that move. In the meantime, I don't understand if there would be any problems if the shebang line were just changed by the installer to match the binary that it was invoked with.

e.g. at https://github.com/sdispater/poetry/blob/master/get-poetry.py#L207:

PYTHON_BIN=os.path.basename(sys.executable)
BIN=BIN.replace('python', PYTHON_BIN)

@mattburgess
Copy link

Turns out I'm not the only one to think that approach might work: #1042 does functionally the same thing.

@caleb15
Copy link

caleb15 commented May 14, 2019

Nowadays however you can run "poetry env use /usr/bin/python3".

When I ran that I got

[CommandNotFound]
Command "env" is not defined.

@andreas-vester
Copy link

Maybe setting the default python version to python3 may help.
In Ubuntu, the following line of code seems to work:

update-alternatives --install /usr/bin/python python /usr/bin/python3 10

Credit to @pardhu on https://stackoverflow.com/questions/41986507/unable-to-set-default-python-version-to-python3-in-ubuntu/50331137#50331137

@bersace
Copy link

bersace commented Sep 12, 2019

This issue should not be confused with #536 . As far as I understand, this issue is about what python versions runs poetry itself, independently of what is the requirements of the managed project. One could use poetry, running system python3.7 and still manage projects in 2.7 or 3.6 managed by pyenv.

I'm used to have curl get-foo.py | pythonX.Y to ensure that foo will be executed by the same interpreter as stated in the command line. Juste like pip3.7 will install your package for it's own running interpreter.

why does get-poetry.py always write python in shebang ? Why does get-poetry use /usr/bin/env ?

#1042 does the trick.

@geryogam
Copy link
Contributor

geryogam commented Sep 15, 2019

I think that @bersace is right, the Python interpreter is used in two different contexts that should not be confused:

  • for running the poetry script;
  • for running Python projects.

The Python interpreter used for running the poetry script may be determined by the Python interpreter used to install Poetry. So the #!/usr/bin/env python shebang line of the ~/.poetry/bin/poetry script may reflect that (#!/usr/bin/env {}) instead of hardcoding python. There is the pending PR #1042 for that. But I don't think it is necessary. The current situation is fine as the poetry script can be run by both Python 2 and Python 3 anyway, and I suspect this is the reason why @sdispater did not merge.

The Python interpreter used for running Python projects should be determined by the virtual environment. The poetry add, poetry build, poetry install, poetry lock, poetry remove, poetry run, poetry shell, poetry show and poetry update commands implicitly create and activate a virtual environment (since @sdispater's vision is that every Python project should run within a virtual environment). But they choose the Python interpreter that is running the ~/.poetry/bin/poetry script, that is to say the Python interpreter called by the #!/usr/bin/env python shebang line of the ~/.poetry/bin/poetry script.

@sdispater introduced a separate poetry env command in the merged PR #731, which allows users explicitly creating virtual environments with a specified Python interpreter before running the previous commands. This great feature will be released in Poetry 1.0.0.

However the implicit way is still broken. Poetry should implicitly create a built-in virtual environment with the Python version specified by the tool.poetry.dependencies.python property of the pyproject.toml file, not the Python version resolved by the #!/usr/bin/env python shebang line of the ~/.poetry/bin/poetry script.

@stale
Copy link

stale bot commented Nov 17, 2019

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the stale label Nov 17, 2019
@geryogam
Copy link
Contributor

geryogam commented Nov 17, 2019

@sdispater solved this issue in PR #1477 (I have just tested it on the latest Poetry 1.0.0b5 on MacOS 10.15.1). @nottrobin could you close this issue?

@stale stale bot removed the stale label Nov 17, 2019
@nottrobin
Copy link
Author

Great thanks! 🎉

@kigawas
Copy link

kigawas commented May 12, 2020

This was causing me grief. On a Debian buster system with a system installation of Python 2.7.16 and an alternative installation of Python 3.7.3, poetry itself will run under python2.7 and throw runtime warnings.

Until this gets fixed, I made a gist with a small patch. I use this command to install poetry to run with python3:

curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | eval $(curl -sSL https://git.io/JvXVq) | python3

Note: this is a very trusting thing to do. Don't run this command (or any like it) if you have any doubts about what it's doing.

For details see this gist: get-poetry-1.0.5-pep394-compliance

Pretty dope!

@jabbalaci
Copy link

Why is it still an issue? Python 2 was deprecated in January 2020. A new version of poetry should come out with Python 3 support only! I installed Poetry yesterday (28 May 2020) and it broke on Ubuntu 20.04.

@finswimmer
Copy link
Member

finswimmer commented May 29, 2020

This is fixed by #1878 .

@therefromhere
Copy link

therefromhere commented Jun 7, 2020

I think I'm still hitting this issue with poetry 1.0.8 - on Ubuntu 20.04 without python2 installed (and installing with curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python3) - though I realise that's not the documented install process.

Hacking the shebang of ~/.poetry/bin/poetry from python to python3 seems to fix it for me.

@bersace
Copy link

bersace commented Jun 7, 2020

Same here. I don't understand the point of running Poetry with anything else that sys.executable running get-poetry.py.

@finswimmer
Copy link
Member

The linked PR is currently only in the development branch. This will be available with the 1.1 release.

@therefromhere
Copy link

Fair enough, sorry I noticed 1.0.8 was a new release & assumed this was part of it.

Emerentius added a commit to Emerentius/virtpy that referenced this issue Aug 30, 2020
Workaround for poetry's inability to find a proper python version.
python-poetry/poetry#721
Emerentius added a commit to Emerentius/virtpy that referenced this issue Aug 30, 2020
Workaround for poetry's inability to find a proper python version.
python-poetry/poetry#721
@rarenatoe
Copy link

rarenatoe commented Oct 1, 2020

Here on MacOs 10.15.7, Poetry 1.1.0.

Despite installing with:
url -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python3

I am still getting:

Python 2.7 will no longer be supported in the next feature release of Poetry (1.2).
You should consider updating your Python version to a supported one.

Might be because I previously installed it with python (2.7) but I'm not sure. Did it maybe not uninstall completely?

@finswimmer
Copy link
Member

@rarenatoe, it doesn't matter with which python executable you've installed poetry. poetry will pickup whatever python links at the moment of invoking poetry. This is a little bit different if your system does not provide a python executable. Then poetry will try to find python3 during installation and if this fails python2. Whatever works will be added to the shebang and used afterwards.

abn pushed a commit that referenced this issue Oct 1, 2020
Fixes #3026

If the context wrapped by the temporary_directory() context manager
raised ImportError (for example because distutils.util cannot be
imported, #721 #1837), it would previously keep going, causing a
RuntimeError from contextlib:

    RuntimeError: generator didn't stop after throw()
abn pushed a commit to abn/poetry that referenced this issue Oct 2, 2020
Fixes python-poetry#3026

If the context wrapped by the temporary_directory() context manager
raised ImportError (for example because distutils.util cannot be
imported, python-poetry#721 python-poetry#1837), it would previously keep going, causing a
RuntimeError from contextlib:

    RuntimeError: generator didn't stop after throw()
finswimmer pushed a commit that referenced this issue Oct 2, 2020
Fixes #3026

If the context wrapped by the temporary_directory() context manager
raised ImportError (for example because distutils.util cannot be
imported, #721 #1837), it would previously keep going, causing a
RuntimeError from contextlib:

    RuntimeError: generator didn't stop after throw()

Co-authored-by: Remi Rampin <r@remirampin.com>
@silverwind
Copy link

silverwind commented Oct 15, 2020

Any pointers for the distutils.util error encountered on Ubuntu 20.04 when using python-is-python3?

$ docker run --entrypoint /bin/bash -it 'ubuntu:latest'
$ apt update && apt install python-is-python3 curl
$ curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python -
$ source $HOME/.poetry/env
$ poetry -v
Traceback (most recent call last):
  File "/root/.poetry/bin/poetry", line 17, in <module>
    from poetry.console import main
  File "/root/.poetry/lib/poetry/console/__init__.py", line 1, in <module>
    from .application import Application
  File "/root/.poetry/lib/poetry/console/application.py", line 7, in <module>
    from .commands.about import AboutCommand
  File "/root/.poetry/lib/poetry/console/commands/__init__.py", line 4, in <module>
    from .check import CheckCommand
  File "/root/.poetry/lib/poetry/console/commands/check.py", line 2, in <module>
    from poetry.factory import Factory
  File "/root/.poetry/lib/poetry/factory.py", line 18, in <module>
    from .repositories.pypi_repository import PyPiRepository
  File "/root/.poetry/lib/poetry/repositories/pypi_repository.py", line 33, in <module>
    from ..inspection.info import PackageInfo
  File "/root/.poetry/lib/poetry/inspection/info.py", line 25, in <module>
    from poetry.utils.env import EnvCommandError
  File "/root/.poetry/lib/poetry/utils/env.py", line 20, in <module>
    import packaging.tags
  File "/root/.poetry/lib/poetry/_vendor/py3.8/poetry/core/_vendor/packaging/tags.py", line 7, in <module>
    import distutils.util
ModuleNotFoundError: No module named 'distutils.util'

Edit: This resolved it:

apt install python3-distutils

@Lakitna
Copy link

Lakitna commented Jan 13, 2021

This has still not been solved over a year after Python2 was deprecated. Poetry defaults to my Python2.7 installation over Python3.9. I honestly don't know how to fix an issue I got two minutes into using Poetry.

@sontek
Copy link

sontek commented Jan 13, 2021

I ran:

curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python3.9 -

and then got:

poetry show

Python 2.7 will no longer be supported in the next feature release of Poetry (1.2).
You should consider updating your Python version to a supported one.

Note that you will still be able to manage Python 2.7 projects by using the env command.
See https://python-poetry.org/docs/managing-environments/ for more information.

Is there still no solution for this outside of making "python" be python3? (which isn't the default on OSX and would certainly break some things).

I know I can manually edit ~/.poetry/bin/poetry and change #!/usr/bin/env python -> #!/usr/bin/env python3 but seems like this is something poetry should fix?

@sinoroc
Copy link

sinoroc commented Jan 13, 2021

As far as I understood, the intent of this implementation is that you're supposed to use the get-poetry.py installer in combination with pyenv. And it is pyenv that is supposed to take care of letting the python command point to your preferred Python interpreter.

A related discussion: #3288

@rarenatoe
Copy link

I've never used pyenv, why should I need it just for this? I'm a little confused.

@sontek
Copy link

sontek commented Jan 14, 2021

@sinoroc I don't think using pyenv is a very good option/solution for this. The value of using poetry instead of pip is that you can install it globally and utilize it for all your projects. If you have to use pyenv and activate specific versions of python in every shell you open to interact with poetry then a lot of the value is lost. You might as well just manage your own virtualenv and use pip.

@sinoroc
Copy link

sinoroc commented Jan 14, 2021

@sinoroc I don't think using pyenv is a very good option/solution for this. The value of using poetry instead of pip is that you can install it globally and utilize it for all your projects. If you have to use pyenv and activate specific versions of python in every shell you open to interact with poetry then a lot of the value is lost. You might as well just manage your own virtualenv and use pip.

I don't know. I don't use pyenv. I rarely use poetry. But as far as I understood, this is the intended workflow if you use the get-poetry.py installer (as briefly documented here).

If you have issues with that, I would recommend using something like pipx (or dephell jails) to install poetry (and other similar tools, such as virtualenv, tox, pipenv, pex, shiv, etc.).

abenbrahim78 added a commit to abenbrahim78/python-tracing-demo that referenced this issue Oct 4, 2022
abenbrahim78 added a commit to abenbrahim78/python-tracing-demo that referenced this issue Oct 4, 2022
Copy link

github-actions bot commented Mar 2, 2024

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Mar 2, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests