ModuleNotFoundError: No module named 'twine'

[kiennt23]

Run all the step in this tutorial
https://packaging.python.org/tutorials/packaging-projects/, but when run the step

twine upload --repository-url https://test.pypi.org/legacy/ dist/*

It throw this error

twine upload --repository-url https://test.pypi.org/legacy/ dist/*
Traceback (most recent call last):
  File "/Users/kiennguyen/Library/Python/3.6/bin/twine", line 7, in <module>
    from twine.__main__ import main
ModuleNotFoundError: No module named 'twine'

[sigmavirus24]

Hi there @kiennt23!

Thanks for letting us know about this issue! I think this may be a problem with your environment, not with twine. It seems as if your environment doesn't know how to find user-installed modules. I wonder if you could share the output of

echo "PYTHONPATH=$PYTHONPATH"

[calocan]

I had the same problem. It turns out that $PYTHONPATH needs to have the current directory in it to support twine's relative imports. So putting:

export PYTHONPATH=.:my_other_dirs

in ~/.bash_profile or equivalent fixes it.

[sigmavirus24]

@calocan how did you install twine? I can't imagine why this is the case

[calocan]

@sigmavirus24 I followed the instructions at https://packaging.python.org/tutorials/packaging-projects/. Python was installed on OSX with brew and I used to python3 -m venv avirtualenv to create a virtual environment in which to install and run twine.

[sigmavirus24]

As virtualenvs are no in that document, I still do not have enough information to attempt to reproduce this. I can use twine from a virtualenv without issue. I want to help you, but you have to help me help you

[scottclowe]

I also had this issue.

The problem is following the instruction

python3 -m pip install --user --upgrade twine

without reading the command closely.

This (rightly) installs the twine package into the venv, but (wrongly) puts a copy of the executable in ./local/bin, which (rightly) isn't on the path while the venv is activated.

There is a warning message about this in the pip output:

  Downloading https://files.pythonhosted.org/packages/f4/24/2a3e3df732393fed8b3ebf2ec078f05546de641fe1b667ee316ec1dcf3b7/webencodings-0.5.1-py2.py3-none-any.whl
Installing collected packages: pkginfo, chardet, idna, certifi, urllib3, requests, requests-toolbelt, six, docutils, Pygments, webencodings, bleach, readme-renderer, tqdm, twine
  The script pkginfo is installed in '~/.local/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
  The script chardetect is installed in '~/.local/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
  The script pygmentize is installed in '~/.local/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
  The script tqdm is installed in '~/.local/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
  The script twine is installed in '~/.local/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Successfully installed Pygments-2.3.1 bleach-3.1.0 certifi-2018.11.29 chardet-3.0.4 docutils-0.14 idna-2.8 pkginfo-1.5.0.1 readme-renderer-24.0 requests-2.21.0 requests-toolbelt-0.9.1 six-1.12.0 tqdm-4.31.1 twine-1.13.0 urllib3-1.24.1 webencodings-0.5.1

(Where ~ is replaced with the literal path to the home directory.)
But it is just a warning amongst a large amount of pip output and can easily be missed or overlooked.

For completeness, I will mention I tried to fix this by copying over the executable, without avail.

cp ~/.local/bin/twine path/to/venv/bin/

The solution was to remove the --user flag so pip put the executable in the correct place.

python3 -m pip install --upgrade twine

Note that you don't get this problem with virtualenv. If you try to install into a venv with the user flag:

virtualenv ~/venvs/tmp_venv3 -p /usr/bin/python3
source ~/venvs/tmp_venv3/bin/activate
python3 -m pip install --user --upgrade twine

You get an error message, and nothing is installed.

Can not perform a '--user' install. User site-packages are not visible in this virtualenv.

This makes it much easier to realise the error has occurred and remove the unwanted --user flag and try again.

These are both on Python 3.5 with the latest release of pip (19.0.2).

I would say the problem lies either with venv, which should copy the behaviour of virtualenv and refuse to let pip operate inside a venv with a --user flag, or possibly with pip.

However, if you want to make the PyPI documentation more complete, you could update it to say:
Install twine globally with

python3 -m pip install --user --upgrade twine

if you are installing it into a venv, omit the --user flag.

Or something along those lines.

[sigmavirus24]

So I'm confused as to why y'all are running into this. In searching the guide, I can't find instructions for a virtualenv until this point. Regardless, it's clear that it's not entirely clear to everyone what they should be doing so I'd suggest sending some changes to the guide to help other people avoid this same mistake.

[scottclowe]

For myself, I could not perform the preceeding step

python3 -m pip install --user --upgrade setuptools wheel

in my usual environment because my installed version of tensorflow has an inflexible requirement on its version of setuptools.

So I span up a venv to work out of.

[daudn]

I cracked the code! Simply run the command with sudo.
sudo twine upload --repository-url https://test.pypi.org/legacy/ dist/*

[bhrutledge]

I discovered this issue via @daudn's comment. It sounds like @scottclowe identified an issue with the tutorial, rather than Twine. Maybe that should be reported/fixed at https://github.com/pypa/packaging.python.org/issues (if it hasn't already).

For future reference: pipx is a handy tool for installing Python command-line tools in a dedicated virtual environment, with the executable on your $PATH. For example, without an active virtual environment:

$ pipx install twine
  installed package twine 1.13.0, Python 3.7.3
  These binaries are now globally available
    - twine
done! ✨ 🌟 ✨

$ ls -l $(which twine)
/Users/brian/.local/bin/twine -> /Users/brian/.local/pipx/venvs/twine/bin/twine

[sigmavirus24]

To be entirely clear, twine works inside virtual environments. That's how I use twine every single time. I'm still baffled about how y'all are running into these issues installing it in a virtualenv. I'd love more detail to see if there is a bug here or a way to improve the documentation for the guide.

Regardless, @bhrutledge's pipx recommendation seems excellent and I've now learned of a new awesome tool. Thanks Brian

[bhrutledge]

My read (not verified) of @scottclowe's comment is that python3 -m pip install --user twine doesn't work as desired in a virtual environment. The tutorial seems to assume that the reader won't be using a virtual environment, but some folks might be, in which case they could get tripped up by the --user flag.

[scottclowe]

My read (not verified) of @scottclowe's comment is that python3 -m pip install --user twine doesn't work as desired in a virtual environment. The tutorial seems to assume that the reader won't be using a virtual environment, but some folks might be, in which case they could get tripped up by the --user flag.

Yes @bhrutledge, that's right. It isn't a problem with twine but with the tutorial for advising the use of the --user flag without mentioning that this should be omitted if you are inside a virtual environment.

If one is inside a virtual environment created with the venv module (introduced to Python's standard library in version 3.3) and runs the command

python3 -m pip install --user --upgrade twine

it will give a warning message. But it will continue to try to do what you've asked for and produce this peculiar end result in which the twine package is installed to venv/lib/pythonX.Y/site-packages/twine but the executable is at ~/.local/bin/twine.

Then if you try to use the executable, you get a ModuleNotFoundError (the title of the issue). This is because (without a twine executable in your environment) the shell is finding your user's twine executable and using this. But you don't have the twine package installed for the user, only inside the environment.

Note that if you do have twine installed for the user, you won't get ModuleNotFoundError but may get unexpected side-effects if the twine version installed for the user doesn't match the version you had just installed into your virtual environment. This may be why @sigmavirus24 was not able to replicate the issue.

Also, note that this problem doesn't arise if you are inside a virtual environment created with virtualenv instead of with the venv module. In this situation you get an error instead of a warning, and pip refuses to install anything until you either remove the --user flag, or you leave the virtual environment.

The problem really is down to pip and venv misbehaving, and is not the fault of twine at all. The documented behaviour for pip here states:

Note that the --user flag has no effect when inside a virtual environment - all installation commands will affect the virtual environment.

which suggests the --user flag is supposed to be ignored inside a venv (but this isn't what happens).

Meanwhile, here it says:

3. pip will not perform a --user install in a --no-site-packages virtualenv (i.e. the default kind of virtualenv), due to the user site not being on the python path. The installation would be pointless.

which suggests the --user flag inside a venv causes nothing to happen (an error message). This is the behaviour when inside a virtual environment created with virtualenv, but not the venv module.

[scottclowe]

I cracked the code! Simply run the command with sudo.
sudo twine upload --repository-url https://test.pypi.org/legacy/ dist/*

@daudn, twine does not need sudo permissions. So you shouldn't need to run twine with sudo.

If you find that you do need to use sudo, it will be because you have incorrectly installed twine into your global python packages (using the incorrect command sudo pip install twine) instead of into your user packages (achieved by passing the --user flag to pip with pip install --user twine) or into a virtual environment (achieved by being inside a virtual environment and then running pip install twine).

[sigmavirus24]

Aha. I had no clue the --user flag did that. That's super confusing and explains everyone's confusion here as well as my confusion about how they got their systems into such a state. That's mind boggling. Thanks for clearing that up @bhrutledge and @scottclowe. Yeah, the packaging docs definitely need to be clearer about that. Thanks for taking the time to beat the explanation into my head :)

[Zachary4biz]

Hope this can help.
In my case, I installed twine in a virtual environment (anaconda). The system-python has no package as twine, thus install twine in system python works for my case.
(verify this by vi /usr/local/bin/twine. first line is indicating which python twine is using)

# return to basic system environment
conda deactivate 
pip install twine