Strange install behavior - pip always installs unidecode

[tedmiston]

I'm trying to test installing python-slugify with unidecode vs text-unidecode for Airflow, however, for some reason I'm getting the unidecode package installed by pip always even when passing the explicit flag for text-unidecode. I can see in the setup.py that this shouldn't happen, so I'm not really sure how it is.

Can you replicate what I'm experiencing below?

Note: I'm using pyenv and virtualenvwrapper commands.

Case 1 - Without SLUGIFY_USES_TEXT_UNIDECODE (expected unidecode)

t-mbp:~ taylor$ pyenv shell 3.6.5 && mktmpenv -i python-slugify
Using base prefix '/Users/taylor/.pyenv/versions/3.6.5'
New python executable in /Users/taylor/.local/share/virtualenvs/tmp-63bc530c9c5a5f1/bin/python3.6
Also creating executable in /Users/taylor/.local/share/virtualenvs/tmp-63bc530c9c5a5f1/bin/python
Installing setuptools, pip, wheel...done.
virtualenvwrapper.user_scripts creating /Users/taylor/.virtualenvs/tmp-63bc530c9c5a5f1/bin/predeactivate
virtualenvwrapper.user_scripts creating /Users/taylor/.virtualenvs/tmp-63bc530c9c5a5f1/bin/postdeactivate
virtualenvwrapper.user_scripts creating /Users/taylor/.virtualenvs/tmp-63bc530c9c5a5f1/bin/preactivate
virtualenvwrapper.user_scripts creating /Users/taylor/.virtualenvs/tmp-63bc530c9c5a5f1/bin/postactivate
virtualenvwrapper.user_scripts creating /Users/taylor/.virtualenvs/tmp-63bc530c9c5a5f1/bin/get_env_details
Collecting python-slugify
Collecting Unidecode>=0.04.16 (from python-slugify)
  Using cached https://files.pythonhosted.org/packages/59/ef/67085e30e8bbcdd76e2f0a4ad8151c13a2c5bce77c85f8cad6e1f16fb141/Unidecode-1.0.22-py2.py3-none-any.whl
Installing collected packages: Unidecode, python-slugify
Successfully installed Unidecode-1.0.22 python-slugify-1.2.5
This is a temporary environment. It will be deleted when you run 'deactivate'.

Case 2 - With SLUGIFY_USES_TEXT_UNIDECODE=yes (expected text-unidecode)

t-mbp:~ taylor$ pyenv shell 3.6.5 && export SLUGIFY_USES_TEXT_UNIDECODE=yes && mktmpenv -i python-slugify
Using base prefix '/Users/taylor/.pyenv/versions/3.6.5'
New python executable in /Users/taylor/.local/share/virtualenvs/tmp-6e020af8417d51b/bin/python3.6
Also creating executable in /Users/taylor/.local/share/virtualenvs/tmp-6e020af8417d51b/bin/python
Installing setuptools, pip, wheel...done.
virtualenvwrapper.user_scripts creating /Users/taylor/.virtualenvs/tmp-6e020af8417d51b/bin/predeactivate
virtualenvwrapper.user_scripts creating /Users/taylor/.virtualenvs/tmp-6e020af8417d51b/bin/postdeactivate
virtualenvwrapper.user_scripts creating /Users/taylor/.virtualenvs/tmp-6e020af8417d51b/bin/preactivate
virtualenvwrapper.user_scripts creating /Users/taylor/.virtualenvs/tmp-6e020af8417d51b/bin/postactivate
virtualenvwrapper.user_scripts creating /Users/taylor/.virtualenvs/tmp-6e020af8417d51b/bin/get_env_details
Collecting python-slugify
Collecting Unidecode>=0.04.16 (from python-slugify)
  Using cached https://files.pythonhosted.org/packages/59/ef/67085e30e8bbcdd76e2f0a4ad8151c13a2c5bce77c85f8cad6e1f16fb141/Unidecode-1.0.22-py2.py3-none-any.whl
Installing collected packages: Unidecode, python-slugify
Successfully installed Unidecode-1.0.22 python-slugify-1.2.5
This is a temporary environment. It will be deleted when you run 'deactivate'.

Case 3 - With text-unidecode pre-installed (expected text-unidecode)

Last login: Tue Aug 28 15:21:47 on ttys004
t-mbp:~ taylor$ pyenv shell 3.6.5 && export SLUGIFY_USES_TEXT_UNIDECODE=yes && mktmpenv -i text-unidecode -i python-slugify
Using base prefix '/Users/taylor/.pyenv/versions/3.6.5'
New python executable in /Users/taylor/.local/share/virtualenvs/tmp-8bec3e8a00b8e0b/bin/python3.6
Also creating executable in /Users/taylor/.local/share/virtualenvs/tmp-8bec3e8a00b8e0b/bin/python
Installing setuptools, pip, wheel...done.
virtualenvwrapper.user_scripts creating /Users/taylor/.virtualenvs/tmp-8bec3e8a00b8e0b/bin/predeactivate
virtualenvwrapper.user_scripts creating /Users/taylor/.virtualenvs/tmp-8bec3e8a00b8e0b/bin/postdeactivate
virtualenvwrapper.user_scripts creating /Users/taylor/.virtualenvs/tmp-8bec3e8a00b8e0b/bin/preactivate
virtualenvwrapper.user_scripts creating /Users/taylor/.virtualenvs/tmp-8bec3e8a00b8e0b/bin/postactivate
virtualenvwrapper.user_scripts creating /Users/taylor/.virtualenvs/tmp-8bec3e8a00b8e0b/bin/get_env_details
Collecting text-unidecode
  Using cached https://files.pythonhosted.org/packages/79/42/d717cc2b4520fb09e45b344b1b0b4e81aa672001dd128c180fabc655c341/text_unidecode-1.2-py2.py3-none-any.whl
Installing collected packages: text-unidecode
Successfully installed text-unidecode-1.2
Collecting python-slugify
Collecting Unidecode>=0.04.16 (from python-slugify)
  Using cached https://files.pythonhosted.org/packages/59/ef/67085e30e8bbcdd76e2f0a4ad8151c13a2c5bce77c85f8cad6e1f16fb141/Unidecode-1.0.22-py2.py3-none-any.whl
Installing collected packages: Unidecode, python-slugify
Successfully installed Unidecode-1.0.22 python-slugify-1.2.5
This is a temporary environment. It will be deleted when you run 'deactivate'.

[un33k]

@tedmiston The culprit may be the &&. You may need to use ; between an environment variable and the command that is using that variable.

Something like the following.

$ <set the variable here> ; <install python-slugify>

[tammoippen]

The issue still persists:

bash-3.2$ export SLUGIFY_USES_TEXT_UNIDECODE=yes ; pip install python-slugify
Collecting python-slugify
Collecting Unidecode>=0.04.16 (from python-slugify)
  Using cached https://files.pythonhosted.org/packages/59/ef/67085e30e8bbcdd76e2f0a4ad8151c13a2c5bce77c85f8cad6e1f16fb141/Unidecode-1.0.22-py2.py3-none-any.whl
Installing collected packages: Unidecode, python-slugify
Successfully installed Unidecode-1.0.22 python-slugify-1.2.6

This is because of the install_requires logic in setup.py using the environment variable SLUGIFY_USES_TEXT_UNIDECODE. When creating the distribution, the environment variable was not set and hence the generated python_slugify.egg-info/requires.txt does only contain the Unidecode dependency (see the uploaded distribution here: https://pypi.org/project/python-slugify/1.2.6/#files). It is not possible to install python-slugify with text-unidecode via pip install python-slugify, regardless of what is in the environment.

A working approach would be to define the *unidecode dependencies as extras_require in setup.py:

setup(
  ...
  extras_require={
    'gpl': ['Unidecode>=0.04.16'],
    'artistic': ['text-unidecode>=1.2'],
  },
  ...
)

Then installing would work with pip install python-slugify[gpl] or pip install python-slugify[artistic]. The downside is, that the default installation would fail with an ImportError, as neither of the *unidecode libs are installed.

[un33k]

@tammoippen @tammoippen @bolkedebruin @artwr -- Yep, it seems it keeps falling on the default option during the build. This passed the test (Travis) as it was installing slugify from the source. That is how/why to PR got merged in. :(

At this time, there is no clean PR on this.

[bolkedebruin]

I'll try to work something out that is more resilient and pythonic. Input is definitely appreciated.

[uneekvu]

I'll put a permanent fix in when I push a major revision out. It will install Unidecode by default, however, it would check for text-unidecode first. The users that think they have licensing issue with Unidecode will install text-unidecode on their own via pip etc. The rest of the folks will have all they need by just installing this package.

[un33k]

@tedmiston PR 63 addressed this. This module always prefers and attempts to install and use Unidecode as its default decode module. However, if it finds that text-unidecode is already installed, it will skip installing Unidecode and falls back on using text-unidecode.

With that said, for those who want to use text-unidecode for any reasons (including licensing), please make sure text-unidecode shows up before python-slugify in your requirement file.

[bolkedebruin]

@un33k Thanks for this! It's (hopefully) a much cleaner solution. I'll check how this works out for us.