'chardet' not appended to requirements

[rmiller-sgntr]

Hi, chardet is required in some parts of the code and is supposed to be appended to requirements with this code in setup.py:

requires = ['six', 'pycryptodome', 'sortedcontainers']
if sys.version_info >= (3, 0):
    requires.append('chardet')

Which does not work as I am getting the following error using python 3.6:

File "/home/ronen/reach/reach_venv3.6/lib/python3.6/site-packages/pdfminer/utils.py", line 13, in
import chardet # For str encoding detection in Py3
ModuleNotFoundError: No module named 'chardet'

I suggest using the following fix, add chardet in required array like so:

requires = ['six', 'pycryptodome', 'sortedcontainers', 'chardet; python_version<"3.0"']

And remove the other two lines. Setup-tools supports the python_version<"x.x" syntax.
Until fixed, chardet has to be added manually to requirements on each project.

[tataganesh]

Hi @rmiller-sgntr. Have you tried adding chardet without the version check? Does it still work on python 3.6? I will check whether it works for python 2.7, and if it does, we can do away with the version check.

[rmiller-sgntr]

Yes, it works with python3.6

[vinayak-mehta]

I'm also getting this error on Python 3.6 and 3.7 (WSL), issue referenced above. I'm not sure why this is happening. When I install pdfminer.six from source using pip (18.1) (pip install .), it installs chardet but when I try to install from PyPI (pip install pdfminer.six), it doesn't.

[rmiller-sgntr]

I seems to be a mismatch between the global pipenv lock and the actual python env used. You can look here for more details.

[vinayak-mehta]

I'm not using pipenv (don't have it installed) so that seems unrelated.

[vinayak-mehta]

@ganeshtata I checked the wheel available on the pdfminer.six PyPI page. The pdfminer.six-20181108.dist-info/METADATA file doesn't have a Requires-Dist entry for chardet (it has those entries for six, pycryptodome and sortedcontainers). Could this be the reason that chardet isn't getting installed when I do pip install pdfminer.six? (It gets installed when I clone the repo and do pip install . since it uses the setup.py in the second case I guess.)

$ pip -version
pip 18.1 from /home/vinayak/.local/share/virtualenvs/camelot-3.6/lib/python3.6/site-packages/pip (python 3.6)

[vinayak-mehta]

I'm not able to figure out the part where Travis is able to install pdfminer.six correctly using pip but I'm not able to do the same on my local machine.

[vinayak-mehta]

I'm not able to figure out the part where Travis is able to install pdfminer.six correctly using pip but I'm not able to do the same on my local machine.

So I figured this out. Looked through the logs of this Travis job (which installs dev requirements for my package). chardet is being installed with requests instead of pdfminer.six which is why the build doesn't fail.

[vinayak-mehta]

As @rmiller-sgntr suggested in the first comment:

Until fixed, chardet has to be added manually to requirements on each project.

I'm adding chardet to install_requires till this is fixed. Sorry for so many comments.

[QuLogic]

The wheel is universal (2 or 3), but the requirements are not; they differ depending on what ran setup.py. It was probably uploaded with Python 2, so the metadata does not include chardet. That's why it works from source, but not from the wheel. Encoding the version-specificity into the requirement instead of conditionally adding it should fix things.

[anacrolix]

I can confirm I got this error, and it was fixed by manually installing chardet, for Python 3.7.

[pietermarsman]

Fixed by #219