Not all required packages installed on Windows from pypi

[cdgriffith]

Installed this from pypi and seems some packages are missing by default. Needed to install windows-curses and fonttools to get it going.

ModuleNotFoundError: No module named '_curses'
...
ModuleNotFoundError: No module named 'fontTools'

[jorisschellekens]

Hi there,

Thanks for your feedback.
I love these little opportunities to improve pText.

I did notice the missing dependency.
I must have forgotten to update the setup script after I did the font-refactor (that refactor introduced ttf fonts, but needs the fonttools dependency).

I'll fix it in the next release.

Again, my sincere thanks for your feedback.

Kind regards,
Joris Schellekens

[jorisschellekens]

I just checked my virtual environment. I can't see any import that looks like "curses".
Perhaps this is a transient import from something?
The only thing I can think of is the argparse library. (I recently added some easy command line options, so you can just call ptext to quickly convert markdown, or render some text to a pdf, etc)

I did add fonttools to the setup.py script.

Kind regards,
Joris Schellekens

[cdgriffith]

I am using Python 3.8 x64 on Windows and created new venv tried installing fonttools first manually to see if it was a import from that, but still getting the error:

Traceback (most recent call last):
  File "C:/Users/Chris/PycharmProjects/scanfix/create_pdf_bad.py", line 7, in <module>
    from ptext.pdf.pdf import PDF
  File "C:\Users\Chris\PycharmProjects\scanfix\venv\lib\site-packages\ptext\pdf\pdf.py", line 14, in <module>
    from ptext.io.read.read_any_object_transformer import ReadAnyObjectTransformer
  File "C:\Users\Chris\PycharmProjects\scanfix\venv\lib\site-packages\ptext\io\read\read_any_object_transformer.py", line 38, in <module>
    from ptext.io.read.page.read_page_dictionary_transformer import (
  File "C:\Users\Chris\PycharmProjects\scanfix\venv\lib\site-packages\ptext\io\read\page\read_page_dictionary_transformer.py", line 17, in <module>
    from ptext.pdf.canvas.canvas_stream_processor import CanvasStreamProcessor
  File "C:\Users\Chris\PycharmProjects\scanfix\venv\lib\site-packages\ptext\pdf\canvas\canvas_stream_processor.py", line 87, in <module>
    from ptext.pdf.canvas.operator.text.show_text import ShowText
  File "C:\Users\Chris\PycharmProjects\scanfix\venv\lib\site-packages\ptext\pdf\canvas\operator\text\show_text.py", line 11, in <module>
    from ptext.pdf.canvas.event.chunk_of_text_render_event import ChunkOfTextRenderEvent
  File "C:\Users\Chris\PycharmProjects\scanfix\venv\lib\site-packages\ptext\pdf\canvas\event\chunk_of_text_render_event.py", line 14, in <module>
    from ptext.pdf.canvas.font.glyph_line import GlyphLine
  File "C:\Users\Chris\PycharmProjects\scanfix\venv\lib\site-packages\ptext\pdf\canvas\font\glyph_line.py", line 10, in <module>
    from curses.ascii import isspace
  File "C:\Program Files\Python38\lib\curses\__init__.py", line 13, in <module>
    from _curses import *
ModuleNotFoundError: No module named '_curses'

You might have curses installed in your global space if the venv is set to inherent? I think it comes with python on linux by default, so maybe just a windows thing as well.

[jorisschellekens]

I just executed pip list outside my virtual env.

Package                              Version              Location
------------------------------------ -------------------- -----------------------------------------
aiocontextvars                       0.2.2
aioredis                             1.3.1
alabaster                            0.7.8
ansible                              2.9.6
ansicolors                           1.1.8
apache-libcloud                      2.8.0
appdirs                              1.4.4
apt-xapian-index                     0.49
argcomplete                          1.8.1
asgiref                              3.3.4
astroid                              2.3.3
async-timeout                        3.0.1
attrs                                21.2.0
autobahn                             21.3.1
Automat                              20.2.0
Babel                                2.9.1
backports.tempfile                   1.0
backports.weakref                    1.0.post1
bcrypt                               3.2.0
beautifulsoup4                       4.9.3
black                                21.6b0
bleach                               3.3.0
blinker                              1.4
cached-property                      1.5.2
cachetools                           4.2.2
certifi                              2020.12.5
cffi                                 1.14.5
cfgv                                 3.2.0
channels                             3.0.3
channels-redis                       3.2.0
chardet                              4.0.0
clamd                                1.0.2
click                                7.1.2
colorama                             0.4.4
command-not-found                    0.3
constantly                           15.1.0
coverage                             5.5
cryptography                         3.4.7
cssselect                            1.1.0
cssutils                             2.2.0
cupshelpers                          1.0
cycler                               0.10.0
daphne                               3.0.2
dbus-python                          1.2.16
defusedxml                           0.7.1
distlib                              0.3.1
distro                               1.4.0
distro-info                          0.23ubuntu1
Django                               3.2.3
django-admin-sortable                2.2.3
django-adminplus                     0.5
django-app-namespace-template-loader 0.4.1
django-autocomplete-light            3.8.2
django-babel                         0.6.2
django-background-tasks              1.2.5
django-bootstrap3                    11.0.0
django-bootstrap3-datetimepicker-2   2.8.2
django-clamd                         0.4.0
django-compat                        1.0.15
django-contrib-comments              1.9.2
django-cors-headers                  3.7.0
django-countries                     7.2.1
django-crispy-forms                  1.11.2
django-crum                          0.7.9
django-debug-toolbar                 3.2.1
django-dynamic-preferences           1.10.1
django-extensions                    3.1.3
django-filter                        2.4.0
django-leaflet                       0.24.0
django-localflavor                   3.0.1
django-loginas                       0.3.9
django-model-utils                   4.1.1
django-modeltranslation              0.17.1
django-pipeline                      2.0.6
django-post-office                   3.5.3
django-private-storage               2.2.2
django-redis                         4.12.1
django-resized                       0.3.11
django-sendfile                      0.3.11
django-tables2                       2.3.4
django-test-without-migrations       0.6
django-uuidfield                     0.5.0
django-webpack-loader                1.0.0
django-websocket-redis               0.6.0
djangorestframework                  3.12.4
dnspython                            1.16.0
docutils                             0.16
drf-nested-routers                   0.93.3
enmerkar                             0.7.1
entrypoints                            0.3
exactonline                          0.3.5
fakeredis                            1.5.2
fcm-django                           0.3.10
filelock                             3.0.12
filemagic                            1.6
fonttools                            4.22.1
freezegun                            1.1.0
GDAL                                 3.2.1
geographiclib                        1.50
geopy                                2.1.0
gevent                               21.1.2
google-api-core                      1.26.3
google-auth                          1.30.0
google-cloud-bigquery                2.16.1
google-cloud-core                    1.6.0
google-crc32c                        1.1.2
google-resumable-media               1.2.0
googleapis-common-protos             1.53.0
googlemaps                           4.4.5
gpg                                  1.13.1-unknown
gprof2dot                            2021.2.21
greenlet                             1.1.0
grpcio                               1.37.1
gTTS                                 2.2.2
gunicorn                             20.1.0
hiredis                              2.0.0
html5lib                             1.1
httplib2                             0.14.0
hyperlink                            21.0.0
identify                             1.5.5
idna                                 2.10
imagesize                            1.2.0
incremental                          21.3.0
isodate                              0.6.0
isort                                4.3.4
Jinja2                               3.0.0
jmespath                             0.9.4
jsonfield                            3.1.0
jsonfield2                           3.0.3
kazam                                1.4.5
keyring                              18.0.1
kiwisolver                           1.3.1
language-selector                    0.1
launchpadlib                         1.10.13
lazr.restfulclient                   0.14.2
lazr.uri                             1.0.3
lazy-object-proxy                    1.4.3
lockfile                             0.12.2
logilab-common                       1.4.3
lxml                                 4.6.3
Markdown                             3.3.4
MarkupSafe                           2.0.0
matplotlib                           3.3.4
mccabe                               0.6.1
msgpack                              1.0.2
mypy                                 0.761
mypy-extensions                      0.4.3
netaddr                              0.7.19
netifaces                            0.10.4
nodeenv                              1.5.0
ntlm-auth                            1.1.0
numpy                                1.17.4
oauthlib                             3.1.0
olefile                              0.46
overrides                            6.1.0
packaging                            20.9
pandas                               1.2.2
pathspec                             0.8.1
pdfminer.six                         20201018
pep517                               0.10.0
persisting-theory                    0.2.1
pexpect                              4.6.0
phonenumbers                         8.12.23
pika                                 1.2.0
pikepdf                              2.12.0
Pillow                               8.2.0
pip                                  21.0.1
pip-tools                            6.1.0
pre-commit                           2.7.1
premailer                            3.8.0
proto-plus                           1.18.1
protobuf                             3.17.0
psutil                               5.5.1
psycopg2                             2.8.6
pyasn1                               0.4.8
pyasn1-modules                       0.2.8
pycairo                              1.16.2
pycparser                            2.20
pycrypto                             2.6.1
pycups                               1.9.73
pydantic                             1.8.2
PyExcelerate                         0.10.0
pyfcm                                1.4.9
Pygments                             2.9.0
PyGObject                            3.36.0
PyHamcrest                           2.0.2
PyJWT                                1.7.1
pykerberos                           1.1.14
pylint                               2.4.4
pymacaroons                          0.13.0
pymaging                             0.1
pymaging-png                         0.1
PyNaCl                               1.3.0
pyOpenSSL                            20.0.1
pyparsing                            2.4.7
PyPDF2                               1.26.0
PyQt5                                5.14.1
pytesseract                          0.3.7
python-apt                           2.0.0+ubuntu0.20.4.4
python-barcode                       0.13.1
python-dateutil                      2.8.1
python-debian                        0.1.36ubuntu1
python-Levenshtein                   0.12.0
python-stdnum                        1.16
pytz                                 2021.1
pywinrm                              0.3.0
pyxdg                                0.26
PyYAML                               5.3.1
qrcode                               6.1
redis                                3.5.3
regex                                2020.7.14
reportlab                            3.5.67
requests                             2.25.1
requests-file                        1.5.1
requests-kerberos                    0.12.0
requests-mock                        1.9.3
requests-ntlm                        1.1.0
requests-toolbelt                    0.9.1
requests-unixsocket                  0.2.0
roman                                2.0.0
rsa                                  4.7.2
SecretStorage                        2.3.1
selenium                             3.141.0
selinux                              3.0
sentry-sdk                           1.1.0
service-identity                     21.1.0
setuptools                           51.1.2
simplejson                           3.17.2
sip                                  4.19.21
six                                  1.16.0
snakeviz                             2.1.0
sortedcontainers                     2.4.0
soupsieve                            2.2.1
sqlparse                             0.4.1
stripe                               2.56.0
systemd-python                       234
tblib                                1.7.0
text-unidecode                       1.3
toml                                 0.10.1
tornado                              6.1
Twisted                              21.2.0
txaio                                21.2.1
typed-ast                            1.4.1
typing-extensions                    3.10.0.0
typing-utils                         0.0.3
ubuntu-advantage-tools               20.3
ubuntu-drivers-common                0.0.0
ufw                                  0.36
unattended-upgrades                  0.1
unittest-xml-reporting               3.0.4
urllib3                              1.26.4
usb-creator                          0.3.7
vat-format-checker                   0.0.4
virtualenv                           20.0.31
wadllib                              1.3.3
webencodings                         0.5.1
websockets                           9.1
wheel                                0.35.1
wrapt                                1.11.2
xkit                                 0.0.0
XlsxWriter                           1.4.3
xmltodict                            0.12.0
z3c.rml                              4.1.2
zeep                                 4.0.0
zope.event                           4.5.0
zope.interface                       5.4.0
zope.schema                          6.1.0

[cdgriffith]

Intersting, I wounder if something there is acting in it's place. Mine is tiny it seems:

Package           Version
----------------- --------
appdirs           1.4.4
cfgv              3.2.0
distlib           0.3.1
filelock          3.0.12
identify          2.2.4
mypy              0.812
mypy-extensions   0.4.3
nodeenv           1.6.0
pip               20.2.3
pre-commit        2.12.1
python-box        5.3.0
PyYAML            5.4.1
ruamel.yaml       0.17.7
ruamel.yaml.clib  0.2.2
setuptools        49.2.1
six               1.16.0
toml              0.10.2
typed-ast         1.4.3
typing-extensions 3.10.0.0
virtualenv        20.4.6

[jorisschellekens]

Yeah, but as you can see, no "_curses" in mine.

[cdgriffith]

Doesn't really matter what the names of the packages are sadly, as different things can be renamed (yours itself is ptext_joris_schellekens in pip but imported as ptext), and can be totally different like pillow you import PIL and so on.

You may be able to do a search through the site-packages directory in each *.info sub dirs top_level.txt file for the _curses keyword.

[jorisschellekens]

I'm afraid that didn't yield much either.

joris@dell:~/.local/lib/python3.8/site-packages$ grep -Ril "_curses" .
joris@dell:~/.local/lib/python3.8/site-packages$

(command just returns nothing)

[cdgriffith]

Wait, are you on linux or Windows (just the previous command output looks a lot more Linuxy)?

curses is included with Linux Python. Hence specifying this was a Windows issue (aka might have to do a detect in the setup.py for windows and append windows-curses to the requirements list)

[jorisschellekens]

Definitely on Linux.
I have to say I'm no great shakes at this setup.py thing.
But I'll have a look later.

[jorisschellekens]

I changed part of setup.py to:

# open requirements
required = (
    [
        "fonttools>=4.22.1",  # TTF fonts
        "Pillow>=7.1.0",  # image processing
        "python-barcode>=0.13.1",  # generating barcodes
        "qrcode[pil]>=6.1",  # generating QR codes
        "requests>=2.24.0"  # generating images from a URL
        "setuptools~=51.1.1",  # standard
    ]
    + ["windows-curses>=2.2.0"]
    if sys.platform.startswith("win")
    else []
)

That should do the trick, although I can't verify it (I don't have any windows laptops in my life 😄 ).
So I'm going to leave this ticket open until you can verify it, if that's ok with you.

[cdgriffith]

ouh be careful with that inline if, def need parens around (["windows-curses>=2.2.0"] if sys.platform.startswith("win") else [])

Example why:

>>> [1] + [1] if True else [2]
[1, 1]
>>> [1] + [1] if False else [2]
[2]
>>> [1] + ([1] if False else [2])
[1, 2]

[jorisschellekens]

# open requirements
required = [
    "fonttools>=4.22.1",  # TTF fonts
    "Pillow>=7.1.0",  # image processing
    "python-barcode>=0.13.1",  # generating barcodes
    "qrcode[pil]>=6.1",  # generating QR codes
    "requests>=2.24.0" "setuptools~=51.1.1",  # generating images from a URL  # standard
] + (["windows-curses>=2.2.0"] if sys.platform.startswith("win") else [])

[jorisschellekens]

Hi,

Just wanted to let you know the new release is out.
And it should have the setup.py fix in it.

Kind regards,
Joris Schellekens