Windows Installation Issue

[william-silversmith]

Hi again! Sorry to bug you. I was compiling a module against Windows 64-bit with Python 3.6, 3.7, and 3.8 on AppVeyor that includes deflate. Deflate appeared to successfully install, but was not found as a module. Here's my output:

uild started
git clone -q --branch=main https://github.com/seung-lab/mapbuffer.git C:\projects\mapbuffer
git checkout -qf 8d71424c63cb13c9babbdf307ff2d2ca6e1e4d31
Running Install scripts
%PYTHON%\python.exe -m pip install wheel
Collecting wheel
  Downloading https://files.pythonhosted.org/packages/c9/0b/e0fd299d93cd9331657f415085a4956422959897b333e3791dde40bd711d/wheel-0.36.1-py2.py3-none-any.whl
Installing collected packages: wheel
  WARNING: The script wheel.exe is installed in 'C:\Python38-x64\Scripts' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Successfully installed wheel-0.36.1
WARNING: You are using pip version 19.3.1; however, version 20.3.1 is available.
You should consider upgrading via the 'python -m pip install --upgrade pip' command.
%PYTHON%\python.exe -m pip install -r requirements.txt -r requirements_dev.txt
Collecting brotli
  Downloading https://files.pythonhosted.org/packages/63/29/1b104b5915e61d9f7443889657d93937b7e0b33b331609b82693547934a0/Brotli-1.0.9-cp38-cp38-win_amd64.whl (365kB)
Collecting deflate>=0.2.0
  Downloading https://files.pythonhosted.org/packages/c2/42/49d6f7eecca4768e045b5bf25c8b7f650ffd88b5c0d927c5833161c496c2/deflate-0.2.0-cp38-cp38-win_amd64.whl
Collecting numpy
  Downloading https://files.pythonhosted.org/packages/40/db/5060f18b0116f00ee73f8365efc9c95bd5496946290b0e7c97b6ee89dffe/numpy-1.19.4-cp38-cp38-win_amd64.whl (13.0MB)
Collecting tqdm
  Downloading https://files.pythonhosted.org/packages/8a/54/115f0c28a61d56674c3a5e05c46d6c3523ad196e1dcd3e2d8b119026df36/tqdm-4.54.1-py2.py3-none-any.whl (69kB)
Collecting zstandard
  Downloading https://files.pythonhosted.org/packages/1d/60/888086d0239520c087da33ca2a2f1c185b24ce1d882ba68abbd8787c7884/zstandard-0.14.1-cp38-cp38-win_amd64.whl (551kB)
Collecting pytest
  Downloading https://files.pythonhosted.org/packages/b1/ee/53945d50284906adb1e613fabf2e1b8b25926e8676854bb25b93564c0ce7/pytest-6.1.2-py3-none-any.whl (272kB)
Collecting cffi>=1.11
  Downloading https://files.pythonhosted.org/packages/f6/d4/bf5516922cc914174a936e7094d5f136ee4178b3509069dd3da8e3040a1a/cffi-1.14.4-cp38-cp38-win_amd64.whl (179kB)
Collecting attrs>=17.4.0
  Downloading https://files.pythonhosted.org/packages/c3/aa/cb45262569fcc047bf070b5de61813724d6726db83259222cd7b4c79821a/attrs-20.3.0-py2.py3-none-any.whl (49kB)
Collecting iniconfig
  Downloading https://files.pythonhosted.org/packages/9b/dd/b3c12c6d707058fa947864b67f0c4e0c39ef8610988d7baea9578f3c48f3/iniconfig-1.1.1-py2.py3-none-any.whl
Collecting toml
  Downloading https://files.pythonhosted.org/packages/44/6f/7120676b6d73228c96e17f1f794d8ab046fc910d781c8d151120c3f1569e/toml-0.10.2-py2.py3-none-any.whl
Collecting packaging
  Downloading https://files.pythonhosted.org/packages/28/87/8edcf555adaf60d053ead881bc056079e29319b643ca710339ce84413136/packaging-20.7-py2.py3-none-any.whl
Collecting colorama; sys_platform == "win32"
  Downloading https://files.pythonhosted.org/packages/44/98/5b86278fbbf250d239ae0ecb724f8572af1c91f4a11edf4d36a206189440/colorama-0.4.4-py2.py3-none-any.whl
Collecting atomicwrites>=1.0; sys_platform == "win32"
  Downloading https://files.pythonhosted.org/packages/2c/a0/da5f49008ec6e9a658dbf5d7310a4debd397bce0b4db03cf8a410066bb87/atomicwrites-1.4.0-py2.py3-none-any.whl
Collecting py>=1.8.2
  Downloading https://files.pythonhosted.org/packages/68/0f/41a43535b52a81e4f29e420a151032d26f08b62206840c48d14b70e53376/py-1.9.0-py2.py3-none-any.whl (99kB)
Collecting pluggy<1.0,>=0.12
  Downloading https://files.pythonhosted.org/packages/a0/28/85c7aa31b80d150b772fbe4a229487bc6644da9ccb7e427dd8cc60cb8a62/pluggy-0.13.1-py2.py3-none-any.whl
Collecting pycparser
  Downloading https://files.pythonhosted.org/packages/ae/e7/d9c3a176ca4b02024debf82342dab36efadfc5776f9c8db077e8f6e71821/pycparser-2.20-py2.py3-none-any.whl (112kB)
Collecting pyparsing>=2.0.2
  Downloading https://files.pythonhosted.org/packages/8a/bb/488841f56197b13700afd5658fc279a2025a39e22449b7cf29864669b15d/pyparsing-2.4.7-py2.py3-none-any.whl (67kB)
Installing collected packages: brotli, deflate, numpy, tqdm, pycparser, cffi, zstandard, attrs, iniconfig, toml, pyparsing, packaging, colorama, atomicwrites, py, pluggy, pytest
  WARNING: The script f2py.exe is installed in 'C:\Python38-x64\Scripts' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
  WARNING: The script tqdm.exe is installed in 'C:\Python38-x64\Scripts' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
  WARNING: The scripts py.test.exe and pytest.exe are installed in 'C:\Python38-x64\Scripts' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Successfully installed atomicwrites-1.4.0 attrs-20.3.0 brotli-1.0.9 cffi-1.14.4 colorama-0.4.4 deflate-0.2.0 iniconfig-1.1.1 numpy-1.19.4 packaging-20.7 pluggy-0.13.1 py-1.9.0 pycparser-2.20 pyparsing-2.4.7 pytest-6.1.2 toml-0.10.2 tqdm-4.54.1 zstandard-0.14.1
WARNING: You are using pip version 19.3.1; however, version 20.3.1 is available.
You should consider upgrading via the 'python -m pip install --upgrade pip' command.
%PYTHON%\python.exe -m pip install -e .
Obtaining file:///C:/projects/mapbuffer
Requirement already satisfied: brotli in c:\python38-x64\lib\site-packages (from mapbuffer==0.2.1.dev5) (1.0.9)
Requirement already satisfied: deflate>=0.2.0 in c:\python38-x64\lib\site-packages (from mapbuffer==0.2.1.dev5) (0.2.0)
Requirement already satisfied: numpy in c:\python38-x64\lib\site-packages (from mapbuffer==0.2.1.dev5) (1.19.4)
Requirement already satisfied: tqdm in c:\python38-x64\lib\site-packages (from mapbuffer==0.2.1.dev5) (4.54.1)
Requirement already satisfied: zstandard in c:\python38-x64\lib\site-packages (from mapbuffer==0.2.1.dev5) (0.14.1)
Requirement already satisfied: cffi>=1.11 in c:\python38-x64\lib\site-packages (from zstandard->mapbuffer==0.2.1.dev5) (1.14.4)
Requirement already satisfied: pycparser in c:\python38-x64\lib\site-packages (from cffi>=1.11->zstandard->mapbuffer==0.2.1.dev5) (2.20)
Installing collected packages: mapbuffer
  Found existing installation: mapbuffer 0.2.1.dev5
    Can't uninstall 'mapbuffer'. No files were found to uninstall.
  Running setup.py develop for mapbuffer
Successfully installed mapbuffer
WARNING: You are using pip version 19.3.1; however, version 20.3.1 is available.
You should consider upgrading via the 'python -m pip install --upgrade pip' command.
%PYTHON%\python.exe -m pytest -v -x automated_test.py
============================= test session starts =============================
platform win32 -- Python 3.8.0, pytest-6.1.2, py-1.9.0, pluggy-0.13.1 -- C:\Python38-x64\python.exe
cachedir: .pytest_cache
rootdir: C:\projects\mapbuffer
collecting ... collected 0 items / 1 error
=================================== ERRORS ====================================
_____________________ ERROR collecting automated_test.py ______________________
ImportError while importing test module 'C:\projects\mapbuffer\automated_test.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
C:\Python38-x64\lib\importlib\__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
automated_test.py:3: in <module>
    from mapbuffer import MapBuffer, HEADER_LENGTH
mapbuffer\__init__.py:30: in <module>
    from .mapbuffer import MapBuffer, HEADER_LENGTH, MAGIC_NUMBERS, FORMAT_VERSION
mapbuffer\mapbuffer.py:3: in <module>
    from . import compression
mapbuffer\compression.py:5: in <module>
    import deflate
E   ModuleNotFoundError: No module named 'deflate'
=========================== short test summary info ===========================
ERROR automated_test.py

Thanks again for deflate! It's a really great module. ^_^

My code is located here if you'd like to look: https://github.com/seung-lab/mapbuffer

I don't have access to a Windows machine so I'm not sure what else I can do.

Will

[dcwatson]

Thanks for the report. I'm guessing that maybe Windows builds the library with a different suffix (not .a) and isn't being included in the package. Should probably be an easy conditional platform.system() == "Windows" in setup.py, but I don't know for sure until I can get a Windows system set up to try it.

I do have a Windows machine laying around (in pieces), so I could theoretically make this happen, but it might be easier to fiddle/guess and use GItHub actions until something works.

[dcwatson]

Can you give the Windows wheels from the artifacts here a try? I don't know why I was trying so hard to use libdeflate's build system instead of letting distutils/setuptools just compile it.

[dcwatson]

I optimistically released 0.3 to PyPI, with Windows wheels that I've built and run on an actual Windows machine. Let me know if you still have any trouble.

[william-silversmith]

That seems to work for my build system!