cannot install from pip

[12rambau]

I tried to install the lib from pip using the simple pip install pydsstools command but i'm facing the following error:

      Building external library: grid.a
      building 'pydsstools._lib.x64.core_heclib' extension
      creating build/temp.linux-x86_64-cpython-39
      creating build/temp.linux-x86_64-cpython-39/pydsstools
      creating build/temp.linux-x86_64-cpython-39/pydsstools/src
      x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -Ipydsstools/src/external/dss/headers -Ipydsstools/src/external/gridv6/headers -I/tmp/pip-build-env-ol686mme/overlay/lib/python3.9/site-packages/numpy/core/include -I/mnt/e/WSL/Users/rambap/azure/LDCRiverLevelMonitor/.nox/test/include -I/usr/include/python3.9 -c pydsstools/src/core_heclib.c -o build/temp.linux-x86_64-cpython-39/pydsstools/src/core_heclib.o
      pydsstools/src/core_heclib.c:44:10: fatal error: Python.h: No such file or directory
         44 | #include "Python.h"
            |          ^~~~~~~~~~
      compilation terminated.
      error: command '/usr/bin/x86_64-linux-gnu-gcc' failed with exit code 1
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for pydsstools
Successfully built LDCRiverlevelMonitor
Failed to build pydsstools
ERROR: Could not build wheels for pydsstools, which is required to install pyproject.toml-based projects

It's been way too long that i've dropped compiled languages so I don't exactly understand what is going on here.
I'm coding in a ubuntu machine using the WLS. To make sure my environement is well isolated from the rest I created a specific conda environment:

conda create --name=riverlevel python=3.10

from this environement I simply run:

pip install pydsstools

What other information would you require to make a reproducible example ?

[mxkpp]

I'll chime in to reference #45 which was the last time I attempted (successfully) to install pydsstools in a Linux environment. The main problem I had at the time was that USACE's heclib binary (a dependency for pydsstools) was corrupted, so I needed to download and point to an alternate version of that binary. As mentioned in the issue, since then I have stayed on an old commit of pydsstools. It could be that things are smoother now, but I didn't want to take the risk of updating to the latest due to this: #43 which references a function rename here: HydrologicEngineeringCenter/hec-dss#165.

I don't want to dissuade you from trying the latest, I just wanted to give a complete answer since this is pretty nuanced. If you want a method that 100% works, see my issue #45 which calls out a specific commit to use, and specific heclib binary to use.

[gyanz]

@12rambau The compiler can't find the Python header file, which is unrelated to this library. Ensure your conda environment has access to the header files and static libraries of Python. You may need to install python3-dev or the equivalent for your Linux installation.

[12rambau]

I did several time, the machine is a bit weird I think you are searching for the exact Python header that you would expect on a classic Linux machine but for exotic machine like mine it doesn't work. Where do you expect Python.h to be ?

[mxkpp]

@12rambau you can probably ignore my earlier comment since as @gyanz pointed out, the problem is at the highest level (the environment) rather than among some dependency chain specific to this library. Conda makes things hard.

[gyanz]

You are supposedly using a Python 3.10 environment in Conda; however, the compiler indicates you are using Python 3.9, which is likely the system Python. I recommend using the system or non-Conda Python through a virtual environment (using venv, virtualenv, etc.) for compiling pydsstools to avoid accidentally messing up the system Python and potentially breaking the OS. As I mentioned earlier, you may need to install the python3-dev and additional development packages via apt install package_name.

[12rambau]

I'm on purpose installing version 3.9 to matchthe databricks environment of my company. It's too error pronefor us to rely on a setup.py file that compiles a lib upon installation. Isn't it possible from your side to compile it when you make a new release of the lib ?
The same way GDAL is compiled by rasterio for example ? This way you always controls what is installed with the python package and 100% sure things are actually compatible.

[gyanz]

Binary wheels can't be made available for Linux via pip due to the guidelines outlined in PEP 513.

[12rambau]

ok then I'll stop bother you, my company machine have a very exotic setup (that I'm complaining about to the IT services since day 1) that prevent any installation that is not a python wheel to run smoothly.

Since I started Conda have been a life saver has it supports compilation of other tools but if you don't want to support it, I will not be able to proceed.

Feel free to close my issue if it's a won't fix and/or ping me if you need a beta tester for a potential conda release.

[gyanz]

It is not up to me to support or not support wheels for Linux; PyPi/pip do not support Linux wheels. For this reason, I am closing this issue.