[BUG] installation of version 0.14.8 fails

[YuriOsokin]

Describe the bug
Installation of version 0.14.8 fails with the following exception:

  Error compiling Cython file:
  ------------------------------------------------------------
  ...
  
  cpdef disable_gpu():
      global GPU
      GPU = False
  
  cdef ndarray_wrap_cpointer(void* data, numpy.npy_intp n):
       ^
  ------------------------------------------------------------

It happened on Jul. 19. Before that date the installation worked.

To Reproduce

  pip install --no-cache-dir pomegranate==0.14.8

Response time
Versions 0.14.7, 0.14.6 also fail.

[yairm210]

Cython released a major release (3.0) last week.

This changed some major things, and is almost certainly the cause of this error
https://cython.readthedocs.io/en/latest/src/userguide/migrating_to_cy30.html

The solution would be to limit Cython to disallow version 3.0, but I can't find where in the install process it specifies the Cython version :/
This may be a side-effect of requiring scipy for installation, which itself allows Cython 3.0 but it manages to compile with that - Pomegranate does not.
Solution would probably be to limit the cython version in the requires_install file

[YuriOsokin]

The installation fails with the same error with cythons:
pip install cython==0.29.35
pip install cython==0.29.36

Is there a workaround, the produciton code fails to install environments?

[YuriOsokin]

Maybe there was an update of pomegranate PYPI packages? Because if I build wheel from an old vesion, it installs correctly.

[yairm210]

Installing a specific cython version locally before build does not - unfortunately - mean that Pomegranate will use that version to build...
What version is working for you?

[YuriOsokin]

None, I am unable to install ~=.28.0, because these versions incompatiable with python 3.10.6.
Whay do you mean by installing the cython locally, I do not see any log that pomegranate installs another cython version as a requirment?

[yairm210]

Sorry, only now do I understand what's going on
Pomegranate used to use Cython to compile, the new versions don't

The Cython was required from https://github.com/jmschrei/pomegranate/blob/272039bf3fa208a9ec0f67b53f03cea280abcc1b/setup.py (in setup_requires section)

So in that file you'd need to limit the Cython version to now be 3.0... but all those versions are already deprecated :(

[YuriOsokin]

Can you explain please, I succesfully installed cython 0.29.35, so pomegranate during installation used this version. However the compilation still fails.

[ymorgenstern]

When older versions of pomegranate install themselves, they download their own set of build dependencies - as shown above in the setup_requires section - for their installation process only.

The old setup.py specified to download "cython >= 0.22.1" - but with the way that pip version resolution works, this means that it will try to take the latest version of Cython, and thus (since last week) would download Cython 3.0 which fails to compile the library :/

[YuriOsokin]

So the workaround is to download the wheel and patch the setup.py and then install from local wheel?
It breaks the installation flow.
Maybe you can fix it and re-upload the 0.14.8 package to PYPI?

[yairm210]

I can't, only the repo owner can upload packages to pypi :/
But what I tried was to fork this repo and make the change there, so you can pip install from git instead
It didn't work :( this error is still happening even with older Cython, so I'm not sure what's going on - maybe newer numpy version? Newer scipy? I have no idea but I'll keep trying

[jmschrei]

Can you download a version of Cython around 0.22 first and then try to pip install?

[yairm210]

Will do 👍🏿

[yairm210]

It would not let me pip install Cython for any version before 0.28 because of "InvalidSyntax" errors :/
Installing Cython 0.28 first did nothing - same error

[kkchau]

I was able to get an install working by pinning the cython version in build-system in pyproject.toml to enable cythonize during setup. Pinning in setup.py/*requirements.txt didn't make much of a difference. Releases <= 0.14.8 might need re-release with this?.

[yairm210]

@kkchau I see! That explains why it was failing - I was installing using Poetry, and there it probably takes the build instructions from the pyproject.toml file... Nice catch!

@jmschrei is there a chance you can merge @kkchau 's branch and release that as version 0.14.9?

[YuriOsokin]

Is there a way to re-release tag 0.14.8 with these patches?

[yairm210]

For now what I did in my pyproject.toml is:

pomegranate = { git = "https://github.com/yairm210/pomegranate", branch = "0.14.8" }

and it works :) So as a temporary solution you can use that

[tyrrellgit]

Stuck here too.

Strange thing is I can get it to work on my local windows machine (Python==3.9.12 && Cython==3.0.0) but not any other machine I have tried (all linux). They fail with this exact error.

I use v0.14.8 because it's plays nice with some tools I use over Bayesian Networks. Also it's more feature rich than v1.0.0 for now. Would be great to keep this version alive until 1.0.0 really takes over. I love the torch implementation.

[jmschrei]

I forked v0.14.8 and added the changes suggested by @kkchau. If you could confirm that this resolves the issues you're facing, I'll release it on PyPI. https://github.com/jmschrei/pomegranate/tree/v0.14.9

[kkchau]

@jmschrei Confirmed that that branch should work!

$ grep "pomegranate" 3rdparty/cnvkit/requirements.txt && pip install -r 3rdparty/cnvkit/requirements.txt
pomegranate==0.14.8
...
      Cython.Compiler.Errors.CompileError: pomegranate/utils.pyx
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

$ grep "pomegranate" 3rdparty/cnvkit/requirements.txt && pip install -r 3rdparty/cnvkit/requirements.txt
pomegranate @ git+https://github.com/jmschrei/pomegranate.git@v0.14.9
Collecting pomegranate@ git+https://github.com/jmschrei/pomegranate.git@v0.14.9
...
Building wheels for collected packages: pomegranate
  Building wheel for pomegranate (pyproject.toml) ... done
  Created wheel for pomegranate: filename=pomegranate-0.14.8-cp38-cp38-linux_x86_64.whl size=22860327 sha256=e51a8bd483c270abf8c64cf82613f65f48449cbede6dc419713f1aeb379dd5d9
  Stored in directory: /tmp/pip-ephem-wheel-cache-z_7y38yi/wheels/10/30/1f/36a54ae8a45c72c6429ba3313677fb99c6895e2baca11dd293
Successfully built pomegranate

[kkchau]

Hi @jmschrei, wanted to check in and see if you had a timeline for releasing 0.14.9?

[jmschrei]

@kkchau I just got back from some travel. Can you try pip install pomegranate==0.14.9 and let me know if that works for you?

[kkchau]

Ah sorry, yeah it works now! Thanks for the update

[jmschrei]

No problem, thanks for checking!

@tyrrellgit does 0.14.9 work for you as well?