Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Python3.8 + cythonize #78

Closed
wants to merge 7 commits into from

Conversation

ashwinvis
Copy link
Contributor

Merges #76 and #69

@johanneskoester
Copy link

@kmike @superbobry any chance that this will be merged and released? We urgently need this fix for Snakemake. We are happy to help in any way if there are further showstoppers.

@sarnold
Copy link

sarnold commented Mar 16, 2020

Apparently I'm (half)blind and didn't notice this PR until after I updated the bindings for 3.8+ (and dropped python2 support) in my fork. My changes are simpler(?) but instead of messing with an old submodule commit I made it depend on libdatrie instead (currently at 0.2.12). My plan was to make a new tag (in my fork) and push new packages to a PPA and Gentoo overlay. Does that help at all?

@ashwinvis
Copy link
Contributor Author

@sarnold Sure go ahead. I am on Arch and I did something similar which has libdatrie 0.2.12-1. I simply added a symlink, instead of relying on the submodule.

❯ ls -l datrie/libdatrie
total 4
lrwxrwxrwx 1 avmo avmo 19 14.01.2020 13:28 datrie -> /usr/include/datrie/

However, I wonder how one can support Windows by this approach? Maybe you have a better idea.

@sarnold
Copy link

sarnold commented Mar 17, 2020

I tested your branch yesterday, and also made a (hopefully) one-off github release for python3 only:

https://github.com/freepn/datrie/releases/tag/0.8.1

Still I didn't manage to fix the build warnings; I had the pointer type warning fixed yesterday but didn't get it in yet, and I'm not quite sure what the right/pythonic answer is for the collections import (since py27 is special there). Got any ideas for the handling the import issue?

Also I have PPA pkgs for xenial, helpful for travis builds, er, here even:

https://launchpad.net/~nerdboy/+archive/ubuntu/embedded

@johanneskoester
Copy link

johanneskoester commented Mar 18, 2020

@sarnold @ashwinvis what do you think about a pypi release. Something like datrie-<areasonablesuffix>? This would make it very easy to get it into bioconda or conda-forge, where I need it as a snakemake dependency.

@ashwinvis
Copy link
Contributor Author

Apart from Python + Cython + packaging knowledge, I have neither the expertise on tries nor the long term motivation to be a package maintainer for datrie. I do like to see this happen, but it would be mistake if I take up this responsibility now, only to abdicate it in the future.

@ashwinvis
Copy link
Contributor Author

@sarnold I compared your branch, and as suspected you have many instance of /usr/include/libdatrie in the cythonized sources of your branch instead of ../libdatrie. This almost certainly will break in non-Linux OSs.

@johanneskoester
Copy link

Maybe I should move it under the hood of snakemake.

@johanneskoester
Copy link

johanneskoester commented Mar 18, 2020

@ashwinvis is it ok if I fork your repo, merge in your fix and release this as datrie-snakemake on pypi?

@johanneskoester
Copy link

Or, probably cleaner, I fork this original repo, and you add your PR against my fork?

@ashwinvis
Copy link
Contributor Author

Sounds good @johanneskoester!

@tacaswell
Copy link
Contributor

If you are going to require cython as a build time dependency you can drop the bundled c-files all together which will prevent a similar "the c-files are broken" issues in the future.

@tacaswell
Copy link
Contributor

@ashwinvis is it ok if I fork your repo, merge in your fix and release this as datrie-snakemake on pypi?

Between that and the 0.8.1 tag this package is getting pretty forked. Has anyone tried reaching out to @kmike or @superbobry not through github?

@johanneskoester
Copy link

johanneskoester commented Mar 24, 2020

I wasn't able to reach them via mail. From one, I did not find an email, from the other, the published mail address is not existent any more.

@tacaswell
Copy link
Contributor

ok.

Very short term, I think just the "do the cythonization" patch to setup.py will un-block conda-forge so we can get that moving again.

@johanneskoester
Copy link

@ashwinvis I am done with the setup of the fork and github actions for testing. Awaiting your PR. Thanks a lot!

@tacaswell
Copy link
Contributor

conda-forge/datrie-feedstock#5 <- CF pr

@tacaswell
Copy link
Contributor

@ashwinvis @johanneskoester @sarnold I now have commit rights on this repo and should have pypi permissions soon. Should be able to get this done and release in the next ~24 hours. Going to make another issue to make clear what my plan is and track the work.

@tacaswell tacaswell mentioned this pull request Mar 25, 2020
10 tasks
@johanneskoester
Copy link

@tacaswell datrie-snakemake is now available on pypi, with the fixes of @ashwinvis. I can create a new conda-forge recipe for it, or we keep the old and refer to datrie-snakemake in it. What do you think?

@johanneskoester
Copy link

Wait, I have just seen your release plan. Are you one of the maintainers? In any case, I am happy to remove my fork again.

@tacaswell
Copy link
Contributor

@johanneskoester Yes, @kmike gave me commit rights this morning and I just confirmed I have access to pypi as well.

As for conda, snakemake now at least installs on py38 (see log in https://discourse.covid-oss-help.org/t/helping-nextstrain/284/20) from the bioconda + conda-forge channels.

@tacaswell
Copy link
Contributor

Commits merged in #80. Thank you @ashwinvis you did most of the hard work here!

@tacaswell tacaswell closed this Mar 26, 2020
@johanneskoester
Copy link

Awesome @tacaswell! I will remove my fork then!

@johanneskoester
Copy link

Done.

@sarnold
Copy link

sarnold commented Mar 30, 2020

Sorry, got distracted with stuff... Feel free to use/fork whatever is useful; I just made a (local) tag so I could have something to package/deploy for now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants