Build linux wheels for typed_python #393
Conversation
Using manylinux we can build wheels across different python versions. This makes it easier to create binary distributions of typed_python so that it doesn't always need to be built from the source distro when installing.
|
not sure why the tests are failing -- I've not touched any of the existing code |
|
Very cool! I'll take a look when I get a chance. On the numpy thing, here's
how it works:
1. we have a dependency on numpy in typed python but we can use any version
2. you're supposed to depend in the pyproject.toml for the build on the
"earliest installable version of numpy for this python" because if you
build a C shared object against a specific version of numpy, it's binary
compatible with forward versions, and this strategy ensures that when you
build a version of the shared object you get one that will be compatible
with _any_ version of numpy that's installed on that target system.
3. unfortunately, the c++ code in object_database depends on typed python
and must be compiled against the _exact_ version of typed python that's
currently installed. This goes against the assumptions that pip wants to
make. I have it as a project to decouple ODB's dependency on TP to the
point where we could be similarly "forward compatible" but have yet to do
that
I don't know how the manylinux wheels works regarding numpy - if you build
a wheel against the "earliest installable version of numpy for this
python", that has to be a version that's determined, I guess, simply by the
version of python you're on and the platform you're targeting. But I would
imagine we can just do what other libraries that depend on numpy have done
and it ought to work.
…On Fri, Jul 29, 2022 at 3:31 PM Chang She ***@***.***> wrote:
@changhiskhan <https://github.com/changhiskhan> requested your review on:
#393 <#393> Build
linux wheels for typed_python.
—
Reply to this email directly, view it on GitHub
<#393 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AB6OHBDVMJOJBCIEYFXAPEDVWQWRVANCNFSM55BUEQLA>
.
You are receiving this because your review was requested.Message ID:
***@***.***
com>
|
|
Manylinux is agnostic to how you want to manage dependencies. What I've done here is that I'm using the Pipfile.lock for each python version to build to install the dependencies before building the corresponding wheel. Using pipenv these are all isolated in separate virtualenvs. WRT object_database, we could do the same thing. And as long as we keep the typed_python versioned pinned in object_database requirements then if we installed the odb wheel it should be able to automatically grab the right wheel for typed_python if the wrong version is installed. I wish there was an easier way to be macOS builds working properly. I don't want @dkrasner 's hair to get even crazier |
|
The Pipfile.lock all have different numpy versions but looking at them it doesn't seem like they're the oldest available. Should be a minor change though to get it using the build requires in pyproject.toml |
|
ok i switched to using |
Using manylinux we can build wheels across different python versions. This makes it easier to create binary distributions of typed_python so that it doesn't always need to be built from the source distro when installing.
More details are in the
build_scripts/README.mdattached in this PR.I've tested this by installing the wheel into a conda environment and going through the typed_python
I haven't dug into the numpy version compat issues. @braxtonmckee would you happen to have examples I can use to try and repro any such issues? Potentially if we need an exact version match, I think we could generate the requirements file during the wheels build using the Pipfile.lock, so then setup.py can pick up the pinned requirements versions and install those exact versions.
We can do the same thing in object_database to build wheels and make the distro for both a lot simpler?