Projects using cibuildwheel

[henryiii]

This is a place to post projects that are using cibuildwheel. Every so often we will bundle the projects listed here and add them to the examples.

[henryiii]

RTree: Toblerity/rtree#163

[henryiii]

Interesting, discussions don't seem to cross reference.

[alfaro96]

We are pleased to announce that the wheel builder of scikit-learn has been successfully moved to cibuildwheel: scikit-learn/scikit-learn#17921.

[henryiii]

And looks like ARM emulation is now in, too!

[Czaki]

psutil was migrated to cibuildwheel giampaolo/psutil#1550 (comment)

[henryiii]

I guess they just don't provide wheels for 2.6 & 3.4. First action file I've seen testing FreeBSD. :) Also one of the few projects I've seen that doesn't pin cibuildwheel.

[Czaki]

Also one of the few projects I've seen that doesn't pin cibuildwheel.

This may be my fault. I help during the integration of cibuildwheel and may forget to pin cibuildwheel on end.

[henryiii]

I have an idea for pinning in #492

[Czaki]

It looks very nice. Dependabot does not work well with python dependencies pinned in yaml file. But with actions may work

[henryiii]

Looks like https://github.com/ets-labs/python-dependency-injector, Travis CI (but maybe moving to GHA?)

[henryiii]

Adding a few:

• numexpr
• PyTables
• dd-trace-py (using alternate arch emulation)
• Tokenizer
• matrixprofile

[henryiii]

Something to keep an eye on, perhaps? mypyc/mypy_mypyc-wheels#11

[YannickJadoul]

PR seems to have been merged :-)

[henryiii]

First release of MyPy built using cibuildwheel looks like it's due out tomorrow! :) python/mypy#9926

[henryiii]

deepcharles/ruptures, which has a very comprehensive pyproject.toml that was just added in a big build-system update. A) very useful for the Cython projects I still manage, and b) makes be very glad I'm pretty much free of Cython and use pybind11 instead. :)

[Czaki]

imagecodecs package https://github.com/cgohlke/imagecodecs wheels for linux and macos are build using cibuildwheel in repository: https://github.com/czaki/imagecodecs_build repository.

It may be interesing as it has over 20 external dependecies in compiled libraries. For speedup this project use custom docker images (because often macos needs much more steeps to get working)

Interesting thing is usage of libomp which not comes from brew so wheel is 10.9. Its also use libraries installed in user space and install_name_tool for fix to allow delocate to work.

it also depends on numpy and build wheels for pypy using openblas from brew (#402)

[guillaumekln]

Hi there!

We just moved to cibuildwheel in https://github.com/OpenNMT/CTranslate2. We build wheels for Linux and macOS that include libraries from the oneAPI toolkit. The Linux wheels also include CUDA libraries for GPU execution.

cibuildwheel has proved to be very useful to add macOS support in addition to Linux. Thanks for your work!

[henryiii]

Looks like markupsafe (from pallets project, think used by flask and jinja) uses cibuildwheel: https://github.com/pallets/markupsafe/blob/master/.github/workflows/build.yaml

[henryiii]

https://github.com/dib-lab/sourmash - includes emulated Linux ARM builds. :)

[Czaki]

I push python-sanppy to use cibuildwheel and its use all Linux emulation now:
intake/python-snappy#103 (comment)

[maxbachmann]

I use cibuildwheel in my project https://github.com/maxbachmann/RapidFuzz as well to build wheels for all platform/architecture/python combinations starting Python 3.5

[henryiii]

Looks like CMS (the CERN experiment) is using cibuildwheel in a few places now. :) Not sure there's a published package that we could add to our list (CC @nsmith- ?). Here's the original comment: "cibuildwheel taught me things I didn’t want to know, like that i686 platforms have different floating point math behavior", see https://github.com/cms-nanoAOD/correctionlib/blob/9c8d10f4d9330a612b4e7872615b38db012434f3/tests/test_core.py#L124-L127 and cms-nanoAOD/correctionlib@f852688

[henryiii]

Looks like it's here: https://pypi.org/project/correctionlib/

[nsmith-]

👋 if anyone can tell me why those tests only fail on i686 containers I'd buy them a 🍺. In practice its not a huge concern.

[Czaki]

I'm not sure which float use your code. but there may be difference with precision of representation (64 vs 32 bits)

[joaander]

I don't know if this is related to your test failures, but the FPUs on x86 32-bit processors use an internal 80-bit floating point representation, and round down to 64-bit for double or 32-bit for float. When building for x86_64, most compilers (i.e. gcc) default to using SSE instructions which maintain 64-bit precision throughout double precision calculations (or 32-bit for float). The differences in rounding for intermediate values can lead to different defaults.

[nsmith-]

I consistently use double throughout. In a few spot checks it seems that the python math stays consistent between platforms but my ast evaluator does not. Its possible my parsing grammar produces a different order of operations that somehow only causes enough rounding errors on i686. I'm not sure.

[joaander]

I just started using cibuildwheel for one of our projects: https://github.com/glotzerlab/gsd . GSD uses cython and numpy. GSD builds on 64-bit architectures and runs unit tests on the built wheels.

The conversion process went very well thanks to the excellent documentation for cibuildwheel! In the future, our group will look into using cibuildwheel for other projects.

[henryiii]

Looks like uber/h3-py provides cibuiltwheels now. :)

Also robotology/idyntree.

[xuyxu]

Our research group has been using cibuildwheel to publish the Python package of deep forest.

Repo | PyPI | CI

Thanks for developing this wonderful tool ;-)

[MarshalX]

Iam using cibuildwheel in tgcalls to build wheels for macOS (intel) and Windows (x32, x64). My library it’s a binding via pybind11 to c++ library that uses webrtc. So in my GitHub actions workflow there is many 3dparty deps like OpenSSL, ffmpeg, webrtc and so on. I think it’s useful example for other devs.

My PR with adding this project: #800

[bigcat88]

Wrapper of libheif
x86_64+ aarch64 - MacOS,Linux binary wheels + sdist.

Using CFFI, build libheif, libde265, x265 libs from source.
Creating github release and publish on pypi.

pypi page

[henryiii]

From a quick search, I found a few more that have around 2k stars or more (was going for 1k+, but there were too many - picked a few fun ones at the end a bit under 1k):

• tornadoweb/tornado (travis)
• pytorch/fairseq
• MagicStack/uvloop
• MagicStack/asyncpg
• samuelcolvin/pydantic
• vaexio/vaex
• cvxpy/cvxpy
• duckdb/duckdb (travis)
• open/triton
• vispy/vispy
• confluentinc/confluent-kafka-python › tools/wheels/build-wheels.bat ?
• tinyobjloader/tinyobjloader (Asure)
• indygreg/PyOxidizer
• deepmind/open_spiel
• google/benchmark
• apache/beam
• Legrandin/pycryptodome
• yaml/pyyaml
• pyutils/line_profiler
• PixarAnimationStudios/OpenTimelineIO
• AcademySoftwareFoundation/OpenColorIO

Total there are over 1K matches for CIBW_BUILD, though that's total matches not repositories that had a match.

[henryiii]

https://sourcegraph.com/search?q=context:global+CIBW_BUILD+count:2000+select:repo&patternType=literal&case=yes shows 393 repos. (Edit: now over 600, and we've passed 1.1K stars)

[henryiii]

This one is really cool, IMO: https://github.com/ssciwr/clang-format-wheel

See https://twitter.com/HenrySchreiner3/status/1442877941910802433?s=20

[dvarrazzo]

psycopg is using cibuildwheel from release 3.0 for Linux, macOS, WIndows, and, from 3.0.3, for Musl Linux too. Thank you very much!

[TonyXiang8787]

Hi, our recently published project power-grid-model also uses this tool to build.

https://github.com/alliander-opensource/power-grid-model

• Short description: Python/C++ library for distribution power system analysis
• Platform: Linux-x86_64, windows-amd64, macos-x86_64 and macos-arm64

[benfred]

I'm using cibuildwheel with scikit-build to build wheels for https://github.com/benfred/implicit - a library for implicit feedback collaborative filtering. This includes GPU support for linux wheels.

[MarshalX]

I am using cibuildwheel with Python WebRTC to build wheels for all platforms. My project is a Python extension that provides bindings to WebRTC M92. It follows the W3C specification with some modifications and additions to make it work better with Python applications and with useful APIs like programmatic audio and video.

[eriknw]

I just set up cytoolz to use cibuildwheel on Github Actions here. This builds all wheels except for win_arm64, which comes out to 83 different wheels.

This creates one wheel per CI job, which I doubt is the preferred way to do it.

[Czaki]

This creates one wheel per CI job, which I doubt is the preferred way to do it.

no. The default way is to create one job per system with a possible split for architecture. Especially on linux there is a used docker container that contains all python versions for a given architecture.

[henryiii]

It depends on how long it takes to build each wheel and how many parallel runners you have, and if you have shared before-all setup you can reuse (not common). One job per wheel does maximize parallel runner usage. I usually split up wheels into jobs, but usually have 2-4 wheels per job.

[eriknw]

Cool, thanks for replying. One wheel per job makes it easier to isolate failures, which is the main reason I did this. It's also plenty fast for me. It would be easy to change though now that it's working.

If you had to compile an external C library that takes, say, 2 hours to build, what's the best way to split up jobs? By system and architecture? What about for Windows and osx?

[bjlittle]

We're super stoked to be using cibuildwheel to automate the building and testing of binary wheels for our cf-units package on both macos and linux. See our GitHub Action ci-wheels.yml for details.

cf-units is a Scientific Python package that provides units of measure as required by the Climate and Forecast (CF) Metadata Conventions.

Thanks for such an awesome package! It's made our lives as package maintainers so much easier! 🍻

[wbarnha]

cibuildwheel is so great! We are finally able to automate wheel builds for faust-streaming and its python-rocksdb fork, thank you all so much!

[JelleBakker]

x-IMU3 Software and Fusion is using cibuildwheel for building and testing wheels for Mac, Ubuntu and Windows. It's working great!

[Marco-Sulla]

Well, I add also my little project:

https://github.com/Marco-Sulla/python-frozendict/blob/master/.github/workflows/build_primary_wheels.yml
https://github.com/Marco-Sulla/python-frozendict/blob/master/.github/workflows/build_secondary_wheels.yml