-
Notifications
You must be signed in to change notification settings - Fork 229
Description
I'm working on ecosystem support for free-threaded Python as part of the broader effort to introduce free-threaded Python. Lately, I've been actively working on adding support for free-threaded Python 3.14. Ultimately I'd like to make it possible to run vLLM on free-threaded Python 3.14: vllm-project/vllm#28762.
Currently openai-harmony ships abi3 wheels. Unfortunately, the free-threaded build cannot support abi3 and the existing wheels cannot be installed on the free-threaded build. Hopefully Python 3.15 will support a new version of the stable ABI, but for Python 3.14 you'll need to build version-specific wheels if you'd like to support the free-threaded build.
Are you open to this? I'm happy to put in the bulk of the work. I'm also a PyO3 maintainer and helped add support in PyO3 for the free-threaded build.
I'm able to build the project from source on free-threaded 3.14.0 and all the tests pass when I force-disable the GIL:
goldbaum at Nathans-MBP in ~/Documents/harmony on main
± PYTHON_GIL=0 pytest
================================================================================= test session starts ==================================================================================
platform darwin -- Python 3.14.0, pytest-8.3.3, pluggy-1.6.0
Using --randomly-seed=624209616
rootdir: /Users/goldbaum/Documents/harmony
configfile: pyproject.toml
testpaths: tests
plugins: hypothesis-6.137.1, anyio-4.11.0, scipy_doctest-2.0.1, xdist-3.8.0, timeout-2.4.0, cov-4.1.0, run-parallel-0.7.0, randomly-4.0.1
collected 42 items
Collected 0 items to run in parallel
tests/test_harmony.py .......................................... [100%]
================================================================================== 42 passed in 4.43s ==================================================================================
I don't see any explicitly multithreaded tests. It'd be nice to have some multithreaded testing to probe behavior that will only happen on the free-threaded build.
I don't see any unsafe impl Send or Sync so I think the extension itself should be thread-safe and doesn't rely on the GIL. That said, I haven't done a detailed thread safety pass.