This issue is incited from (re)reading this article on upper bound versions in Python:
https://iscinumpy.dev/post/bound-version-constraints/
From what I can tell, these points seem most relevant to this project:
- Version limits break code too: users can normally cap their versions as needed to fix dependency conflicts, but if we cap the Python version, users can never install our package on newer versions of Python without bypassing the dependency solver.
- It doesn't scale: when multiple packages have upper bounds, it becomes increasingly likely that conflicts will occur when users need them all installed in the same environment.
- Backsolving is usually wrong: most dependency solvers have become "smart" enough to backsolve to older versions, meaning upper bounds can make users inadvertantly install old versions of the library where the upper bound isn't present. 1
- Pinning the Python version is special:
requires-python was intended to help with dropping older versions of Python, not blocking newer versions. Solvers will scroll back through releases as described in point (3) to attempt bypassing the requirement. In addition, Poetry will cap the Python version of the entire project if a single dependency has a cap, even if a later release of that dependency raises or removes the cap.
At the time, I thought this made sense since the original codebase (the start of the fork) embedded pybind11 as a git submodule pinned to a single version, and I didn't expect builds to work on newer versions of Python until pybind11 was updated. However, not only have I since replaced the git submodule with a build dependency on pybind11, meaning users can retrieve newer versions of pybind11 to build from source, but pybind11 doesn't even have an upper bound on the Python version - pybind11's C++ headers are forwards-compatible with Python, meaning old bindings can be built against newer Python versions. Whoops!
The other aspect is that we didn't want users to build their own wheels from source if we could supply it for them. After all, that was the primary reason for making LinkPython-extern. However, I'm realizing that upper bounds don't actually help with this purpose; we still have to release pre-built wheels on a yearly schedule to keep up with Python. If we miss the schedule, our upper bound will actively fight users on the newest Python release who already don't have a choice but to build from source. Removing the cap in this case at least gives them the opportunity to try building it before we can publish prebuilt wheels.
For reference, this is the error message that would result from building the project on Python 3.15.0b4:
$ pip install .
Processing .\.
Installing build dependencies ... done
Getting requirements to build wheel ... done
Installing backend dependencies ... done
Preparing metadata (pyproject.toml) ... done
INFO: pip is looking at multiple versions of linkpython-extern to determine which version is compatible with other requirements. This could take a while.
ERROR: Package 'linkpython-extern' requires a different Python: 3.15.0 not in '<3.15,>=3.9'
When installing from PyPI, the error message becomes more obtuse:
$ pip install linkpython-extern
ERROR: Ignored the following versions that require a different python version: 1.0.0 Requires-Python >=3.6,<3.11; 1.0.1 Requires-Python >=3.6,<=3.11; 1.0.2 Requires-Python >=3.6,<3.12; 1.0.3 Requires-Python >=3.6,<3.12; 1.0.4 Requires-Python >=3.6,<3.13; 1.0.4a1 Requires-Python >=3.6,<3.13; 1.1.0 Requires-Python >=3.7,<3.14; 1.1.0a1 Requires-Python >=3.7,<3.13; 1.1.1 Requires-Python >=3.7,<3.14; 1.2.0 Requires-Python >=3.8,<3.15; 1.2.1 Requires-Python >=3.8,<3.15; 1.3.0a1 Requires-Python >=3.9,<3.15; 1.3.0a2 Requires-Python >=3.9,<3.15
ERROR: Could not find a version that satisfies the requirement linkpython-extern (from versions: none)
ERROR: No matching distribution found for linkpython-extern
When compiling without build tools, the error message is quite long:
$ pip install .
Processing ./.
Installing build dependencies ... done
Getting requirements to build wheel ... done
Installing backend dependencies ... done
Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: LinkPython-extern
Building wheel for LinkPython-extern (pyproject.toml) ... error
error: subprocess-exited-with-error
× Building wheel for LinkPython-extern (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [16 lines of output]
*** scikit-build-core 1.0.3 using CMake 4.4.0 (wheel)
*** Configuring CMake...
loading initial cache file /tmp/tmp1n_4n17n/build/CMakeInit.txt
CMake Error at /tmp/pip-build-env-5k8x4ext/normal/lib/python3.12/site-packages/cmake/data/share/cmake-4.4/Modules/CMakeDetermineCXXCompiler.cmake:47 (message):
Could not find the compiler specified in the environment variable CXX:
x86_64-linux-gnu-g++.
Call Stack (most recent call first):
CMakeLists.txt:2 (project)
CMake Error: CMAKE_CXX_COMPILER not set, after EnableLanguage
-- Configuring incomplete, errors occurred!
*** CMake configuration failed
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for LinkPython-extern
Failed to build LinkPython-extern
error: failed-wheel-build-for-install
× Failed to build installable wheels for some pyproject.toml based projects
╰─> LinkPython-externAlongside removing the upper bound, it would be helpful to replace this message with something more user-friendly like:
LinkPython-extern cannot be compiled without a C++ compiler!
You are using CPython 3.15, which may not have prebuilt wheels for your system on PyPI.
You will need <g++ | Visual Studio C++ Build Tools | Xcode> to compile this project.
If you have a C++ compiler, you may need to set the CMAKE_CXX_COMPILER environment variable.
This issue is incited from (re)reading this article on upper bound versions in Python:
https://iscinumpy.dev/post/bound-version-constraints/
From what I can tell, these points seem most relevant to this project:
requires-pythonwas intended to help with dropping older versions of Python, not blocking newer versions. Solvers will scroll back through releases as described in point (3) to attempt bypassing the requirement. In addition, Poetry will cap the Python version of the entire project if a single dependency has a cap, even if a later release of that dependency raises or removes the cap.At the time, I thought this made sense since the original codebase (the start of the fork) embedded pybind11 as a git submodule pinned to a single version, and I didn't expect builds to work on newer versions of Python until pybind11 was updated. However, not only have I since replaced the git submodule with a build dependency on pybind11, meaning users can retrieve newer versions of pybind11 to build from source, but pybind11 doesn't even have an upper bound on the Python version - pybind11's C++ headers are forwards-compatible with Python, meaning old bindings can be built against newer Python versions. Whoops!
The other aspect is that we didn't want users to build their own wheels from source if we could supply it for them. After all, that was the primary reason for making LinkPython-extern. However, I'm realizing that upper bounds don't actually help with this purpose; we still have to release pre-built wheels on a yearly schedule to keep up with Python. If we miss the schedule, our upper bound will actively fight users on the newest Python release who already don't have a choice but to build from source. Removing the cap in this case at least gives them the opportunity to try building it before we can publish prebuilt wheels.
For reference, this is the error message that would result from building the project on Python 3.15.0b4:
When installing from PyPI, the error message becomes more obtuse:
When compiling without build tools, the error message is quite long:
Alongside removing the upper bound, it would be helpful to replace this message with something more user-friendly like:
Footnotes
This shouldn't actually apply to us because we started capping the Python version from the first release of LinkPython-extern (7ea2358), so we can't get backsolved to an older version. However, if we do remove the upper bound, we can't re-introduce it in a future release without falling into this trap, unless we were to yank all releases without an upper bound. ↩