Skip to content

Require Python 3.12+ (Python 3.10/3.11 hang on macOS 15 during corpus download) #17

@janhoy

Description

@janhoy

Problem

On macOS 15 (Darwin 25.4.0), running Solr Orbit with Python 3.10 or 3.11 causes the benchmark to hang indefinitely at ~200% CPU during the first corpus download (when the local data cache is empty).

Root Cause

Python 3.10/3.11 has a bug triggered by macOS 15's high-precision nanosecond filesystem timestamps. When os.stat() is called on any recently-created file, the fill_time() function in CPython's posixmodule.c converts the nanosecond timestamp using PyNumber_Multiply. For large nanosecond values, this triggers Karatsuba big-integer multiplication (k_mul), which calls _PyErr_CheckSignalsTstate. If a Python signal handler is active at that moment, it executes Python bytecode that calls os.stat() again — creating infinite recursion at ~200% CPU.

This was discovered by profiling the hung process with sample, which showed:

os_stat → fill_time → PyNumber_Multiply → k_mul → _PyErr_CheckSignalsTstate
  → (Python signal handler) → os_stat → fill_time → ... (infinite recursion)

The hang affects urllib3/SSL corpus downloads. It does not affect runs where corpus data is already cached.

Fix

Upgrade to Python 3.12+, which does not exhibit this behavior.

Suggested Changes

  • Update .python-version to 3.12.x
  • Update supported_python_versions in setup.py to start from (3, 12)
  • Update check_python_version() in osbenchmark/__init__.py to require >= 3.12
  • Update tox.ini to drop py310 and py311 environments
  • Update DEVELOPER_GUIDE.md prerequisite to "Python 3.12+" with a note about the macOS 15 issue

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No fields configured for Bug.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions