Replies: 2 comments
|
It's an async library. I think our understanding of free-threaded support is that we don't block the GIL for users, but we don't expect our own code to be running threaded. We don't want to sacrifice performance to locks etc. that are simply not needed in an async application. I suspect we'd be happy with the first suggestion in the Cython code, but have no plans to complete this list. |
C Extension Analysis ReportGenerated from https://github.com/devdanzin/cext-review-toolkit, in conjunction with Claude Code Extension: aiohttp (4 Cython extensions)Scope: Agents Run
Executive Summaryaiohttp's Cython layer has correctly declared Extension Profile
Key Metrics
G = No FIX findings · Y = 1–3 FIX findings · R = 4+ FIX findings Findings by PriorityMust Fix (FIX) — 3
Should Consider (CONSIDER) — 15
Tensions
Policy Decisions (POLICY) — 7
Dismissed / Superseded Findingsft-analysis.md Finding 5 — The llhttp callbacks ( Note: Finding 4 in this report identifies a residual, narrower gap on the HPE_PAUSED Strengths
Git History Insights
Notable commit history:
The fix-to-feature ratio of ~27% across all parser files is elevated relative to a Recommended Action PlanImmediate (FIX items)
Short-term (CONSIDER items)
Longer-term (POLICY / structural)
Notes
|
Uh oh!
There was an error while loading. Please reload this page.
Free-Threading Analysis Report
Generated from https://github.com/devdanzin/ft-review-toolkit, in conjunction with Claude Code
Extension: aiohttp (4 Cython extensions + pure-Python asyncio layer)
Files covered:
aiohttp/_http_parser.pyx,aiohttp/_http_writer.pyx,aiohttp/_websocket/mask.pyx,aiohttp/_websocket/reader_py.py(compiled asreader_c),aiohttp/connector.py,aiohttp/cookiejar.py,aiohttp/client.py,aiohttp/web_protocol.py,aiohttp/web_server.py,aiohttp/_websocket/writer.pyMigration Status
Status: DECLARED — structural migration done, significant runtime gaps remain
Timeline:
945837cc2— Fix thread safety of http writer; addfreethreading_compatible=Trueto every Cython invocation inMakefile; add 3.14t CI matrix entry (fix thread safety of http writer #11464, add free-threading CI for Python 3.14 #11466)2d03bc457— Fix test race on Python 3.14t (test_data_file) (#?)4eb358863— Fix race condition inTCPConnector.close()(fix(connector): resolve race condition in TCPConnector.close() #12787)Declaration:
freethreading_compatible=Truein Makefile causes Cython to emitPy_MOD_GIL_NOT_USEDin the generated C. The extension will run without the GILon CPython 3.13t/3.14t.
What has been done:
_http_writer.pyx(the headline Cython race) — fixed 2025-09-04PYTHON_GIL=0— added 2025-09-04TCPConnector.close()TOCTOU — fixed 2026-06-02What remains: No
_PyEval_StopTheWorldusage anywhere (confirmed). Allremaining gaps are fixable with asyncio locks, threading locks,
call_soon_threadsafe, or immutable type substitutions.Executive Summary
pool, and DNS cache each have unaddressed races that affect any session with
concurrent requests
Findings by Priority
RACE Findings (fix immediately) — 3
@freelist(250)onRawRequestMessage/RawResponseMessage— module-global C array and counter written non-atomically on every alloc/dealloc; concurrent parsers cause use-after-free_http_parser.pyx:129,229CookieJar.filter_cookies/update_cookies/_do_expirationrace — concurrent calls from different request coroutines mutate_cookies,_expire_heap,_morsel_cachesimultaneously with no lockcookiejar.py:265–481_SIMPLE_COOKIE = SimpleCookie()singleton mutated by concurrent_build_morselcalls —value_encodemodifiesBaseCookie.__data; concurrent calls corrupt coded valuescookiejar.py:38,489UNSAFE Findings (fix before declaring free-threading support) — 4
Connection.__del__GC finalizer calls_releaseon connector pool from any GC thread —_acquired,_conns, and_cleanup_closed_transportsmutated outside the event loopconnector.py:132–145,748–781PyBuffer_Releasenot called on exception paths infeed_data— when a Python callback (e.g.BadHttpMessage) raises duringllhttp_execute, thePy_bufferexport on the data object is never released_http_parser.pyx:634–652_DNSCacheTable.next_addrsraces on shareditertools.cycleiterator andOrderedDict.move_to_end— no lock; concurrent requests to the same host corrupt round-robin stateconnector.py:831–837_cleanup_closed_transportsappended from GC-thread_releasewhile event-loop timer iterates and clears it — list realloc during iteration is use-after-free (root cause: Finding 4)connector.py:429–433,778PROTECT Findings (add synchronization) — 5
Server.requests_count += 1is a non-atomic read-modify-write; gunicorn monitoring reads it cross-thread; lost updates causemax_requestsshutdown to trigger late or neverweb_server.py:72,web_protocol.py:679WebSocketWriter.send_frame()uncompressed/control-frame path bypasses_send_lock— a PING can interleave bytes with a large in-flight compressed frame, corrupting the WebSocket stream_websocket/writer.py:80–82,171BaseConnector._closedcheck-then-set in_close_immediatelyis unguarded — two concurrentclose()callers (e.g. user + GC finalizer) both pass the check and double-execute teardownconnector.py:462–465ClientSession.closedproperty loadsself._connectortwice — another thread callingclose()can null it between theis Nonecheck and the.closedaccess, raisingAttributeErrorclient.py:1377BaseConnector.connect()checksself._closedafterawait _create_connection— concurrentclose()between the await and the check can leak the just-created TCP socket (never closed, never tracked)connector.py:599–636MIGRATE Findings (structural changes) — 5
cdef list _http_methodpopulated by aforloop at module init — change tocdef tupleto prevent any future post-init mutation and remove mutable-container concern_http_parser.pyx:105–109ALLOWED_CLOSE_CODES: Final[set[int]]—Finaldoes not prevent runtime mutation; replace withfrozensetto match intent and make reads lock-free_websocket/reader_py.py:27WebSocketDataQueue.read()enforces single-consumer with a bareassert(stripped by-O) — replace with an explicitRuntimeErroror addasyncio.Lockfor multi-consumer safety_websocket/reader_py.py:117–126cpXXXt-tagged) wheels published to PyPI —cibuildwheelhas no[tool.cibuildwheel.freethreading]section; users on 3.14t get a GIL-tagged wheel that silently re-enables the GILpyproject.toml(missing)cdef objectglobals in_http_parser.pyx+_istrin_http_writer.pyxusem_size=-1(no per-module state struct) — broken subinterpreter isolation; should migrate toPyModule_GetState_http_parser.pyx:62–74,_http_writer.pyx:12SAFE Patterns (confirmed safe)
cdef frozenset ALLOWED_UPGRADES/SINGLETON_HEADERS(_http_parser.pyx:51,81): Immutable after init;frozenset.__contains__is lock-free under free-threadingcdef objectclass-cache globals (_http_parser.pyx:62–74): Written once at import under the import lock; read-only thereafter; safe for concurrent reads_http_methodlist reads (_http_parser.pyx:112–116): No post-init writes;http_method_str()is a pure reader — safe today (hardening totupleis recommended)nogil/Py_BEGIN_ALLOW_THREADSregions in any.pyxfile — zero instances of "unsafe API called without GIL" in the extension code_PyEval_StopTheWorldusage — confirmed by full-repo grep; none of the races require global quiescence to fixRecommendations
Immediate (RACE — Findings 1–3)
1. Fix Finding 1 — disable Cython freelist for free-threading builds
Pass
DEFAULT_FREELIST_SIZE=0when building forPy_GIL_DISABLED, or guard thedecorator with a conditional. Cython ≥ 3.1 automatically disables freelists when
freethreading_compatible=Trueif and only if it is compiled against afree-threading Python; verify the build environment and add an explicit guard:
Or simply remove
@cython.freelistentirely from bothRawRequestMessage(line 129)and
RawResponseMessage(line 229) and add a tracking issue to restore it onlyfor GIL-enabled builds.
2. Fix Finding 2 — add
asyncio.LocktoCookieJarAll callers of
filter_cookiesandupdate_cookiesare alreadyasynccontexts(
client.py:642, response processing). Anasyncio.Lockis the correct primitivehere — it does not block any OS thread and keeps everything on the event loop.
3. Fix Finding 3 — replace
_SIMPLE_COOKIEsingleton with a local instanceThe per-call allocation is negligible compared to the network I/O surrounding it.
Immediate (UNSAFE — Findings 4–7)
4. Fix Finding 4 — marshal
Connection.__del__back to the event looploop.call_soon_threadsafeis one of the few APIs explicitly safe to call fromany OS thread. This single change also resolves Finding 7 by keeping all
_cleanup_closed_transportsmutations on the event loop thread.5. Fix Finding 5 — wrap
llhttp_executeintry/finallyfor buffer release6. Fix Finding 6 — add
threading.Lockto_DNSCacheTableA
threading.Lock(notasyncio.Lock) is correct here becausenext_addrsiscalled from synchronous code paths. Finding 7 resolves automatically once
Finding 4 is fixed.
Short-term (PROTECT — Findings 8–12)
8. Fix Finding 8 — protect
requests_countincrementIf exact counts are required for
max_requestsenforcement, wrap the increment:If a slightly imprecise count is acceptable (monitoring use only), document the
known imprecision and accept the race.
9. Fix Finding 9 — extend
_send_lockto uncompressed/control-frame pathThe
asyncio.Lockalready exists (self._send_lock); the fix is to acquire iton the uncompressed branch:
10. Fix Finding 10 — guard
_closedtransitions in_close_immediately11. Fix Finding 11 — single local capture in
ClientSession.closed12. Fix Finding 12 — close leaked protocol in
connect()error pathLonger-term (MIGRATE — Findings 13–17)
13. Fix Finding 13 — convert
_http_methodtotuple14. Fix Finding 14 — replace
ALLOWED_CLOSE_CODESwithfrozenset15. Fix Finding 15 — harden
WebSocketDataQueue.read()single-consumer check16. Fix Finding 16 — publish free-threaded wheels on PyPI
Add to
pyproject.toml:Or set
CIBW_FREE_THREADED_SUPPORT=1in CI. Also consider adding theProgramming Language :: Python :: Free Threading :: 3 - StablePyPI classifieronce RACE/UNSAFE findings above are resolved.
17. Fix Finding 17 — migrate Cython module globals to per-module state
Long-term refactor: change all
cdef objectmodule-level variables in_http_parser.pyx(lines 62–74) and_istrin_http_writer.pyx(line 12) touse a
m_size > 0module state struct accessed viaPyModule_GetState. This isrequired for correct subinterpreter isolation and is the CPython-recommended
pattern for free-threading extensions.
For a phased migration plan:
/ft-review-toolkit:planAll reactions