Skip to content

Fix Triton AOT multi-context launches#195

Merged
voltjia merged 4 commits into
ssa-compiler-with-multibackendfrom
fix-triton-aot-multi-context
Jul 16, 2026
Merged

Fix Triton AOT multi-context launches#195
voltjia merged 4 commits into
ssa-compiler-with-multibackendfrom
fix-triton-aot-multi-context

Conversation

@voltjia

@voltjia voltjia commented Jul 15, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Cache every Triton AOT module/function pair by the current CUcontext while serializing host launch submission.
  • Invalidate stale cached launchers with a manifest schema and reject reloaded artifacts that do not provide the context guard exports.
  • Cover fresh and reloaded handles across repeated CUDA context transitions, including CPU-visible launcher contracts.

Root cause

Triton's generated AOT launcher stores one CUmodule and CUfunction in process-global variables. After the first launch initializes those values in one CUDA context, a launch through the same DSO in another context reuses invalid handles and returns CUDA_ERROR_INVALID_HANDLE (400).

Validation

  • ruff format --check src/ninetoothed/backends/materializers/triton.py tests/test_built_artifact_reload.py tests/test_triton_aot_materializer.py
  • ruff check src/ninetoothed/backends/materializers/triton.py tests/test_built_artifact_reload.py tests/test_triton_aot_materializer.py
  • python scripts/check_contributing_style.py
  • pytest tests/test_triton_aot_materializer.py -q
  • pytest tests/test_built_artifact_reload.py::test_triton_aot_handle_is_reusable_across_cuda_contexts -q
  • pytest 'tests/test_aot.py::test_add[True-45327-dtype0-bf16-cuda]' -q

pytest output:

$ pytest tests/test_triton_aot_materializer.py -q
...............                                                          [100%]
15 passed in 1.13s

$ pytest tests/test_built_artifact_reload.py::test_triton_aot_handle_is_reusable_across_cuda_contexts -q
.                                                                        [100%]
1 passed in 8.26s

$ pytest 'tests/test_aot.py::test_add[True-45327-dtype0-bf16-cuda]' -q
.                                                                        [100%]
1 passed in 8.54s

$ pytest
============================= test session starts ==============================
platform linux -- Python 3.10.16, pytest-9.0.2, pluggy-1.6.0
rootdir: /tmp/ninetoothed-context-guard-red.zl9Xdv
configfile: pyproject.toml
collected 353 items

======================= 353 passed in 125.75s (0:02:05) ========================

@voltjia
voltjia marked this pull request as ready for review July 15, 2026 06:04
@voltjia
voltjia merged commit a75d742 into ssa-compiler-with-multibackend Jul 16, 2026
10 checks passed
@voltjia
voltjia deleted the fix-triton-aot-multi-context branch July 16, 2026 09:10
voltjia added a commit that referenced this pull request Jul 16, 2026
* Add SSA compiler pipeline and unified backends

* refine ssa multibackend PR boundaries and pass registration

* move backend ssa pass registration into backend modules

* Add backend SSA pass contracts and registry validation

* revert test_aot_auto_tuning.py

* fix ruff

* style: satisfy contributing style checks after rebase

* Refactor SSA compiler naming (#169)

* Refactor IR dataclass constructors (#172)

* unify SSA compilation and executable multi-backend support

* Fix ruff error, refine SSA compiler structure and address review feedback

* Refactor SSA backend pipeline and restore Triton runtime autotining

* Removing TVM backend and refine Triton, CUDA, TileLang backends

* Fix zero-like GPU launches, architecture-aware caching, and JIT API

* Fix Triton AOT multi-context launches (#195)

* Add Triton AOT multi-context design

* Fix Triton AOT multi-context launches

* Cover fresh Triton AOT context guard

* Remove development process documents

* Address remaining SSA review comments (#196)

* Address remaining SSA review comments

* Sort test imports for Ruff

* Remove redundant entrypoint assertions

---------

Co-authored-by: Jiacheng Huang <45955067+voltjia@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants