Skip to content
BranchesTags

Repository files navigation

Talyn: Robust, Stable AsyncIO Event Loop for Python

License Python Compatibility Linux Compatibility Zig Compatibility PyPI Version

Talyn is a robust, exceptionally stable, and realistically fast asyncio event loop drop-in replacement for Python, powered by the asynchronous capabilities of Zig and io_uring.

Talyn prioritizes correctness, complete system safety, and high usability over artificial micro-benchmark superiority. It is fully compatible with CPython's standard single-threaded and free-threaded (GIL-disabled) runtimes.

🚀 Features

  • Realistic Speed: Designed to deliver solid and reliable I/O performance on Linux by leveraging io_uring's native kernel-side asynchronous completion queues.
  • Robust & Crash-Resistant: Meticulously hardened against circular reference memory leaks, stack alignment faults, signal interrupt deadlocks, and use-after-free bugs.
  • Full Asyncio Compatibility: Passes 100% of standard Python asyncio, subprocess, transports, and connection-lifecycle test suites.
  • Modern Packaging: Fully migrated to PEP 517/518 standard declarative pyproject.toml configuration.
  • GIL-Disabled Free-Threading Ready: Fully compatible with python3.13t and python3.14t without memory races.

📜 Requirements

  • Python: >= 3.13 (Tested and verified under CPython 3.13, 3.14, 3.13t (free-threaded), and 3.14t (free-threaded))
  • Linux Kernel: >= 7.0 (Verified on Linux Kernel 7.0.x)
  • Zig Compiler (for source builds): 0.16.0 (Fedora packages)

Note

Tested Platform Verification: Talyn is primarily developed and tested under Fedora 44 on an x86_64 architecture equipped with an Intel(R) Core(TM) Ultra 7 265 processor. Since v0.8.7 we also build and verify aarch64 and riscv64 wheels on the same x86_64 host: the native extension is cross-compiled by Zig, and the full test suite is executed inside Fedora 44 QEMU VMs (see Linux Development below).

🔧 Installation & Testing

To compile and install Talyn locally on a native Linux machine, run:

pip install -e .

🍎 macOS Development & Testing via Podman (Apple Silicon)

Since Talyn is a Linux-only project leveraging io_uring, development and testing on macOS require running inside a Linux environment.

We provide a frictionless, automated setup using Podman and a Fedora 44 (AARCH64) container:

  1. Install Podman: 
    brew install podman
  2. Initialize the Podman VM (only needed once): 
    podman machine init
  3. Run the test suite (the script automatically starts the VM if it's stopped, builds the image, runs the tests with proper permissions, and stops the VM at the end if it started it): 
    ./scripts/macos/run_tests.sh

You can pass any options supported by test_all.sh (e.g., --verbose or --python=3.13):

./scripts/macos/run_tests.sh --verbose --python=3.13

To force a rebuild of the Fedora testing image:

./scripts/macos/run_tests.sh --rebuild-image

📊 Benchmarking

We also provide a wrapper to run the benchmark suite inside the Fedora container using the optimized ReleaseFast target:

./scripts/macos/benchmark.sh --python=python3.13

You can target a specific benchmark using --bench (note that benchmark names with spaces must be quoted):

./scripts/macos/benchmark.sh --python=python3.13 --bench="task spawn"

The generated comparison plots will be automatically saved to benchmarks/output/.

📦 Building & Publishing Multi-Architecture Wheels

If you are developing on macOS (Apple Silicon) and need to build and publish wheels for both aarch64 and x86_64 architectures to PyPI, you can do so in a single command using Podman's emulation:

  1. Build all wheels: This script builds a native aarch64 container image and an emulated x86_64 container image, runs scripts/build.sh inside both, and collects all 8 wheels (4 Python versions × 2 architectures) in the ./dist/ directory:

    ./scripts/macos/build_all_wheels.sh

  2. Publish to PyPI: Upload the built distributions in ./dist/ using twine (pre-configured in your repository):

    ./scripts/publish.sh

🖥️ Linux (x86_64) Development: Build, Cross-Compile & Multi-Arch Testing

All of Talyn's development—including aarch64 and riscv64 wheels and full test-suite runs—can be done on an x86_64 (Intel) Linux PC, with no MacBook required:

  • Building: Zig is a native cross-compiler, so foreign-architecture wheels and the extension are produced at full host speed — no QEMU involved.
  • Testing: QEMU user-mode emulation (e.g. podman run --platform linux/arm64) cannot run Talyn, because io_uring_setup returns ENOSYS under user-mode emulation and Talyn requires io_uring with no fallback. Instead, we boot a full-system Fedora 44 VM per architecture, whose real guest kernel passes io_uring syscalls through to the host kernel.

📋 1. Fedora 44 required packages

Install Zig, all four Python interpreters with development headers, and the stdlib test suites:

sudo dnf install zig \
    python3.13 python3.13-devel python3.13-freethreading \
    python3.14 python3.14-devel python3.14-freethreading python3.14-freethreading-devel \
    python3.13-test python3.14-test python3.14-freethreading-test

Bootstrap pip and the test tooling for each interpreter:

for p in python3.13 python3.13t python3.14 python3.14t; do
    $p -m ensurepip --upgrade
    $p -m pip install --user pytest pytest-asyncio
done

For foreign-architecture VM testing (aarch64 / riscv64), additionally:

sudo dnf install qemu-system-aarch64 edk2-aarch64 qemu-system-riscv edk2-riscv64 \
    genisoimage openssh-clients

Note

python3.13-freethreading-test is not packaged for Fedora 44; the stdlib asyncio modules for the free-threaded 3.13 build are skipped automatically by the test suite.

🏗️ 2. Build all wheels natively (x86_64 + aarch64 + riscv64)

./scripts/linux/build_all_wheels.sh

Uses Zig's native cross-compiler — no QEMU needed. Produces 12 wheels (4 Python variants × 3 architectures) in ./dist/, ready for ./scripts/publish.sh.

🧪 3. Run the test suite natively (x86_64)

./scripts/test_all.sh                  # Debug build
./scripts/test_all.sh --starburst      # ReleaseFast build

🖥️ 4. Test foreign-architecture wheels in VMs

Boots a real Fedora 44 VM per architecture (first run downloads the ~0.5 GB cloud image to ~/.cache/talyn-*-vm/) and runs the full pytest suite against each installed wheel:

./scripts/linux/run_tests.sh                     # aarch64 wheels
./scripts/linux/run_tests.sh --arch=riscv64      # riscv64 wheels
./scripts/linux/run_tests.sh --smoke             # quick import + event-loop smoke test only
./scripts/linux/run_tests.sh --shutdown          # stop the VM after the run

⚡ 5. Run the full test_all.sh suite in VMs (cross-compiled, fast)

test_all.sh normally compiles the extension inside the VM, which is slow under QEMU TCG. Instead, cross-compile the four extension variants natively on the host (the same build flags the wheels use) and run test_all.sh --no-build in the VM:

./scripts/linux/run_test_all.sh --arch=aarch64 --starburst
./scripts/linux/run_test_all.sh --arch=riscv64 --starburst

Under heavy emulation, a stdlib module may exceed its per-module timeout (e.g. test_subprocess on riscv64); raise it with TALYN_STDLIB_TIMEOUT=600 and the memory-safety repro timeout with TALYN_REPRO_TIMEOUT=900.

⏱️ Measured timings

scripts/test_all.sh --starburst (4 Python variants: pytest suite + stdlib asyncio suite):

Environment Total time
x86_64 (native) 8m28s
aarch64 VM, in-VM build 42m33s
riscv64 VM, in-VM build 44m56s
aarch64 VM, cross-built (run_test_all.sh) 17m47s
riscv64 VM, cross-built (run_test_all.sh) 24m48s

All configurations pass the pytest suite; under QEMU TCG the emulated runs are ~5× slower than native, and the cross-built runs cut the in-VM build phase (~2.4× speedup on aarch64). An occasional stdlib-asyncio module may time out under heavy emulation (see note above about TALYN_STDLIB_TIMEOUT).

⚡ Optimization & Target Compilation (For Developers & Power Users)

By default, the pre-built wheels generated by build.sh are compiled targeting a generic x86_64 CPU architecture baseline to ensure 100% universal compatibility across all 64-bit modern x86 Linux processors (e.g., matching standard PyPI manylinux wheel compatibility).

Based on our benchmarks and comprehensive validation—including 100% passing results in the standard asyncio test suite across four distinct Python versions—we publish the official binary packages (wheels) compiled in Starburst mode (Zig built with ReleaseFast) to deliver peak performance out of the box with proven stability.

If you are compiling from source, you can customize the compilation optimize mode and target CPU architecture to unleash maximum performance:

1. Compile and Optimize for your Native CPU (Highly Recommended)

To compile Talyn so that it takes full advantage of your host's exact CPU instructions (such as AVX2, AVX-512, cache alignment, etc.):

# Omit TALYN_CPU so Zig defaults to native CPU optimization
TALYN_OPTIMIZE=ReleaseFast pip install .

2. Configure Compilation Modes

You can control Zig's optimize mode by setting the TALYN_OPTIMIZE environment variable (defaults to Debug for developer convenience):

  • TALYN_OPTIMIZE=Debug (Default): Compiles with heavy runtime assertions and debug symbols.
  • TALYN_OPTIMIZE=ReleaseSafe: Compiles with full optimizations but keeps safety checks (e.g. out-of-bounds, overflows).
  • TALYN_OPTIMIZE=ReleaseFast: Compiles with maximum optimizations (Starburst mode), disabling safety checks for peak execution speed.

3. Target a Specific CPU microarchitecture

You can force Zig to compile for a specific target CPU microarchitecture (like x86_64_v2 or x86_64_v3):

TALYN_OPTIMIZE=ReleaseFast TALYN_CPU=x86_64_v3 pip install .

📦 Usage

Basic Usage

To run a coroutine directly with the Talyn event loop:

import talyn
import asyncio

async def main():
    print("Hello from Talyn!")
    await asyncio.sleep(1)
    print("Goodbye from Talyn!")

# Run using Talyn event loop
talyn.run(main())

Graceful Fallback & Hardening Check

For production configurations, you may want to support fallback options. If Talyn is not installed, or if the host Linux kernel does not meet Talyn's safety baseline (monitored by the HARD-01 Kernel Version Guard which requires kernel >= 6.0), the code will gracefully fall back to uvloop (if available), and finally to standard Python asyncio:

import asyncio

# 1. Try to initialize Talyn (performs HARD-01 kernel version check)
try:
    import talyn
    talyn.install()
except (ImportError, RuntimeError):
    # 2. Fallback to uvloop if Talyn is missing or kernel version is too old
    try:
        import uvloop
        uvloop.install()
    except (ImportError, AttributeError):
        # 3. Fallback to standard Python asyncio (do nothing)
        pass

async def main():
    print("Running with the best available event loop!")

asyncio.run(main())

💝 Historical Credits & Origin

Talyn is spun off from Leviathan, an event loop originally pioneered by Enrique Mora. Enrique Mora's creative spark and vision of merging Zig, io_uring, and asyncio laid the critical foundation and architecture of this project.

As Talyn evolved, the implementation underwent a complete systems-level refactoring to transition from a theoretical prototype to a production-grade, crash-resistant runtime:

  • Eliminated multi-crossing Zig/Python vectorcall overhead by implementing a fused scheduler step trampoline in pure Zig.
  • Redesigned completion handlers into flat, GC-safe ring buffers.
  • Fully audited and resolved all memory-leak reference cycles under concurrent connections.

To honor the project's roots and Enrique's early work:

📖 Project Story

  • Development Journey — The full story: from discovery to challenges, the shift from "ultra-fast" to "realistic fast and stable", and how Talyn was built.
  • Why Talyn? — The personal story and meaning behind the new name.

📄 License

This project is licensed under the MIT License. See LICENSE.md for details.

About

A robust io_uring‑driven event loop for Python AsyncIO.

Resources

Readme
MIT license
Activity

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

Packages

Contributors

Languages