feat(runtimed-py): introduce Python bindings for daemon client#348
Merged
feat(runtimed-py): introduce Python bindings for daemon client#348
Conversation
3ebe9dd to
0411e8f
Compare
| return Path(os.environ["RUNTIMED_BINARY"]) | ||
|
|
||
| # Check relative to this repo | ||
| repo_root = Path(__file__).parent.parent.parent.parent.parent |
Member
There was a problem hiding this comment.
Can we use the CONDUCTOR_WORKSPACE_PATH here as a better fallback?
Member
Author
There was a problem hiding this comment.
Good idea! We'll use that in CI too.
Add Python bindings for runtimed daemon operations: - DaemonClient: pool status, ping, list rooms, flush, shutdown - Session: connect to notebook room, start kernel, execute code - ExecutionResult/Output: structured output types Uses PyO3 0.28 with pyo3-async-runtimes for tokio integration. Note: Current Session.execute() uses QueueCell shortcut which bypasses automerge doc. See design gap notes for proper doc-based execution flow.
- Add *.so to root gitignore for Python extension modules - Remove contributing/architecture.md (lives on architectural-principles branch)
Switch runtimed-py bindings to use ExecuteCell instead of deprecated QueueCell. The daemon now reads cell source from the automerge document, ensuring all connected clients see the same code being executed. Changes: - Add Cell class to expose cell info to Python - Add document operations: create_cell, set_source, get_cell, get_cells, delete_cell - Add execute_cell() using ExecuteCell (reads from doc) - Add run() convenience method (create + execute) - Fix sync task exit bug by storing sync_rx in SessionState - Add timeout to recv_frame_any() to prevent blocking - Add biased select! to prioritize commands over polling
Add comprehensive integration tests for the document-first execution pattern. Tests cover: - Basic connectivity to daemon - Document operations (create/update/get/delete cells) - Cell execution via ExecuteCell (reading from automerge doc) - Multi-client synchronization (two sessions sharing a notebook) - Kernel lifecycle (start/interrupt/shutdown) - Output types (stdout/stderr/display_data) - Error handling Supports two modes: - Dev mode: uses existing daemon via `cargo xtask dev-daemon` - CI mode: spawns isolated daemon with log capture 24 tests covering the core document-first architecture.
Add a new job to the build workflow that runs the runtimed-py integration tests. The job: - Builds runtimed binary and runtimed-py Python bindings - Runs 24 integration tests in CI mode (spawns isolated daemon) - Uploads test logs as artifacts for debugging Tests verify document-first execution, multi-client sync, kernel lifecycle, output capture, and error handling.
Comprehensive guide for the runtimed Python package covering: - Session API for code execution - DaemonClient for low-level operations - Document-first execution pattern - Multi-client scenarios - Result types (ExecutionResult, Output, Cell) - Sidecar launcher for rich output
1. Socket path mismatch (Issue #1): Session.connect() now respects RUNTIMED_SOCKET_PATH env var, and test fixtures set it when spawning daemon in CI mode. 2. Nested timeouts (Issue #2): Remove outer 10ms timeout wrapping recv_frame_any() since it already has internal 100ms timeout. This prevents repeatedly canceling mid-read. 3. sync_rx not drained (Issue #3): Use try_send() instead of send().await for changes channel. If receiver isn't keeping up, skip the update rather than blocking. Python bindings keep sync_rx alive but don't consume it. 4. Parse failure semantics (Issue #4): When output_type is "error" but parsing fails, create an error Output to preserve success=false semantics. 5. CONDUCTOR_WORKSPACE_PATH (Safia's comment): Use env var as preferred repo root fallback before walking up parent directories.
3679986 to
f4454c6
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Introduces the
runtimedPython package - PyO3 bindings that enable programmatic notebook execution from Python. This is the foundation for agent tooling, MCP integrations, and automation workflows.What's included:
New Components
Rust Crate:
crates/runtimed-py/DaemonClientSessionCellExecutionResultOutputRuntimedErrorPython Package:
python/runtimed/API Examples
Session (code execution)
DaemonClient (low-level ops)
Document-First Execution
The bindings implement the architectural principle that execution reads from the automerge document rather than receiving code directly:
session.create_cell(source)- Creates cell in automerge docsession.execute_cell(cell_id)- Daemon reads source from docThis enables multi-client scenarios where two Python sessions (or a Python session + the notebook app) can share a notebook and see each other's changes.
Integration Tests
24 tests covering:
Test plan
runtimed-py-integrationjob)