Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 17 additions & 10 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,18 +18,22 @@ jobs:
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
- name: Install
run: |
python -m pip install --upgrade pip
pip install -e ".[dev]"
- name: Install uv
uses: astral-sh/setup-uv@v5
with:
enable-cache: true
# setup-uv installs uv only, not interpreters; setup-python (above)
# provides the matrix Python so `uv --system` can find it.
- name: Install (uv, pinned interpreter)
run: uv pip install --system --python ${{ matrix.python-version }} -e ".[dev]"
- name: Ruff lint
run: ruff check .
run: uv run --python ${{ matrix.python-version }} ruff check .
- name: Ruff format check
run: ruff format --check .
run: uv run --python ${{ matrix.python-version }} ruff format --check .
- name: Mypy
run: mypy agentargus
run: uv run --python ${{ matrix.python-version }} mypy agentargus
- name: Pytest
run: pytest --cov=agentargus --cov-report=xml --cov-report=term-missing
run: uv run --python ${{ matrix.python-version }} pytest --cov=agentargus --cov-report=xml --cov-report=term-missing

e2e:
# End-to-end demo runs separately and does not block the merge gate (spec §10).
Expand All @@ -39,10 +43,13 @@ jobs:
continue-on-error: true
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.11"
- name: Install uv
uses: astral-sh/setup-uv@v5
- name: Install
run: pip install -e ".[dev]"
run: uv pip install --system --python 3.11 -e ".[dev]"
- name: Run demo (placeholder until Module 10)
run: echo "e2e demo lands in Module 10"
1 change: 1 addition & 0 deletions .python-version
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
3.12
72 changes: 72 additions & 0 deletions DESIGN_LOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,78 @@ the top.

---

## Module 1 — Agents (`BaseAgent`, `Agent` facade) — 2026-07-19

### 1. What was built
- **`agents/base.py`** — `BaseAgent(ABC)`: abstract `arun(input) -> RunResult`
(the real contract) and a concrete `run()` that drives `arun` via
`asyncio.run`, refusing (with a clear error) to run inside an existing loop.
- **`agents/agent.py`** — `Agent(BaseAgent)`, the facade. Normalises any inner
target into one async callable; orchestrates a run in its final shape; hosts
the `wrap` overload (site #3).
- **`agents/seams.py`** — null-object seams (`NullTracer`, `NullCostTracker`,
`PassthroughReliability`) plus their `Protocol` contracts (`TracerSeam`,
`CostSeam`, `ReliabilitySeam`).
- **`observability/conventions.py`** — GenAI semantic-convention keys, single
source of truth (needed one key now; Module 2 extends it).
- **`logging.py`** — added `reset_trace_id(token)` (proper contextvar restore).

### 2. Why this shape
- **Async-core, sync-wraps.** All orchestration lives in `arun`; `run` just
drives it. This is the biggest reuse decision in the project — reliability,
tracing, cost, HITL are written once on the async path and sync borrows them.
Cost: `run()` inside a running loop is a controlled error, not a nested loop.
- **Composition, not inheritance, for collaborators.** `Agent` *has* a tracer /
cost / reliability; it is not one. Injected as null objects now.
- **Null-object seams over `if x is not None`.** `Agent.arun` is written in its
FINAL shape today; Modules 2/3/4/9 swap real objects in with zero edits to
`Agent`. The null classes double as the documented contract each real
collaborator must satisfy.
- **Sync callables run via `asyncio.to_thread`** so a blocking user function
never stalls the event loop; async callables are awaited directly.

### 3. Reuse points introduced
- `reset_trace_id` in `logging.py` — the sanctioned contextvar restore; `Agent`
is its first consumer.
- `observability/conventions.py` — every future span-attribute write imports its
key from here (no scattered string literals).
- The seam `Protocol`s — the single contract later collaborators implement.

### 4. methodoverload decision — site #3 (`Agent.wrap`)
Used, and it genuinely dispatches. Two hard-won findings, both now in
`docs/concepts/methodoverload.md`:
- **`from __future__ import annotations` breaks dispatch.** PEP 563 stringizes
annotations; the library does `isinstance(value, annotation)` at runtime, and
`isinstance(x, "BaseAgent")` raises. `agent.py` therefore omits the future
import. **This constrains every future overload site** (cost, dataset,
metrics) to do the same.
- **A plain method overwrites an `@overload`.** The library only merges
`@overload`-decorated siblings. So the callable "fallback" is itself an
`@overload` dispatching on `object` (matches anything), registered *after* the
`BaseAgent` overload — first-match-wins routes `BaseAgent` to its branch and
everything else to the catch-all. This is the honest resolution of spec
§4.3's callable caution: dispatch really happens, on `object` not a fictional
`Callable`. mypy can't model this runtime pattern, so the second def carries a
scoped `# type: ignore[no-redef]` with an explanatory comment.

### 5. Failure modes
- `run()` inside a running event loop raises rather than deadlocking — correct
for a library, but a caller who doesn't read the message may be surprised.
- Sync callables run in a worker thread, so they do **not** see the `trace_id`
contextvar (contextvars don't cross into raw threads without `copy_context`).
Documented; async callables see it correctly. If trace correlation inside sync
inner functions ever matters, we'll propagate the context explicitly.
- The generated `trace_id` (uuid4) is a placeholder until the real Tracer
(Module 2) supplies the span's trace id; the seam is designed for a clean swap.

### 6. The one thing most likely to be asked in review
"You run sync callables in a thread — so your trace_id contextvar silently
doesn't reach them. Isn't that a correlation hole?" Answer: yes for sync inner
functions, by design (async is the primary path); the fix (`copy_context`) is
known and cheap, deferred until a real need appears.

---

## Module 0 — Core (`RunResult`, config, logging, scaffold) — 2026-07-19

### 1. What was built
Expand Down
52 changes: 52 additions & 0 deletions HARD_QUESTIONS.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,58 @@ gate does not close until the owner can answer its batch in their own words.

---

## Module 1 — Agents (`BaseAgent`, `Agent` facade)

1. You chose "async-core, sync-wraps": `run()` calls `asyncio.run(self.arun())`.
What happens if a user calls `agent.run()` from inside a Jupyter notebook
(which already runs an event loop)? Walk through exactly what your loop-guard
does and why raising is better than the alternatives (nest_asyncio, a new
thread, `run_until_complete`).

2. Sync inner callables are executed via `asyncio.to_thread`. That means they run
in a worker thread and **cannot see** the `trace_id` contextvar. Is that a
silent observability hole? When would it bite, and what's the fix you
deliberately deferred?

3. `Agent.wrap` uses two `@overload`s — one on `BaseAgent`, one on `object`. Why
`object` and not `typing.Callable`? Prove that ordering matters: what happens
if the `object` overload is registered *before* the `BaseAgent` one?

4. `agent.py` is the only module without `from __future__ import annotations`.
Explain precisely why the future import is incompatible with your own
overload library. If a teammate "helpfully" adds it back, what breaks and how
would you catch it in CI?

5. You wrapped the real collaborators as null objects. Isn't that just
over-engineering for something that does nothing? Defend null-object over a
simple `if self._tracer is not None` — in terms of what `Agent.arun` looks
like across the next four modules.

6. `Agent` *is-a* `BaseAgent` (inheritance) but *has-a* tracer/cost/reliability
(composition). Justify each choice. Why is inheriting from `BaseAgent`
correct here but inheriting a tracer would be wrong?

7. `run()` blocks the calling thread until `arun` completes. If someone wraps a
30-second agent and calls `run()` in a web request handler, what's the
consequence? Is that AgentArgus's problem to solve?

8. The facade generates its own `trace_id` even when wrapping an inner
`BaseAgent` that already produced one. Two trace_ids now exist for one
logical run — is that a bug? How will this reconcile when the real Tracer
(Module 2) creates spans with their own trace ids?

9. You put a `# type: ignore[no-redef]` on the second `wrap`. Isn't suppressing a
type error a smell? What exactly can't mypy model here, and what would you
lose by instead not using your overload library at this site at all?

10. `arun` sets the contextvar in a `try` and resets it in `finally`. If the
inner call raises, the exception propagates but the reset still runs — trace
it. Now: two `Agent.arun` calls running concurrently in the same event loop
— do their trace_ids leak into each other? Why or why not (contextvars +
tasks)?

---

## Module 0 — Core (`RunResult`, config, logging)

1. `RunResult` is `frozen=True` and stores collections as tuples — but a `dict`
Expand Down
17 changes: 16 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,13 +12,28 @@ Wrap any agent (a callable, a LangGraph graph, a `BaseAgent`) and get, uniformly

> **Status:** early development (`0.1.0.dev0`). Built module-by-module per the [implementation spec](IMPLEMENTAION.md). See [DESIGN_LOG.md](DESIGN_LOG.md) for the decision record.

## Install
## Install (users)

```bash
pip install agentargus # minimal runtime
pip install "agentargus[dev]" # + test/lint/judge-adapter tooling
```

## Develop (with uv)

This project uses [uv](https://docs.astral.sh/uv/) for a reproducible dev
environment (`uv.lock` is committed; Python pinned in `.python-version`).

```bash
uv venv # create .venv (uses .python-version -> 3.12)
uv pip install -e ".[dev]" # install project + dev tooling into .venv
uv run pytest # run the test suite
uv run ruff check . # lint
uv run mypy agentargus # type-check
```

`uv sync` will also install straight from the lockfile once you have a `.venv`.

## Design docs

- [DESIGN.md](DESIGN.md) — problem, scope, non-goals.
Expand Down
3 changes: 3 additions & 0 deletions agentargus/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@
ConfigError,
SerializationError,
)
from agentargus.agents import Agent, BaseAgent
from agentargus.config import AgentArgusConfig, Judge, batch_complete
from agentargus.core import (
CostBreakdown,
Expand All @@ -28,6 +29,8 @@

__all__ = [
"__version__",
"Agent",
"BaseAgent",
"RunResult",
"Span",
"ToolCall",
Expand Down
6 changes: 6 additions & 0 deletions agentargus/agents/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
"""Agents: the ``BaseAgent`` contract and the ``Agent`` facade."""

from agentargus.agents.agent import Agent
from agentargus.agents.base import BaseAgent

__all__ = ["BaseAgent", "Agent"]
Loading
Loading