docs(blog): add Beyond Function Calling - #4650
Merged
Merged
Conversation
Generated-by: Codex
Astro-Han
reviewed
Sep 3, 2026
Astro-Han
left a comment
Astro-Han
left a comment
Contributor
There was a problem hiding this comment.
Hi @likun666661,
Thanks for putting together this deep dive into Maka's tool architecture! The essay covers critical engineering mechanisms, especially the T1/T2 dispatch boundary, resource authority decoupling from #4542, and disaggregated session state over object storage.
I did a comprehensive polish pass across both the English and Simplified Chinese versions to align with our project's technical writing standards:
- Zero em-dashes and weakened hyphens across both versions.
- Eliminated formulaic "not X, but Y" false dichotomies in favor of direct engineering statements.
- Replaced anthropomorphic metaphors (e.g. "growing hands and feet", "sleeping in S3") with precise systems concepts (the Action Boundary, Ephemeral Compute vs Durable State, on-demand rehydration).
- Streamlined the conclusion into a technical summary of the runtime stack rather than sloganized bullet points.
- Mirrored line-by-line parity (both versions now have exact matching sections, diagrams, and paragraphs).
You can review or copy the revised drafts directly from the expandable blocks below:
docs/blogs/beyond-function-calling.md (English)
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
[简体中文](./beyond-function-calling.zh-CN.md)
# Beyond Function Calling: How Agents Reach the Real World
## Deferred Tools: Even an Unused Tool Has a Cost
In standard programs, an uncalled function incurs negligible runtime overhead. It can sit in a source repository or a dynamic library without consuming CPU cycles or occupying stack frames.
Tools in an agent architecture behave fundamentally differently.
Before a large language model can invoke a tool, it requires explicit awareness of the tool name, behavioral description, and structured parameter schema. Consequently, the runtime must transmit these definitions alongside the system prompt and conversation history on every request. Even if a tool is never triggered during an entire session, its schema continually consumes input tokens across every inference step.
Tool overhead begins long before execution starts. Schemas occupy scarce context windows, dilute model attention during planning, and degrade prefix cache reuse across provider endpoints. As registries expand, the broader action space comes at the expense of task-specific context and introduces higher decision variance.
When an agent exposes only elementary tools like `Read`, `Write`, and `Bash`, the overhead remains manageable. Once the registry includes browser drivers, OS automation routines, subagent delegations, enterprise services, and dozens of Model Context Protocol (MCP) connectors, keeping all schemas resident across all requests breaks scalability.
Maka addresses this limitation through Deferred Tools. The mechanism does not alter execution timing; its purpose is to control when complete schemas become visible to the model.
The runtime continuously retains all registered tool bindings for the active run. However, the initial request exposes only high-frequency primitives alongside a compact `tool_search` utility. Extended tools register solely by name and category in a lightweight inventory, omitting full descriptions and parameter schemas.
```text
Bound Tool Registry
│
├──── Direct Tools ───────────────→ Full schemas in this request
│
└──── Deferred Tools
│
└──── Lightweight Search Inventory
│
tool_search
│
Bounded matches
│
▼
Next provider step
injects matched schemas
```
The `tool_search` utility performs local lookups across capabilities already registered with the runtime. Maka matches queries against tool names, descriptions, and functional categories, returning a size-bounded candidate set. The payload returned to the model contains only the activated tool identifiers. Full schemas are never dumped directly into the tool result payload; they are injected into the subsequent model turn through standard tool projection.
In Maka, tool state is organized into three distinct tiers:
- **Bound:** The runtime possesses an executable implementation, defining the absolute capability ceiling of the run.
- **Discoverable:** The tool is cataloged in the lightweight inventory, making the model aware of its availability.
- **Visible:** The complete schema is injected into the active provider request, enabling the model to construct valid calls.
Capability discovery does not introduce unregistered implementations or exceed the binding ceiling. It serves exclusively to reshape the tool projection presented to subsequent inference steps.
Step boundaries enforce strict temporal separation. Once a provider step is dispatched, its schema set is immutable. If a model generates the following sequence within a single completion:
```text
tool_search("browser click")
browser_click(...)
```
Maka rejects the second call. Results from `tool_search` apply only to subsequent provider interactions; they cannot retroactively amend schemas already committed to the provider. The complete definition of `browser_click` enters context in the next step, allowing the model to construct arguments against a validated interface.
Deferred activation is strictly scoped to the active turn. Discovered tools accumulate monotonically across retries within the turn, and release upon completion. Subsequent user turns reset to the baseline tool set, preventing intermittent tool usage from permanently burdening long-term inference context.
Visibility does not equate to authorization. A visible schema still requires parameter validation, concurrency checks, and permission gates upon invocation. The `tool_search` mechanism regulates cognitive surface area; system safety remains the sole responsibility of runtime enforcement.
Deferred Tools constrain the action space presented to the model. The runtime preserves comprehensive capabilities while exposing only task-relevant subsets per step.
## The Action Boundary: Bridging Probability and System Side Effects
Injecting a tool schema into context only informs the model of available actions. Until the model produces a tool call, the interaction remains strictly within the domain of text tokens.
Language models cannot directly manipulate host systems. They consume input sequences and predict subsequent tokens. Emitting the statement "I have updated the configuration" does not alter any byte on disk. A physical boundary separates descriptive language from concrete system state.
Tool calls bridge this boundary. The model ceases freeform generation and emits a structured action intent specifying the target tool, call arguments, and a correlation identifier (Call ID). The runtime intercepts this intent, executes the real operation within a sandboxed environment, and returns observed outcomes to the model.
```text
LLM
│
│ function_call(name, arguments, call_id)
▼
Runtime
│
├── Resolve tool binding
├── Validate arguments and execution bounds
├── Request required permissions
├── Invoke concrete system operation
▼
Filesystem / Process / Browser / Network / Human
│
│ function_response(call_id, result)
▼
Next LLM inference step
```
This feedback loop allows the model to interact with external environments. File reads inspect workspace state, command executions capture compiler and test diagnostics, file modifications update working trees, network calls interface with external services, and user interaction tools pause for clarification.
Tool results provide the empirical ground truth for subsequent decisions. Without observational feedback, models cannot verify whether operations succeeded or correct invalid assumptions. An end-to-end agent step consists of a closed loop across reasoning, dispatch, and observation:
```text
Reason → Act → Observe → Reason
```
This interaction differs fundamentally from regular software invocation. Conventional programs link callers and callees inside deterministic execution environments. Tool calls generated by language models represent probabilistic action proposals. Arguments may be malformed, environmental preconditions may be stale, and assumptions regarding system state may be incorrect.
Language models do not possess ambient execution authority. External side effects occur strictly through runtime arbitration, validation, and policy checks.
In Maka, invocations face rigorous validation before dispatch: bindings are checked, turn visibility is verified, parameter schemas are enforced, and concurrency policies are applied. Underlying system implementations execute only after all checks pass.
This boundary decouples model intent from system authorization. The model possesses proposal authority; producing syntactically valid JSON cannot grant environmental privileges. Tool schemas define the wire format for proposals, bindings register available capabilities, and runtime permissions arbitrate individual calls.
Upon completion, the runtime normalizes external payloads into provider-neutral tool results, paired via stable Call IDs. Within Maka's `RuntimeEvent Log`, these events are committed as immutable `function_call` and `function_response` entries, establishing an auditable factual foundation for replay and crash recovery.
Call IDs serve as architectural anchors. When an agent dispatches multiple concurrent calls, disparate I/O latencies shuffle completion order. The runtime relies on deterministic identifiers to route results back to their respective causal chains and preserve structural topology during replay.
Tool calls transform model output from human-directed prose into system invocations with irreversible side effects. The runtime must therefore implement rigorous engineering boundaries to manage external consequences safely.
## Reliable Execution: Crash Recovery Over Committed History
Introducing external side effects exposes the agent runtime to real-world infrastructure failures.
Consider a scenario where a model calls `Edit` to update a port from `3000` to `4000`. The disk write completes, but the host process loses power immediately afterward. Upon restart, the runtime observes a dangling call without an associated result. This absence does not mean the filesystem remained untouched.
A missing tool result can signify several conflicting states: the call was never dispatched, execution is still in progress, disk writes succeeded while metadata commits failed, or external processes modified state post-write. Blindly re-executing such calls risks duplicate writes, redundant financial transactions, or persistent data corruption.
Unlike text generation, external system actions cannot be assumed nonexistent simply because the runtime missed the return signal.
Maka encloses every external tool invocation within a lightweight two-phase persistence boundary:
```text
Model generates function_call
│
▼
Validate parameters, visibility, permissions, and bounds
│
▼
T1: Commit Tool Dispatch
│
▼
Execute real-world operation
│
▼
T2: Commit function_response
│
▼
Deliver Tool Result to model
```
T1 signifies that all pre-flight validations passed and execution crossed the dispatch threshold. From this point forward, the runtime cannot safely assume the operation never occurred. T1 must commit before concrete implementations are invoked; if T1 persistence fails, external actions remain blocked.
T2 certifies that execution results have committed as an immutable `function_response` event. Only after T2 commits may the outcome enter subsequent model inference steps. Even if an external operation succeeds, missing T2 persistence prohibits feeding unverified state into the active reasoning loop.
Maka avoids distributed database transactions across external systems. File I/O, shell tasks, browser drivers, and network requests exhibit wide variance in latency, making global ACID transactions impractical. Maka uses two localized storage transactions to bound the external side-effect window:
```text
Committed T1 → External Side Effect → Committed T2
```
When unexpected crashes occur, recovery logic derives exact status from the append-only event prefix:
| Log State | Recovery Disposition |
|---|---|
| No T1 committed | Operation never dispatched; safe to discard or re-evaluate |
| Both T1 and T2 present | Operation completed; reuse committed result without re-execution |
| T1 present, T2 missing | State indeterminate; force reconcile or park |
| Broken ID causality or ordering conflicts | Ledger corrupted; fail closed |
The interval between T1 and T2 represents the critical failure window. The system knows dispatch was authorized, but cannot confirm external completion. Maka prohibits speculative guessing and never defaults missing outcomes to failure. Tool bindings declare specific recovery policies: natural idempotency, queryable status checks, or strict prohibition of automatic retries. When definitive evidence is lacking, the runtime parks the operation, awaiting automated probes or operator intervention.
Recovery operations remain append-only. The runtime never edits prior `function_call` events or fabricates missing history. Dispatches, outcomes, reconciliations, and operator decisions append to the log tail as new facts. Historical facts remain immutable; subsequent events record how dangling operations converged.
Resume routines initiate fresh execution cycles only after all pending operations resolve to Completed or Definitely Not Dispatched.
Replay follows strict architectural boundaries. Maka never re-runs historical tool implementations, nor does it resurrect transient in-memory objects, unresolved promises, or dropped sockets. Replay reconstructs verified causal history: user inputs, reasoning traces, and paired `function_call` and `function_response` events.
```text
Immutable RuntimeEvent Prefix
│
├── Resolve and converge tool states
├── Strip transient streaming chunks
├── Retain paired Call / Response events
├── Prune uncommitted dangling suffixes
└── Verify High-Water mark and Digest
│
▼
Verified Provider Replay Plan
│
▼
Fresh Run / Invocation Instance
```
Leveraging append-only logs, recovery operates independently of volatile memory dumps. The runtime reads the immutable event slice up to the recorded high-water mark, validates its cryptographic digest, and projects canonical context for the subsequent step.
The resumed instance receives distinct Run and Invocation identifiers, noting its parent run and high-water anchor. Original user prompts are not duplicated, and finished operations do not re-run. The continuation inherits verified historical facts rather than an imperative re-execution script.
Before dispatching model requests, Maka verifies fundamental environmental invariants: matching workspace paths, active tool bindings, converged background tasks, and absence of conflicting recoveries. If any condition cannot be confirmed, resume aborts to a parked state, preventing execution within compromised environments.
Maka crash recovery reconstructs execution from verified append-only history, rather than attempting to resurrect volatile process state.
## Code Mode: Programmatic Orchestration and Folded Call Trees
Standard tool calling adheres to a sequential turn pattern: the model proposes an action, the runtime executes it, and the model re-evaluates the prompt. For workflows requiring continuous semantic reasoning at every step, this pattern provides necessary control.
However, for deterministic data transformations, this round-trip structure creates severe latency and token overhead.
Consider multi-package dependency audits: an agent must traverse dozens of directories, inspect `package.json` files, extract version constraints, and report discrepancies. Under sequential tool calling, the agent repeats dozens of inference cycles: generating read requests, waiting for file contents, parsing results, and emitting subsequent calls. Raw file contents flood the context window, and model round trips compound latency.
```text
Reason → Call → Observe → Reason → Call → Observe → ...
```
In these workflows, reasoning is essential for initial planning and final error analysis, while intermediate steps involve deterministic control flow. Forcing models to emulate loops and string parsers incurs unnecessary inference cost and pollutes context with intermediate noise.
Code Mode replaces discrete invocations with programmatic orchestration.
Instead of emitting fragmented tool calls, the model produces an executable program. Iteration, concurrency, branching, parsing, and aggregation execute inside a sandboxed interpreter. The model receives only the final structured output.
```text
┌─ Tool A ─┐
Reason → Program ─┼─ Tool B ─┼→ Filter / Join / Reduce → Observe → Reason
└─ Tool C ─┘
```
Implementations vary across ecosystem providers: OpenAI exposes Programmatic Tool Calling within the Responses API, executing model-generated JavaScript in a secure V8 environment with access to `tools.*`; Anthropic allows Claude to execute Python scripts within a containerized environment, calling whitelisted tools programmatically.
Both approaches share common architectural principles: delegating non-deterministic planning to the model while offloading deterministic control flow to an execution engine.
Sandboxes operate under strict isolation. Code executed within the container accesses only tools explicitly surfaced by the runtime. Writing custom network or filesystem logic cannot bypass runtime permissions. The script acts as an orchestration layer, not an escalation of privilege.
Code Mode does not displace standard tool mechanisms. Instead, it reorganizes linear call sequences into a hierarchical call tree: the root node contains the program payload, while branch nodes represent concrete tool calls issued by the script. Each leaf operation must still pass through runtime validation, permission gates, and transaction boundaries.
```text
Program / exec
├── Tool Call 1
├── Tool Call 2
│ └── Tool Result 2
└── Tool Call 3
└── Tool Result 3
│
▼
Program Result
```
Folding invocations into trees yields two primary benefits: it minimizes round-trip inference steps, and it shields the context window from intermediate telemetry. The sandbox absorbs raw operational payloads, returning only consolidated summaries to the outer context. Full operational details are preserved in audit logs without consuming inference memory.
Programmatic orchestration should not be applied universally. Irreversible side effects, actions requiring human authorization, or workflows where subsequent steps depend on unstructured semantic observations benefit from explicit top-level tool calls. Code Mode is designed for deterministic data pipelines, not for concealing agent decisions.
Maka enforces clear operational bounds within Code Mode. The model submits JavaScript cells via an `exec` primitive, restricted to registered tools marked for nested invocation. The execution environment lacks ambient OS capabilities, constrained by quotas on execution time, memory usage, script size, response size, and concurrency.
Crucially, nested invocations within a cell route through the central `ToolRuntime`. Validation, permission evaluation, and T1/T2 transactions apply uniformly. Maka assigns discrete identifiers to nested calls, maintaining parent-child links with the host `exec` event.
Nested calls retain durable persistence semantics without inflating the model prompt. They commit to the `RuntimeEvent Log` with `modelVisibility: hidden`, while the model sees only the outer `exec` boundary and its aggregated result. Maka preserves complete factual history in storage while projecting a clean abstraction for inference.
Crash recovery also accounts for programmatic execution. A script may execute three nested operations before crashing on the fourth. Re-running the entire script upon recovery would duplicate completed side effects. Maka prohibits automatic retries of unfinalized `exec` cells. Settled nested operations remain in the log, while the outer cell marks an interrupted state, leaving resumption strategy to subsequent model evaluation.
Script environments are ephemeral, yet every nested tool invocation crossing the system boundary remains durably logged and auditable.
## Parallel Tool Execution: Decoupling Task Concurrency from Resource Authority
Agents can emit concurrent tool calls within Code Mode programs or output parallel invocations in a single standard completion step.
Parallel tool calling requires clear architectural definition. When a model outputs a batch of calls in one step, it does so without observing any interim results. Therefore, invocations within that batch cannot possess causal data dependencies on each other.
If an operation depends on data produced by another call, it must be scheduled in a subsequent reasoning step.
```text
Single Assistant Step
┌── Tool Call A ──→ Result A ──┐
Model ──┼── Tool Call B ──→ Result B ──┼──→ Next Model Step
└── Tool Call C ──→ Result C ──┘
Fan-out / Fan-in
```
From an architectural standpoint, batch calls mirror asynchronous I/O primitives. Each tool call is handled as an independently awaitable task. The runtime avoids thread blocking, advancing concurrent tasks until external filesystems, processes, or networks respond. Once the entire batch settles, the runtime aggregates results for the next inference step.
This structure allows independent wait states to overlap. A slow web query does not delay local file inspection or subagent execution. Overall latency converges toward the critical path rather than the sum of independent operations.
However, an absence of data dependencies does not guarantee an absence of resource conflicts.
A model may emit `Read(a)` and `Edit(a)` within the same batch, or instruct multiple tools to update shared session state simultaneously. While neither call consumes the other's return value, both contend for identical physical resources. Handing such batches directly to uncoordinated primitives like `Promise.allSettled()` introduces race conditions governed by nondeterministic execution timing.
Maka addresses this challenge in [PR #4542](https://github.com/apache/maka/pull/4542): the runtime must maximize independent I/O parallelism while guaranteeing deterministic ordering for conflicting operations.
Centralizing all concurrency constraints inside a single Tool Scheduler introduces architectural bottlenecks. Expecting a scheduler to statically deduce read/write sets from tool arguments creates brittle abstractions that fail across dynamic host environments.
Asynchronous system design provides a clear separation of concerns: executors schedule task lifecycles, while resource authorities govern access constraints.
An executor drives ready tasks forward. Mutual exclusion, reader-writer fairness, capacity limits, and wakeup signals belong to authorities positioned beside the underlying resources: asynchronous mutexes, reader-writer locks, semaphores, or state-owning actors.
Agent runtimes follow this division:
```text
Tool Batch
│ Create tasks, allocate result slots, broadcast cancellation
▼
Resource Authority
│ Resolve identity, order, enforce exclusivity, check versions, wake
▼
Filesystem / Terminal / Browser / Session / Remote Service
```
Resource authorities must resolve true resource identity. Raw path arguments cannot reveal physical aliases: distinct paths may point to the same file via symlinks, multiple tools may manipulate the same browser tab, and separate MCP calls may target the same remote session. Only the authority directly managing the resource can arbitrate true contention and commit order.
Batch schedulers reduce unnecessary contention, but cannot serve as the sole source of safety. Schedulers cannot enforce exclusivity across independent turns, concurrent subagents, or external system processes. Safety must close at the resource authority layer.
Different resource types require tailored synchronization models:
- **Filesystems:** Canonical path leases with writer-priority or read-write fairness.
- **Terminals and Browsers:** Single-state actors enforcing strict sequential operations.
- **External APIs and MCP Servers:** Counting semaphores regulating concurrency and request quotas.
- **Versioned Session State:** Optimistic concurrency control via Compare-And-Swap (CAS) on revisions.
These patterns share an asynchronous lifecycle without forcing heterogeneous resources into a single locking model.
This separation clarifies the distinction between resource contention and capacity limits:
- Resource contention determines whether operations can safely execute concurrently without corrupting state.
- Capacity limits determine how many concurrent operations the infrastructure can support.
Treating upstream API rate limits as a global mutex introduces head-of-line blocking, allowing slow network calls to stall unrelated disk reads. Asynchronous runtimes should restrict blocking strictly to genuine physical conflicts, keeping independent work unhindered.
When batch invocations contend for identical resources, the model's generated array order acts as a deterministic tie-breaker. This sequence establishes prioritization during contention, but does not represent causal data flow.
Parallel execution involves four distinct temporal sequences:
```text
Model Generation Order
≠ Task Start Order
≠ Task Completion Order
≠ Runtime Event Arrival Order
```
Independent tasks start and complete out of order. Raw execution events commit to the log as they occur, linked through Tool Call IDs, while payloads returned to the provider reassemble to match original prompt ordering. Historical logs preserve physical facts, while context projection satisfies model protocol requirements.
Aborts and timeouts adhere strictly to structured concurrency rules. Queued tasks that are canceled must not begin execution; tasks that have crossed T1 dispatch cannot simply be abandoned. The runtime must await their convergence and record final dispositions. The batch manager maintains ownership across child tasks, ensuring every operation completes, aborts, or reaches a verifiable state before the next inference step begins.
## Sandboxes, Serverless, and Disaggregated State
Tool invocations must ultimately execute on concrete computing infrastructure.
Models generate action plans and programs coordinate control flow, but operating system processes, memory spaces, and network interfaces require physical or virtual resources. Execution targets span a broad spectrum: lightweight JavaScript V8 isolates, Python container environments with data science toolchains, and full MicroVMs with dedicated kernels and hardware virtualization.
```text
LLM Generates Intent
│
▼
Agent Runtime
│ Select execution environment and capabilities
▼
┌──────────┬──────────────┬─────────────┐
│ V8 │ Python │ MicroVM │
│ Program │ Data/Scripts │ Full OS Tool│
└──────────┴──────────────┴─────────────┘
│
▼
Filesystem / Process / Network / Browser
```
Heavier execution environments carry distinct trade-offs. Booting a full virtual machine for basic string manipulation introduces unnecessary latency, while running untrusted shell scripts directly within the host process creates severe security risks. Runtimes must dynamically match tool requirements against lightweight, securely isolated substrates.
Sandboxes define more than security perimeters; they establish resource, failure, and lifecycle boundaries for agent execution.
Runtimes enforce strict quotas at the sandbox layer: limiting CPU, memory, storage, concurrency, and execution time; restricting network domains; and terminating environments upon memory exhaustion or process failures to prevent systemic instability.
Sandboxing also enables decoupling agent state from host infrastructure.
Conventional applications assume long-running local processes. In contrast, modern agent environments treat compute substrates as disposable: V8 cells terminate upon completion, containers recycle after idle timeouts, and MicroVMs drain during host migrations. Binding persistent agent state to ephemeral compute nodes undermines system reliability.
Append-only logging provides the foundation for this separation.
Conversation traces, tool calls, results, permission records, and recovery events reside in durable logs. Large artifacts and binary outputs persist in object storage, while workspaces mount via copy-on-write snapshots or persistent volumes. Sandboxes act as stateless execution engines, disposable and recreatable across nodes.
```text
Durable State Ephemeral Compute
RuntimeEvent Log ─┐ ┌─ V8 Isolate
Artifact Storage ─┼─→ Rehydrate ────┼─ Container
Workspace Snapshot┘ └─ MicroVM
Preserves "What happened" Executes "Next action"
```
Agent workloads are bursty: sandboxes sit idle during model reasoning, followed by intense spikes during compilation or batch processing. Certain tools run in milliseconds, while others block for hours on external feedback. Modern infrastructure must support rapid scaling to zero during idle periods, provisioning specialized capacity only when invoked.
This differs from traditional Function-as-a-Service (FaaS) abstractions. Standard serverless functions assume brief, stateless execution; agents maintain stateful workspaces, spawn long-running background tasks, pause for human review, and resume hours later.
Agent Serverless decouples session state entirely from compute lifecycle.
When a sandbox terminates, the runtime does not attempt to reconstruct volatile process heaps or unresolved sockets. Instead, it inspects durable logs, rehydrates workspace snapshots into a newly provisioned sandbox, and resumes execution from a verified factual history.
Security architectures also benefit from this model. Disposable sandboxes do not hold static administrative credentials or broad network access. They receive short-lived, minimum-privilege capabilities per task. Credential storage and policy authorization remain within the trusted runtime outside the container. If a sandbox environment is compromised, its authorization scope expires immediately upon termination.
Affordable compute substrates enable fleet-scale agent deployments. A single session can provision lightweight V8 isolates for script orchestration, Python containers for data analysis, and MicroVMs for software compilation, terminating resources as each task completes.
The natural extension of this architecture is complete state disaggregation: persisting session state in cost-effective object storage (such as S3-compatible systems) while compute executes across on-demand, stateless workers.
Sessions cease to correlate with static processes or host directories. They exist as collections of durable objects: append-only event segments, binary artifacts, workspace snapshots, compaction projections, and manifest metadata pointing to current commit boundaries. Between interactions, sessions persist passively in object storage at minimal cost.
```text
Cost-Effective Object Storage (S3)
Session A ── Events / Artifacts / Workspace Snapshots ─┐
Session B ── Events / Artifacts / Workspace Snapshots ─┼── S3
Session C ── Events / Artifacts / Workspace Snapshots ─┘
│
External Event / User / Schedule
│ │
▼ │
Rehydrate Session Context ◀───┘
│
┌───────────┼───────────┐
▼ ▼ ▼
V8 Python MicroVM
│ │ │
└───────────┼───────────┘
│
Append Facts
│
└──────────────→ S3
```
Long-running agents no longer require persistent, dedicated servers.
They remain dormant most of the time. When messages arrive, webhooks trigger, or schedules elapse, the control plane reads the session manifest, mounts the required log prefix and workspace snapshot, and provisions an appropriate sandbox. Once execution settles, new facts sync back to storage, and compute resources release immediately.
The architecture organizes into two coordinated tiers:
- **Data Plane:** Object storage managing immutable, high-volume event logs and filesystem snapshots.
- **Control Plane:** Low-latency storage tracking authoritative head pointers, resource leases, quotas, and pending operations.
This mirrors disaggregated database architectures. Object storage provides durable, cost-effective persistence, while compute resources provision strictly on demand. Context assembly operates like a materialized view query: the runtime reads durable state, applies compaction and result pruning, and projects bounded context for model inference.
```text
Session on S3
│
├── Projection ──→ Model Context ──→ LLM
│ │
├── Rehydrate ───→ Sandbox ──────────→ Tool Call
│ │
└──────────────── Append New Facts ◀───┘
```
Both models and sandboxes function as interchangeable compute utilities.
Model selection scales with reasoning complexity, and sandbox sizing matches workload requirements. A session is never coupled to a single model provider or execution substrate.
Cost efficiency stems from ensuring dormant sessions consume zero active compute.
Disaggregated storage also facilitates spot-instance execution and instantaneous branching. Using append-only logs and copy-on-write snapshots, subagents fork from parent histories without copying storage, writing only delta records going forward.
The future of agent runtime engineering is clear: establishing immutable logs as the authoritative source of truth, relying on disposable sandboxes for safe execution, decoupling resource governance from task scheduling, and dynamically managing schema projections to preserve model focus. Extending language models into external systems requires robust runtime engineering to ensure safety and reliability.docs/blogs/beyond-function-calling.zh-CN.md (中文)
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
[ENGLISH](./beyond-function-calling.md)
# Tool Call 不只是 Function Calling：Agent 如何真正触碰现实世界
## Deferred Tool：没有被调用的 Tool 也有成本
在普通程序中，未被调用的函数几乎不产生运行时开销。它可以静态存放于代码库或动态链接库中，只要执行路径不经过，就不会消耗 CPU，也不会占用调用栈。
Agent 中的 Tool 则完全不同。
大语言模型若要调用某个工具，必须预先获知该工具的名称、用途描述以及结构化的参数定义。因此，Runtime 需要将 Tool Definition 连同 System Prompt 与会话历史一同打包发送给模型。一个工具即使在整个会话中从未被触发，其描述文本和 JSON Schema 也会持续消耗每一次推理的输入 Token。
工具的开销从执行前就已经产生。Schema 挤占宝贵的上下文窗口，分散模型对核心任务的注意力，并可能改变 Provider 侧 KV Cache 的命中前缀。随着可用工具数量的膨胀，模型的行动空间虽然得以拓展，但留给任务本身的有效上下文被不断压缩，动作决策面临的噪声干扰也显著增加。
当 Agent 仅配置 `Read`、`Write`、`Bash` 等基础工具时，这种开销尚可接受。然而，一旦接入浏览器自动化、系统级 GUI 操作、子 Agent 委派、外部企业服务以及大量 MCP Connector，在每一次请求中常驻全量 Tool Schema 将迅速突破架构的可扩展性上限。
Maka 引入 Deferred Tool 机制来应对这一瓶颈。Deferred Tool 并不改变工具的执行时机，核心在于按需控制 Tool Schema 暴露给模型的时间点。
Runtime 始终完整持有当前 Run 所注册的全部 Tool Binding，但在初次调用模型时，仅向其暴露高频基础工具集以及一个专用的轻量级 `tool_search` 工具。其余扩展工具仅以能力分组和名称的形式登记在检索清单（Search Inventory）中，不携带详细的描述文本与参数 Schema。
```text
Bound Tool Registry
│
├──── Direct Tools ───────────────→ 当前请求中的完整 Schema
│
└──── Deferred Tools
│
└──── 轻量 Search Inventory
│
tool_search
│
有界的匹配结果
│
▼
下一次 Provider Step
注入匹配 Tool 的 Schema
```
`tool_search` 面向 Runtime 已注册的内部能力执行本地检索。Maka 依据工具名称、描述语义及所属分类完成本地过滤，提取体积受限且条目数量有界的一组候选结果。返回给模型的内容仅包含被激活工具的名称标识，完整的 Schema 规范不会直接倾倒进 Tool Result，而是在下一次向 Provider 发起请求时，以标准的 Tool Projection 格式透明注入。
在 Maka 中，工具状态严格划分为三个层级：
- **Bound**：Runtime 已完成工具绑定的可执行实现，确立当前 Run 的能力上限。
- **Discoverable**：工具登记于轻量级 Inventory 中，模型感知到该能力的存在。
- **Visible**：完整 Tool Schema 已注入当前 Provider 请求上下文，模型具备构造合法调用的必要信息。
能力检索不会动态引入未经注册的实现，无法突破当前 Run 固有的 Binding 上限，它仅用于动态调整下一次模型交互所见的 Tool Projection。
这里的“下一次”构成了严格的时序边界。Provider Step 一旦建立，本次请求携带的 Tool Schema 集合即行固化。若模型在单次回复中同时生成如下调用序列：
```text
tool_search("browser click")
browser_click(...)
```
Maka 会明确拒绝执行第二个调用。`tool_search` 的生效结果只能投影至后续请求，绝不能反向修改已经交由 Provider 解析的 Schema 集合。只有推进至下一个推理步骤，`browser_click` 的完整规范才会进入上下文，模型方可基于实际观测到的接口定义生成准确参数。
Deferred Tool 的激活范围被严格限定在当前 Turn 之内。在同一次用户交互轮次中，搜索激活的工具单调累积，Provider 侧的重试请求亦完整继承该工作集；一旦 Turn 执行结束，激活集合即刻释放。下一轮交互重新从基线工具集启动，避免偶发性调用的 Schema 成本永久滞留在后续的推理历史中。
工具的可见性亦不等同于执行授权。即使 Schema 已完全可见，真实的调用请求仍须通过参数类型校验、并发配额审计以及环境权限判定。`tool_search` 仅管理模型的认知视野，系统安全边界始终由 Runtime 统一把关。
Deferred Tool 专注于约束进入模型注意力窗口的能力集合。Runtime 维护全局能力空间，模型则按需获取当前任务所需的紧凑子集。
## 意图与边界：从概率生成到系统副作用
Tool Schema 进入上下文后，模型仅获知了可选操作的规范定义。在真正发出 Tool Call 之前，所有交互仍处于纯文本 Token 的范畴。
大语言模型本身不具备直接操作宿主环境的能力。它接受上下文输入，预测下一个 Token；即便输出“已成功修改文件”，磁盘上的实际数据也不会发生任何改变。文本表述与客观系统状态之间存在物理隔离。
Tool Call 建立了跨越这一隔离的控制通道。模型停止输出自然语言，转而依据 Schema 生成结构化的动作意图：包含目标工具名称、调用入参以及用于因果追踪的 Call ID。Runtime 拦截该结构化意图，在受控环境中代为触发实际系统的操作，并将执行得到的客观观测回传给模型。
```text
LLM
│
│ function_call(name, arguments, call_id)
▼
Runtime
│
├── 查找 Tool Binding
├── 校验参数与执行边界
├── 请求必要的执行权限
├── 触发真实环境操作
▼
Filesystem / Process / Browser / Network / Human
│
│ function_response(call_id, result)
▼
LLM 的下一次推理
```
通过这一反馈闭环，模型得以介入外部系统。读取文件获取代码仓的客观状态，执行命令获取编译器与测试套件的即时反馈，修改文件改变工作区结构，网络调用接入外部服务，人机交互工具则在关键决策点获取外部确认。
Tool Result 构成模型感知外部环境的观测凭据。缺失观测反馈，模型无法评估操作的实际效果，也无法修正认知偏差。一个完整的 Agent 执行步由意图生成、系统执行与结果观测三部分闭环构成：
```text
Reason → Act → Observe → Reason
```
这一循环与普通函数调用存在本质区别。常规程序中的调用方与被调方通常运行在确定性受控的运行时中；而 LLM 生成的 Tool Call 本质上是基于概率分布给出的动作建议。参数可能残缺，环境前提可能已经失效，模型对当前系统状态的假设亦可能存在偏差。
LLM 本身不具备环境执行能力；所有的外部交互都由 Runtime 经过校验和鉴权后代为发起。
在 Maka 中，模型产出的调用请求必须经由 Runtime 屏障的严格审查：核实工具绑定有效性、确认当前 Step 可见性、比对参数 Schema、执行并发与权限策略。唯有全部条件满足，真实的底层操作才被允许触发。
这条边界清晰分离了模型意图与系统授权。模型拥有提议权，但生成合法的调用报文并不能凭空赋予自身系统权限。Tool Schema 规范了意图的表达格式，Tool Binding 定义了 Runtime 的实现能力，Permission 机制则裁定特定调用的合法性。
执行完成后，Runtime 将环境返回的原始负载规范化为中立的 Tool Result，借助稳定的 Call ID 与原调用精确配对。在 Maka 的 `RuntimeEvent Log` 中，两者分别沉淀为不可变的 `function_call` 与 `function_response` 事件，作为审计追踪与崩溃重放的法定事实。
Call ID 不仅是消息载荷中的关联字段。在单轮多工具并发调用的场景下，各个调用的实际完成顺序受 I/O 延迟影响可能完全打乱。Runtime 必须依托确定性的标识映射，将各路结果分发至正确的因果链路，并在后续重放时复原精确的逻辑拓扑。
由此，Tool Call 实现了质的跨越：模型输出从面向人类消费的语言描述，转变为可能产生不可逆外部副作用的系统调用。这要求 Runtime 必须建立严谨的工程机制，对每一次真实动作的执行后果负责。
## 可靠执行：基于已提交历史的崩溃恢复
系统副作用的引入，不可避免地将真实世界的不确定性带入了 Agent 运行时。
设想模型调用 `Edit` 工具将配置端口从 `3000` 修改为 `4000`。底层磁盘写入刚刚完成，宿主机器遭遇突发断电。进程重启后，Runtime 观测到的仅是一条缺失对应结果的悬空调用，但这绝不意味着文件未曾被篡改。
缺少 Tool Result 对应着多种互斥的现实状态：操作尚未派发、操作仍在执行、磁盘已变更但元数据落盘失败、或者状态在变更后已被第三方进程二次覆写。若在恢复时盲目重跑该调用，极易引发重复写盘、重复扣款或数据污染等灾难性后果。
与纯文本生成不同，未决的外部系统动作绝不能因为未收到确认信号就被假设为“从未发生”。
Maka 采用轻量级两阶段持久化边界保护所有的外部工具操作：
```text
Model 生成 function_call
│
▼
参数、可见性、权限与执行边界检查
│
▼
T1：提交 Tool Dispatch
│
▼
执行真实系统操作
│
▼
T2：提交 function_response
│
▼
将 Tool Result 移交模型
```
T1 标志着 Runtime 已通过全部前置审查，正式越过调用派发点。自此，系统不再允许做出“该操作绝对未执行”的乐观假设。T1 必须在真实逻辑触发前完成持久化；若 T1 提交失败，外部副作用严禁启动。
T2 标志着工具执行结果已转化为持久化的 `function_response` 事件。唯有 T2 落盘确认，该结果才获准进入下一次模型推理。即便外部操作已成功返回，若 T2 写入失败，Runtime 亦不得将无法持久复现的状态交给模型使用。
Maka 并不试图将异构的外部操作纳入分布式数据库事务。文件 I/O、Shell 执行、浏览器渲染与远程网络调用的耗时跨度极大，强行追求全局 ACID 既不切实际亦不可行。Maka 选择以两次极短的局部数据库事务，清晰标定外部副作用的发生区间：
```text
Committed T1 → External Side Effect → Committed T2
```
当进程发生异常崩溃，恢复逻辑依据 Append-Only Log 中已固化的事件前缀做出精确的状态判定：
| 日志状态 | 恢复判决 |
|---|---|
| 未记录 T1 | 工具确定未派发，可安全忽略或重新评估 |
| T1 与 T2 均完整存在 | 工具已完成闭环，直接复用既有结果，严禁重复执行 |
| 仅存在 T1，缺失 T2 | 副作用状态处于未决区间，强制进入 Reconcile 或 Park |
| 标识乱序或因果链断裂 | 账本完整性受损，立即 Fail Closed |
处于 T1 与 T2 之间的悬空操作具有最高风险。此时系统确知操作已获得派发许可，但无法证实外部效果是否生效。Maka 严禁模型依靠幻觉猜测执行状态，亦不将缺失结果默认视作失败。Tool Binding 支持声明专属的恢复策略（例如：操作具备幂等性、支持外部状态探查、或是严格禁止自动重试）。若无法取得明确证据，Runtime 将该操作置于挂起状态（Park），交由外部探针或人工接入处理。
恢复机制严格遵循追加写原则。Runtime 不会就地篡改早先的 `function_call`，亦不凭空伪造执行记录。正常的 Dispatch、Outcome，以及后续的 Reconcile 和人工判决，均作为全新的增量事实追加至日志尾端。既有事实保持不可变，新增事实明确旧操作的最终收敛结局。
唯有当全部并发调用均被确定性收敛为已完成（Completed）或确定未派发（Definitely Not Dispatched）时，Resume 流程才获准基于已验证的历史构建新轮次。
这里的“重放（Replay）”具有严格的系统边界。Maka 从不重新触发历史工具的执行代码，亦不试图恢复崩溃前进程的瞬时内存指针、Promise 状态机或未结 Socket。它所重放的，纯粹是已经过校验的因果事实序列：用户输入、思考过程、配对完整的 `function_call` 与 `function_response`。
```text
不可变 RuntimeEvent 日志前缀
│
├── 解析并收敛 Tool 状态
├── 剥离流式临时切片
├── 保留成对的 Call / Response
├── 截断未闭合的悬空后缀
└── 校验 High-Water 与 Digest
│
▼
已验证的 Provider Replay Plan
│
▼
全新 Run / Invocation 实例
```
基于 Append-Only 的事件模型，恢复流程无须依赖易失的内存快照。系统仅读取截止至特定水位线（High-Water Mark）的不可变事件切片，校验其摘要哈希，随即向新实例提供规范的上下文投影。
恢复实例获得全新的 Run 与 Invocation 标识，明确记录其承接的源 Run 标识与水位线坐标。原有的用户指令不会发生重复拷贝，已结案的工具调用亦不会二次触发。执行链条继承的是经受验证的客观事实，而非一份重新执行的指令清单。
在拉起模型调用前，Maka 会重新验证底层环境的基线约束：确认 Workspace 物理路径未变、所需 Tool Binding 依然注册就绪、关联的后台进程处于收敛状态、且不存在并发的竞争性恢复实例。任何一项前提无法自证，恢复流程立即转入安全挂起（Park），杜绝在不一致的外部环境中继续执行。
Maka 的 Resume 逻辑不试图恢复崩溃进程的内存堆栈，而是先核实未决动作在日志中的最终收敛状态，再基于已验证的不可变历史前缀启动全新的执行轮次。
## Code Mode：程序化编排与调用树折叠
常规的 Tool Call 遵循单步交互模型：模型判定下一步动作，Runtime 触发工具并返回结果，模型基于最新上下文重新推演后续行动。在每一步骤均高度依赖动态语义决策的场景中，这种模式提供了精细的控制粒度。
然而，在处理确定性较强的组合逻辑时，这一模式的效率瓶颈十分显著。
以排查依赖冲突为例：Agent 需遍历数十个项目目录，读取 `package.json`，提取特定依赖项的版本号，最终输出存在版本分歧的项目列表。若采用逐步 Tool Call 模式，模型需要连续经历数十轮循环：生成单次读取指令、等待结果灌入上下文、重新解析后再生成下一次读取。全部中间内容不仅大幅消耗 Token 配额，亦导致多轮往返的严重网络延迟。
```text
Reason → Call → Observe → Reason → Call → Observe → ...
```
在此类场景中，除初始任务拆解与最终异常评估外，中间环节本质上属于确定性的控制流操作。由大模型反复充当代码解释器去模拟 `for` 循环与字符串过滤，不仅执行迟缓，还会将海量低价值的原始数据永久滞留在推理历史中。
Code Mode 改变了工具的组织形态。
模型不再针对每个细粒度动作生成离散的顶层 Tool Call，而是编写一段结构化的程序脚本。循环、并发抓取、条件分支、数据提取与结果聚合，均在受约束的代码沙箱内自主运行。模型最终仅需消费脚本计算后显式输出的高价值结论。
```text
┌─ Tool A ─┐
Reason → Program ─┼─ Tool B ─┼→ Filter / Join / Reduce → Observe → Reason
└─ Tool C ─┘
```
该机制在业内存在不同实现路径：OpenAI 在 Responses API 中通过 Programmatic Tool Calling 提供该能力，模型产出 JavaScript 代码并在隔离的 V8 运行环境中通过 `tools.*` 操纵工具；Anthropic 亦在 Claude 的代码执行容器中引入了类似的机制，由模型生成 Python 脚本并通过权限白名单调度受限工具。
两者的工程细节虽有差异，但遵循着一致的设计逻辑：让概率模型专注于目标规划与语义解析，由底层程序环境承载确定性的控制流编排。
代码沙箱绝非无边界的特权环境。脚本所能触达的全部能力，依然严格局限于 Runtime 所授予的工具子集。脚本无法因编写了原生系统调用就突破沙箱隔离，亦不能直接绕过既有的权限管控。程序仅扮演工具的编排逻辑层，不构成独立的安全越权来源。
Code Mode 并没有取代底层的 Tool Call 体系。相反，它将扁平的调用序列重构为层次化的调用树：根节点为模型提交的程序载荷，子节点为脚本运行时实际派发的具体工具调用。每一个叶子节点的操作，仍须完整穿透 Runtime 的校验、鉴权与事务边界。
```text
Program / exec
├── Tool Call 1
├── Tool Call 2
│ └── Tool Result 2
└── Tool Call 3
└── Tool Result 3
│
▼
Program Result
```
调用树折叠带来了显著的工程效益。首先，它大幅削减了模型交互的往返频次；原本需耗费多次采样的批量检索，在单次程序执行内即可完成。其次，它有效阻断了上下文污染：沙箱可就地消化海量原始数据，仅向外层模型上下文抛出结构精炼的最终结论。底层工具调用产生的明细未被丢失，只是其中不具推理价值的噪音被精准过滤在模型工作内存之外。
然而，程序化编排并非适用于所有场景。涉及不可逆外部副作用、需要显式人工审批、或是后续操作方向高度取决于非结构化观测结果的环节，保持显式、单步的顶层 Tool Call 能够提供更为清晰的审查线索与干预切入点。Code Mode 专为确定性计算的下沉而设计，不宜用于隐藏关键的 Agent 决策分支。
Maka 的 Code Mode 深度整合了上述边界。模型借助 `exec` 工具向系统提交包含 JavaScript 代码的单元（Cell），该单元仅允许调度当前已激活且声明支持嵌套调用的工具接口。执行容器自身剥离了直接的操作系统原生访问能力，并在执行超时、内存配额、源码长度、输出体积及并发深度等多个维度受到硬性配额约束。
在事务保证上，Cell 内部发起的每一处工具调用，均受控回调至核心 `ToolRuntime`。参数校验、权限审批以及前文阐述的 T1/T2 事务保障全量生效。Maka 会为每个嵌套动作生成独立的调用凭证，并精确记录其与宿主 `exec` 之间的父子关联。
此类嵌套操作被赋予完整的持久化语义，但不会以冗长的调用链形式直接灌入后续模型上下文。它们在 `RuntimeEvent Log` 中均标记为源自 Code Mode，并被设为 `modelVisibility: hidden`；模型仅在上下文中看到顶层的 `exec` 及其聚合产物。这体现了 Maka 一贯的架构哲学：底层日志负责记录全量系统事实，模型上下文仅作为面向推理场景的特定投影。
Code Mode 同样对崩溃恢复提出了严格要求。一段脚本可能在连续完成三项操作后，在第四项操作等待响应时遭遇崩溃。若在恢复时全量重新执行该脚本，先前已完成的系统副作用势必遭受重复触发。因此，Maka 严禁自动重跑未完结的 `exec` 任务。沙箱内已沉淀的工具结果完整封存于日志中，外层 Cell 记录明确的中断事实，交由后续推理轮次评估下一步策略。
脚本环境本质上是瞬时的，其内存栈随着沙箱销毁而湮灭；但其中每一次越过系统边界的真实工具操作，都必须作为不可变记录永久留存于底层审计账本中。
## 并发调度与资源权威分离
模型不仅可以在 Code Mode 内部并发拉起多个工具，亦能在单次常规推理输出中并列生成多个 Tool Call。
这类通常被称为并行工具调用（Parallel Tool Call）的机制需要厘清其本质特征：模型在生成这一批调用指令时，尚未接收到其中任何一项操作的实际执行反馈。因此，同一批次内的调用之间不存在基于返回结果的数据依赖。
若某个操作的前提条件依赖于另一操作的输出载荷，则该操作必须等待下一次推理步骤，而不应归入当前并发批次。
```text
同一个 Assistant Step
┌── Tool Call A ──→ Result A ──┐
Model ──┼── Tool Call B ──→ Result B ──┼──→ 下一次 Model Step
└── Tool Call C ──→ Result C ──┘
Fan-out / Fan-in
```
在 Runtime 的系统视角下，这与经典的异步 I/O 模型高度契合。每一个 Tool Call 被抽象为可独立等待的任务单元。任务派发后，Runtime 无须同步阻塞宿主线程，可继续推进其他就绪任务；底层文件系统、子进程、网络套件或外部微服务产生响应后，依序唤醒对应的调度点。待整批任务全部收敛至终止状态，Runtime 再将聚合后的结果集提交给下一轮模型推理。
该机制的核心收益在于让等待时延充分重叠。网络检索发起后，无须阻碍本地文件的读取或子 Agent 的运算；端到端的系统耗时得以从各个 I/O 延迟的代数累加，收敛至关键路径上的最大时延。
数据依赖的缺位，并不意味着物理资源冲突的消除。
模型可能在同一批次中同时发起 `Read(a)` 与 `Edit(a)`，亦可能驱动两个工具并发写入同一份会话上下文。此类操作虽无返回值的先后依赖，却直接竞争同一系统资源。若 Runtime 仅仅将批次整体交付 `Promise.allSettled()` 盲目并发，底层的读写次序与覆写结局将完全沦为由调度抖动决定的不可控竞态。
Maka 在 [PR #4542](https://github.com/apache/maka/pull/4542) 中深入探讨并明确了这一设计：必须在保障独立 I/O 充分并发的前提下，为共享同一底层资源的互斥操作确立确定性的时序控制。
将全部排他逻辑集中寄托于单一的调度器（Tool Scheduler）并非最佳实践。由中心调度器通过解析调用参数静态预判资源读写集合，虽然能够完成批次内的初步排队，但无法覆盖复杂的动态系统环境。
经典异步系统的实践提供了清晰的职责边界划分：由执行器（Executor）编排任务流，由资源权威（Resource Authority）掌管具体资源的互斥约束。
底层执行器专注于调度已就绪的任务单元；互斥仲裁、读写公平性调度、容量水位管理与唤醒逻辑，则下沉由直接管理具体资源的权威层（如 Async Mutex、RwLock、Semaphore 或独占状态的 Actor）负责裁决。
Agent Runtime 适用同样的架构分工：
```text
Tool Batch
│ 创建 Task、预留结果槽位、广播取消信号
▼
Resource Authority
│ 确认资源身份、排队、互斥排他、版本核对、唤醒
▼
Filesystem / Terminal / Browser / Session / Remote Service
```
资源身份的最终确认必须由权威层执行。入参中的路径字符串并不能代表真实的底层资源对象：不同路径可能通过符号链接汇聚至同一物理文件；不同的操作可能作用于同一个浏览器标签页；多个工具调用可能共享同一远程工作会话。唯有直接操纵物理资源的权威组件，方能确立操作间的真实冲突关系与生效时序。
若互斥保障仅维系在单次批处理的调度器内部，该机制将无法约束其他交互轮次、并发 Agent 实例或底层系统内部发起的竞争访问。安全性必须在最贴近系统副作用的层级闭环。Batch Scheduler 负责消除不必要的批内冲突并优化吞吐，但具体的互斥锁机制应下沉至资源实体。
针对不同类型的系统资源，应当采用差异化的仲裁策略：
- **文件系统**：依据 Canonical Path 构建具备写优先或读写公平的租约（Lease）控制。
- **交互终端与浏览器**：抽象为具备单一有序状态的串行 Actor。
- **外部接口与 MCP 服务**：针对 QPS 与并发上限配置容量型信号量（Semaphore）。
- **版本化会话状态**：采用基于 Revision 的 CAS（Compare-And-Swap）机制实施乐观校验。
各类资源共享统一的异步生命周期模型，但其互斥语义保持独立适配。
这亦要求在架构上明确切分“资源互斥”与“容量限额”：
- 资源互斥仲裁多项并发操作是否会破坏状态的一致性。
- 容量限额裁决系统当前所能承受的最大并发负荷。
将服务端的 QPS 限流粗暴包装为全局互斥锁，势必引发不必要的线头阻塞（Head-of-Line Blocking），导致长耗时的网络请求无端拖死完全无关的本地文件读取。异步运行时应致力于仅阻塞存在真实物理冲突的任务，确保无关联操作顺畅推进。
对于批次内确实存在物理冲突的调用，模型输出的原始数组序可作为确定性的仲裁决胜依据（Tie-breaker）。但须注意，该顺序仅表示在遭遇资源争用时仲裁所有权的先后，并不代表调用之间存在任何因果数据传递。
在并行工具调用中，系统在不同层面面临四种各异的时序状态：
```text
模型生成顺序
≠ Task 启动顺序
≠ Task 完成顺序
≠ Runtime Event 到达顺序
```
互不干扰的任务可以乱序拉起，亦能乱序终结。客观发生的操作事件依实际触发时序落入底层日志，并借助 Tool Call ID 保持逻辑因果；向模型组装回传报文时，则按模型原始请求顺序对号入座。底层日志记录物理事实，上下文装配满足模型协议，两者互为同一执行过程的独立投影。
异常中断与超时取消亦须严格遵守结构化并发（Structured Concurrency）的约束规范。处于等待队列中的任务被取消后严禁悄然启动；一旦越过 T1 派发边界的任务则严禁直接丢弃，Runtime 必须挂起并静待其状态明确收敛。顶层批处理机制对其派生的全部子任务生命周期负有终极管理责任；在下一轮模型推理拉起前，每一项并发动作必须已确切终结或转入可审计的受控状态。
## 沙箱、Serverless 与存算分离
所有工具调用终归需要在特定的物理或虚拟计算载体上运行。
模型产出结构化动作，脚本编排业务控制流，但底层 CPU、内存堆栈、文件系统与网络协议栈必须依托真实的物理环境。从快速启动的 JavaScript V8 Isolate，到承载数据处理与脚本生态的 Python 容器，再到提供完整操作系统能力、独立网络栈与编译环境的 MicroVM，不同工具所需的环境规格存在显著阶梯。
```text
LLM 生成意图
│
▼
Agent Runtime
│ 选择执行环境与 Capability
▼
┌──────────┬──────────────┬─────────────┐
│ V8 │ Python │ MicroVM │
│ 编排调用 │ 数据与脚本 │ 完整 OS Tool│
└──────────┴──────────────┴─────────────┘
│
▼
Filesystem / Process / Network / Browser
```
执行环境并非越重越好。为轻量级的文本解析动用完整虚拟机将带来不合理的开销，而将不受信任的复杂命令置于宿主进程同构运行则构成严重的安全隐患。Runtime 需依据操作所需的最小特权与资源形态，动态适配隔离强度与消耗成本均衡的执行载体。
沙箱绝非仅是防范恶意代码的隔离护栏，它同时界定了单次 Agent 执行的资源边界、故障域以及生命周期轮廓。
Runtime 可以在沙箱层面施加精细配额：限制 CPU、内存、存储用量、并发线程与最大执行窗口；限制网络访问域与外部服务白名单；并在出现内存耗尽、死循环或进程异常时迅速熔断隔离环境，阻断单点故障向整个 Agent 核心运行时的蔓延。
更核心的架构演进，在于沙箱实现了 Agent 运行时与特定计算节点的彻底解耦。
传统桌面软件通常假定宿主进程与本地环境长期稳定存续。而现代 Agent 的执行载体应当被视为随时可被回收的易失资源：V8 Cell 运行完毕即行销毁，容器在空闲超时后自动回收，MicroVM 亦可因节点迁移、负载均衡或硬件异常被随时下线。若将 Agent 的真实状态绑死在单台计算节点的易失内存中，系统的可靠性与扩展性将无从谈起。
不可变追加写日志在此构成了解耦的核心支柱。
会话流程、工具派发与响应、授权记录及恢复决议全量固化于持久化日志中；产生的文件实体、媒体流及大体积结果存入对象存储；工作区结构则依托写时复制（Copy-on-Write）快照或持久卷管理。计算沙箱退化为单纯消费状态并执行任务的无状态处理器，可被随时销毁并在任意其他节点按需重构。
```text
持久化状态 (Durable State) 易失计算 (Ephemeral Compute)
RuntimeEvent Log ─┐ ┌─ V8 Isolate
Artifact Storage ─┼─→ Rehydrate ────┼─ Container
Workspace Snapshot┘ └─ MicroVM
保存“发生过什么” 执行“下一步做什么”
```
Agent 的负载特征天生具备极高的突发性（Burstiness）：在模型深度推理阶段，计算沙箱处于完全空闲；而一旦批处理或编译任务触发，计算需求瞬时拉升；部分操作耗时数毫秒，部分任务则需持续挂起数小时等待网络回调或人工确认。理想的计算架构应当支持按需秒级拉起、闲时归零（Scale to Zero），并精准依据工具的资源画像调配不同梯度的计算单元。
但这并不等同于将传统 FaaS（Function as a Service）做简单套用。无状态函数假定单次调用即完成计算；而 Agent 执行具有长周期的上下文依赖，需要保留持久工作区、支持后台挂起长任务、接纳异步人工审批，并在数小时后依然能够无缝续接。
Agent Serverless 的核心在于状态的彻底解耦：Session 状态脱离任何特定计算节点的生命周期。
当某个执行沙箱因故障或超时回收后，Runtime 绝不试图从内存垃圾中拼凑还原原有的指针与未决调用栈。它依托持久化日志核验先前操作的最终收敛状态，将工作区快照与必要产物水合（Rehydrate）至全新分配的沙箱环境中，基于经过严密验证的历史前缀启动后续执行。
权限管理架构亦由此迎来重构。易失沙箱不持有云服务特权凭证，亦不默认具备泛化公网权限。沙箱内部仅获得当前任务所需的临时最小能力上下文；真正的密钥管理、审计仲裁与核心资源权限始终收拢于沙箱外部的受信 Runtime 中。沙箱内代码提出操作请求，外部权威执行安全鉴权与副作用分发。即使沙箱环境遭受入侵，其权限边界亦随沙箱的销毁而即时作废。
当执行载体的边际成本大幅下降，Agent 体系的规模化应用才具备落地可行性：单一 Agent 会话可针对不同任务阶梯式申请 V8、Python 容器乃至微型虚拟机；亦可并发实例化多个彼此隔离的工作沙箱，驱动多子 Agent 协同作业并在收工时立即释放硬件配额。
这一架构的极致演进，正是存算分离的全面深化：将会话的全量状态沉淀于廉价可靠的对象存储（如 S3-Compatible Storage）中，计算层则完全由轻量、异构、即用即弃的执行节点按需供给。
会话不再绑定于特定机器的目录或常驻进程。它体现为对象存储中一组不可变文件的集合：追加写的事件切片、外部产物实体、工作区文件快照、压缩投影快照以及指向当前合法提交前缀的元数据清单（Manifest）。交互间歇期，无须任何守护进程驻留内存；会话静止存放于存储池中，仅产生极低的静态数据存储成本。
```text
廉价持久化存储 (Object Storage)
Session A ── Events / Artifacts / Workspace Snapshots ─┐
Session B ── Events / Artifacts / Workspace Snapshots ─┼── S3
Session C ── Events / Artifacts / Workspace Snapshots ─┘
│
外部事件 / 用户交互 / 定时触发 │
│ │
▼ │
Rehydrate 会话上下文 ◀─────────┘
│
┌───────────┼───────────┐
▼ ▼ ▼
V8 Python MicroVM
│ │ │
└───────────┼───────────┘
│
追加增量事实
│
└──────────────→ S3
```
长效存续的 Agent 系统，不再依赖长期不关机的物理服务器。
它可以在绝大部分时间内处于完全休眠。用户发送指令、定时周期唤醒、Webhook 抵达或异步监控任务就绪时，调度控制面迅速检索 Session Manifest，挂载所需的日志切片与工作区快照，动态唤起适配规格的沙箱实例。任务执行收敛后，最新增量事实与工作区差分同步写回对象存储，计算资源随即完全归还。
系统由两层核心架构协同运作：
- **数据面（Data Plane）**：由对象存储托管体积庞大、不可变、极低频修改的历史数据与快照实体。
- **控制面（Control Plane）**：依托轻量级存储维护强一致的 Head 指针、资源租约、全局限额及未决操作状态。
这与现代分布式云原生数据库的解耦逻辑如出一辙。对象存储提供近乎无限的耐用性与低成本空间，计算算力仅在执行读写请求时按需供给。在此架构下，模型上下文的装配本质上等价于一次物化视图查询：Runtime 自对象存储提取 Session 状态，执行 Compaction 压缩与 Tool Result 裁剪投影，交付模型展开推理；模型输出的新事实再次以追加写形式沉淀回存储层。
```text
Session on S3
│
├── Projection ──→ Model Context ──→ LLM
│ │
├── Rehydrate ───→ Sandbox ──────────→ Tool Call
│ │
└──────────────── Append New Facts ◀───┘
```
在此架构中，LLM 与 Sandbox 均回归为纯粹的按需计算资源。
模型选型可依推理复杂度动态调配，简单逻辑调用轻量模型，复杂决策转交旗舰模型；计算载体则依能力需求即时匹配，轻量编排交付 Isolate，工程编译挂载 MicroVM。同一个 Session 不与特定模型或特定计算硬件存在排他绑定。
降低 Agent 运行成本的关键，在于会话处于休眠状态时不持有任何常驻计算资源。
存算分离架构亦为分布式抢占式调度（Spot Instances）与灵活分支（Fork / Branch）扫清了障碍。借助不可变日志与写时复制快照，派生子 Agent 仅需引用既有历史前缀与快照标识，即可零开销衍生出独立的并行分支，仅增量数据占用独立存储空间。
面向未来，Agent 运行时的演进方向正逐步明晰：以不可变的真实事实日志作为状态权威，以多级轻量的弹性沙箱作为执行载体，以解耦的资源权限保障并发安全，以按需加载的 Schema 投影消除模型注意力干扰。从概率语言模型向客观系统的每一次延伸，都由严谨的运行时边界构筑起安全与可靠的工程基石。Generated-by: Gemini Flash
Member
Author
|
Applied the reviewer-provided Gemini Flash drafts as exact whole-file replacements in commit 092b0c2.
No passages from the previous drafts were selectively retained. |
M4n5ter
approved these changes
Sep 3, 2026
ggbdpq
pushed a commit
to ggbdpq/maka
that referenced
this pull request
Sep 4, 2026
* docs(blog): add Beyond Function Calling Generated-by: Codex * docs(blog): adopt reviewer-polished drafts Generated-by: Gemini Flash --------- Co-authored-by: likun <kunli@alva.xyz> Generated-by: GLM-5.3-Flash (ZCode)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add English and Simplified Chinese versions of Beyond Function Calling: How Agents Reach the Real World.
Documents for review:
The essay follows the complete lifecycle of agent tools: deferred schema exposure, the model-to-runtime action boundary, durable T1/T2 settlement and replay, Code Mode and programmatic orchestration, parallel calls as async I/O, resource-authority coordination, and disaggregated sandbox compute over durable session storage.
Review focus
Verification
git diff origin/main...HEAD --checknpm run check:asf-headersAI use
Select exactly one:
Tool(s) and scope: OpenAI Codex drafted the original bilingual essay and prepared the pull request. Gemini Flash produced the reviewer-supplied comprehensive polish pass; Codex applied both revised drafts as exact whole-file replacements and verified parity.
Checklist
Does this PR entail a change in behavior?