feat: pluggable sandbox capability

[amithad]

Description

Pluggable sandbox capability for Agent Kernel (#494, formerly AK-133): agents execute code and shell commands in an isolated, permission-bounded environment, with pluggable providers, workload profiles, policy enforcement, and per-user identity. This PR ships the spec set and the first implementation slice (iterations 1–6) — working local_subprocess/docker sandboxes on CLI and REST — plus the shared pluggable-backend factory refactor (#541).

Scope note (updated): an earlier revision of this description said "no implementation code" — that is no longer accurate. The branch now contains the agentkernel.sandbox package, core wiring, tests, examples, and #541 factory changes. Remaining providers (e2b/daytona/kubernetes/bedrock_agentcore/ec2_ssm), the AWS sqs broker plane, and the atexit backstop are deferred to post-merge iterations per plan.md.

Type of Change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Documentation update
• Refactoring (no functional changes) — shared pluggable-backend factory pattern ([AK] Standardize pluggable-backend factory pattern + BYO extension #541)
• Performance improvement
• Test update
• CI/CD update

Related Issues

Closes part of #494 (iterations 1–6); relates to #541.

Changes Made

Sandbox capability (ak-py/src/agentkernel/sandbox/)

• Sandbox/SandboxProvider ABCs with honest per-provider SandboxCapabilities; SandboxError hierarchy
• SandboxManager (profile routing, sessions with nv_cache registry, fail-closed principal/policy enforcement, idle-reset notices)
• SandboxProviderFactory/SandboxBrokerFactory; embedded + thread broker flavors with BrokerWorkerCore
• Eight agent-facing system tools (execution, files, task poll, session lifecycle) + SandboxPreHook task-completion ingestion; self-describing via injected system prompt, optionally scoped via sandbox.agents
• Providers: local_subprocess (no isolation; dev/test) and docker (container); FakeSandboxProvider + reusable SandboxProviderContract

Shared factory pattern (#541)

• New core/util/factory.py (resolve_dotted, require_extra, AKConfigError); trace, guardrail, session/thread/multimodal-store, and sandbox factories converge on the if/elif real-import + dotted-path-BYO shape; unknown type now fails loud

Core wiring (three points)

• core/config.py (sandbox section), core/tool.py (SystemToolFactory, per-agent scoping), core/runtime.py (third system pre-hook); core/base.py passes agent.name to the factory

Examples & docs

• examples/cli/sandbox/{basic,profiles,policy} and examples/api/sandbox-identity (end-to-end per-user identity over REST)
• Docs site guide (docs/docs/advanced/sandbox.md), spec/design/plan under docs/specs/494-sandbox-capability/, new ak-dev-new-sandbox-provider skill

Behavior changes

• Fail loud on unknown backend type across the pluggable factories (previously some silently fell back to in-memory)
• Test framework keeps CLI stderr on a separate pipe (logs no longer pollute captured responses)

Testing

• Unit tests pass locally (cd ak-py && uv run pytest — 851 passed; the only failures are 3 pre-existing test_cli_tester live-LLM tests unrelated to this branch)
• Integration tests pass locally
• Manual testing completed (CLI + REST examples)
• New tests added for changes (test_sandbox.py, test_sandbox_broker.py, test_sandbox_providers.py, test_factory.py, test_store_builders.py, test_trace.py, plus example tests)

Checklist

• My code follows the project's style guidelines
• I have performed a self-review of my code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes
• Any dependent changes have been merged and published

Additional Notes

Iterations 1–6 land here; iterations 7–12 (remaining providers, AWS broker plane, atexit backstop, final docs pass) are sequenced post-merge in docs/specs/494-sandbox-capability/plan.md. Design decisions and dates are recorded in design.md.

[Copilot]

Pull request overview

This PR introduces the initial documentation set for AK-133 (“Sandbox capability”): a stage-1 design.md plus a new developer research skill containing the background research and reference material that informed the design.

Changes:

• Add docs/specs/ak-133/design.md defining the proposed sandbox capability requirements and architecture.
• Add developer skill .agents/skills/ak-dev-sandbox-research/ documenting scope, conclusions, and how to continue the research.
• Add supporting research references (provider landscape, framework prior art, and AK codebase patterns) under the new skill.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
docs/specs/ak-133/design.md	Stage-1 design spec for AK-133 sandbox capability (requirements + proposed architecture).
.agents/skills/ak-dev-sandbox-research/SKILL.md	Dev skill describing the research goal, scope, key findings, and linking to the design spec.
.agents/skills/ak-dev-sandbox-research/references/provider-landscape.md	Provider landscape survey and capability matrix used to inform design constraints.
.agents/skills/ak-dev-sandbox-research/references/framework-abstractions.md	Prior-art survey of executor/sandbox abstractions across major agent frameworks/SDKs.
.agents/skills/ak-dev-sandbox-research/references/ak-codebase-patterns.md	Summary of existing AK pluggable-capability patterns relevant to implementing sandboxing.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[amithad]

Spec-stage review — Stage 1 (design.md + research skill). No implementation code, so dimension 6 (spec conformance) is skipped.

Overall: a strong, correctly staged design PR. Findings are consistency gaps and skill-sync housekeeping, nothing structural.

• design.md verdict: point-form, hierarchical, complete for design altitude, exactly one diagram, and every path:line claim verified against current develop — the greenfield grep, guardrail/guardrail.py:25-68, core/multimodal/storage/storage_manager.py:33-84, core/tool.py:165-179, core/runtime.py:12, core/config.py:375-405, pyproject.toml:23, examples/cli/smolagents/demo_codeagent.py:27 all check out. Open questions record dated resolutions instead of silent decisions, and spec.md is properly deferred until this design review settles.
• Inline findings: 2 [question] (session-end teardown trigger doesn't exist in core today; concurrency-contract scope vs per_runtime/suspend-resume), 3 [suggestion] (SandboxManager referenced but never defined; the skill's Open Design Questions are resolved by design.md in this same PR; stale core/config.py line citations in ak-codebase-patterns.md), 1 [nit].
• Un-anchorable — skill inventory docs not updated: docs/docs/agent-skills.md:116 says "fourteen developer skills" (table at 122–135) and docs/specs/agent-skills.md (table at 58–71 + directory tree) enumerate every dev skill; this PR makes it fifteen but touches neither surface (ak-dev-sync-skills-from-branch Step 6 / validation #4). If ak-dev-sandbox-research is deliberately excluded as a transient research skill, note that somewhere; otherwise add the rows and bump the count.
• Un-anchorable — PR hygiene per ak-dev-write-spec: a spec/skill-only PR should use a docs: title (current: "Feature/sandbox capability"), fill the PR template marking Documentation update, and state that implementation follows in a separate PR — the body is currently empty. The branch's feat: commit messages ("feat: introduce pluggable sandbox capability…") also read as shipped code, which this PR doesn't contain.
• Scope note: external-source claims in provider-landscape.md / framework-abstractions.md (provider capabilities, SDK shapes, pricing, GitHub-issue pain points) were not independently verified; the AK-codebase claims in all three research documents were, and apart from the flagged config.py/runtime.py drift they verify.

[Copilot]

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

[Copilot]

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 5 comments.

[amithad]

Re-review — the PR now carries design.md (stage 1) and a new spec.md (stage 2). Still no implementation code, so dimension 6 (spec conformance vs code) remains skipped.

Overall: the revision quality is high — every finding from the previous review round was addressed substantively (teardown now anchored on a mandatory idle timeout instead of a nonexistent session-end event; the concurrency contract scoped per path; SandboxManager defined; the research skill's questions marked resolved; citations refreshed with symbol-name anchoring), and the PR title/body now follow the docs: spec-PR guidance. The new spec.md is thorough and its factual claims verify.

• spec.md claims verified against the codebase: BaseRunRequest extra="allow" (core/model.py:226), RequestBuilder._attach_additional_context → AgentRequestAny (core/chat_service.py:117-130 — core/runtime.py's docstring even states AgentRequestAny is pre-hook-only), core/runtime.py:49/198, core/tool.py:144-151/165-179/181-197, akagentrunner.py:90, deployment/aws/core/response_store/{redis,valkey,dynamodb}.py, _ResponseStoreConfig, the pickle-based serde, and tests/test_runtime.py:19-22; I independently re-ran the "no test asserts the pre-hook list composition" grep — it holds. The AgentRequestAny deviation is well-founded and properly flagged in both documents — exactly the process the spec flow asks for.
• Design-requirements walk: nearly everything in design.md traces to a spec section (interface, capabilities/isolation tiers, RBAC fail-closed paths, sandbox sessions/self-heal/idle teardown, DB-first completions, dedup, config, factory/BYO, all 7 providers, testing, docs deliverables). The inline findings are the residue: 2 [question] (stage-2 spec shipping while the stage-1 review is still open; broker flavor set deviating from design.md without a callout), 4 [suggestion] (client-side Lambda fail-fast has no worker-ceiling knowledge; the idle sweep reads response-DB session records nothing writes; 3-vs-5 tool count; per_runtime pooling deferred to spec.md but unanswered), 2 [nit].
• Skipped as already raised: the skill-inventory mismatch (PR body claims docs/docs/agent-skills.md + docs/specs/agent-skills.md register the skill, but neither file is in the diff) — flagged by Copilot on this PR; and the feat:-typed commit messages describing unshipped code — raised in the previous review round.
• CI: all checks passing.

[amithad]

Round-3 review — the spec set is now complete: design.md (approved), spec.md (approved), and the new stage-3 plan.md. Still no implementation code, so dimension 6 (spec conformance vs code) remains skipped.

Overall: this round is nearly clean. All 8 findings from the previous review were fixed substantively and verifiably in bea1fec4 — the fail-fast gap became a worker_timeout_ceiling terraform output with client-side SandboxPolicyError rejection; the idle sweep gained a fully specified broker-side session inventory (session:<id> records, per-op last_used_at refresh, max(2×idle_timeout, response_ttl) TTL, per-backend scan); the broker-flavor consolidation was folded back into design.md with the dated-deviation pattern; per_runtime pooling is explicitly closed as "none in v1"; tool count, SandboxManager labeling, and the test-file split are all reconciled. Copilot's round-2 comments (staging wording, design altitude, default_factory, skill status, deviation framing) were addressed in the same commit, and the staging question is settled: the PR body now declares the design approved (2026-07-16) and the single-branch flow deliberate.

• plan.md verdict (stage 3, reviewed at plan altitude): template-conformant — every iteration has Goal/Files/Steps/Verify with concrete test commands, steps reference spec sections instead of restating them, and each iteration leaves the branch testable. I mapped spec.md's components across the iterations: all land in exactly one iteration (7/7 providers, broker flavors, wiring points, terraform), and the mandatory Tests (11) and docs/skills-sync (12) iterations are present — iteration 12 is notably complete (inventories, architecture skill, research-skill status flip, verified-no-update surfaces). The iteration-7 ec2_ssm evaluation checkpoint before the AWS plane is a sensible risk gate. The "Future requirements → seams" section covers every deferred design non-goal that needs one.
• New findings: 1 [suggestion] (plan.md introduces tests/test_sandbox_tools.py, a fourth test file spec.md's Testing section doesn't define, plus an It-11 wording conflict with the spec about touching test_runtime.py), 1 [nit] (the research skill says plan.md "follows" — it now ships in this PR).
• Skipped as already raised (still open): the skill-inventory registration — the PR body's "Changes Made" still claims docs/docs/agent-skills.md and docs/specs/agent-skills.md register the new skill, but neither file is in the diff (Copilot's earlier comment). Note the dependency: plan.md line 149's "fifteen → sixteen" arithmetic assumes this PR already made it fifteen.
• CI: green (one check still completing at review time).

[Copilot]

Pull request overview

Copilot reviewed 91 out of 96 changed files in this pull request and generated 5 comments.

[amithad]

Round-5 re-review — verification of the round-4 findings (head 33a92a1e, five fix commits).

Overall: every round-4 finding is resolved, and the fixes were verified against the code — not just the replies. One residual suggestion (inline) on the completion-retention fix; nothing else found in the fix delta.

• Blockers (3/3 verified fixed):
  • Session/profile routing: re-ran the original repro against 33a92a1e — an explicit sandbox_session_id now resolves under the profile it was minted with (_resolve_profile_name), the gpu-profile session executes on the gpu provider, and a contradicting caller profile raises SandboxConfigError. The regression test test_manager_explicit_session_uses_its_own_profile locks it in.
  • Promotion handle loss: same repro — the promoted run's sandbox_id now lands in the registry via task_status → _write_session(completion.sandbox_session), and the second call reattaches instead of orphaning. The idle-reset notice now rides SandboxTask too (model + tool JSON), closing the "recreation is never silent" gap on the promotion path.
  • stderr e2e: run-tests / e2e-tests (1, cli, examples/cli/openai-dynamic) passes — the test now asserts the observable no-op behavior of a failed !select, and the test/test.py change is documented in spec Consumer + Behavioural changes. Keeping the stderr split is a reasonable call.
• Suggestions/questions/nits: all 32 addressed — 27 fixed as claimed (spot-verified: the seven new/strengthened tests exist and assert what was promised; broker fail-loud flavor error matches the #541 shape; agent routing uses _current_agent_name(); docs/skills/examples corrections all present, including the CJK fix and the identity example's attach=True), 3 resolved by documented deviation (spec rule 2 now lists broker/worker.py; SandboxProvisionError wrapping and the ABC-contract docs recorded as explicit follow-ups in spec/#541 design), 2 deferred with proper dated plan entries (the atexit backstop → new post-merge iteration 8.5 with spec §Error-handling note; the e2b/daytona/kubernetes extras → iterations 9-10, removed from pyproject). The evals must_not_contain answer (local guidance never emits sandbox-docker) is accepted.
• Tests: full suite at head — 190 passed (was 182; the eight additions match the claimed tests). Both prior repro scripts now print the fixed behavior.
• CI: fully green — 0 pending, 0 failing at review time.
• PR hygiene: title is now feat: pluggable sandbox capability and the body accurately describes the iterations 1-6 scope with a dated correction note — the long-standing docs-only staleness (raised by Copilot and carried through earlier rounds) is closed.
• Fix-delta review: the new code introduced by the five commits was reviewed as a delta (manager _resolve_profile_name/_current_agent_name, SandboxBroker.discard, factory flavor fail-loud, SandboxTask.notice, spec/plan/design syncs, identity-example hardening) — apart from the inline residual, no new issues.

With the inline follow-up noted, this is ready for maintainer sign-off from my side.

[Copilot]

Pull request overview

Copilot reviewed 91 out of 96 changed files in this pull request and generated 5 comments.