feat(registry,federation): namespaces, BM25 search, capability manifests

CHANGELOG.md

@@ -7,6 +7,34 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+### Added
+- Capability namespaces and hierarchical discovery in `CapabilityRegistry`:
+  dot-notation `capability_id`s now expose `list_namespaces()` /
+  `list_namespace(prefix)` operations; `register_namespace(prefix, loader=...)`
+  enables deferred registration for large tool ecosystems (the loader runs
+  at most once on first access). `search()` gained an `offset` kwarg for
+  pagination, strips a small stop-word set, and now scores with a
+  BM25-flavoured ranker that weights `capability_id`/`tags` matches above
+  `description`. Flat (un-namespaced) capability IDs continue to work
+  unchanged. (#45)
+- Capability marketplace, part 1 — manifest format & local registry: new
+  `CapabilityDescriptor` and `CapabilityManifest` dataclasses (both
+  JSON-round-trippable via `to_dict`/`from_dict`), new
+  `agent_kernel.federation` module with `build_manifest()`,
+  `import_manifest()`, and `merge_sensitivity()`, and new `Kernel.advertise()`
+  / `Kernel.import_remote()` methods. `Kernel` gained a `kernel_id`
+  argument used as the manifest publisher identity. Three trust policies
+  are honoured at import time (`most_restrictive` (default), `local_only`,
+  `remote_deferred`); imported capabilities are routed through a
+  caller-supplied driver and flow through the full local policy → token →
+  firewall pipeline. HMAC tokens remain kernel-scoped — a token issued by
+  one kernel cannot be verified by another with a different secret. New
+  errors `NamespaceNotFound`, `FederationError`, `ManifestError`,
+  `TrustPolicyError`. (#52)
+- New docs: [`docs/federation.md`](docs/federation.md) for the marketplace
+  protocol and a namespace section in
+  [`docs/capabilities.md`](docs/capabilities.md).
+
 ## [0.7.0] - 2026-05-20
 
 ### Added

docs/capabilities.md

@@ -3,9 +3,60 @@
 ## Naming conventions
 
 - Use `domain.verb_noun` format: `billing.list_invoices`, `users.get_profile`.
+- Prefer fully namespaced IDs (`billing.invoices.list`) over flat ones —
+  the registry will infer namespace operations from the dot-segments and
+  large ecosystems benefit from being able to list/search per namespace.
 - Be specific: prefer `billing.cancel_invoice` over `billing.update`.
 - Avoid generic names like `billing.execute` or `api.call`.
 
+## Namespaces and discovery
+
+`CapabilityRegistry` recognises dot-notation namespaces automatically. No
+extra registration step is required — `register(Capability(capability_id=
+"billing.invoices.list", ...))` is enough to populate the `billing` and
+`billing.invoices` namespaces.
+
+```python
+registry.list_namespaces()
+# ['billing', 'crm']
+
+registry.list_namespace("billing")
+# [Capability('billing.invoices.list'), Capability('billing.payments.refund'), …]
+```
+
+For large tool ecosystems where eagerly registering hundreds of
+capabilities is wasteful, declare a deferred loader. The loader runs at
+most once, the first time the namespace is searched, listed, or any
+capability under it is fetched via `get()`:
+
+```python
+def load_billing() -> list[Capability]:
+    return [
+        Capability(capability_id="billing.invoices.list", …),
+        Capability(capability_id="billing.invoices.create", …),
+        Capability(capability_id="billing.payments.refund", …),
+    ]
+
+registry.register_namespace(
+    "billing",
+    description="Billing and invoicing tools",
+    loader=load_billing,
+)
+```
+
+Search ranks matches with a BM25-flavoured scorer that weights
+`capability_id` and `tags` higher than `description`, strips a small
+stop-word set (`a`, `the`, `please`, …), and offers `offset` for
+pagination:
+
+```python
+results = registry.search("list invoices", max_results=10, offset=0)
+```
+
+Search is deterministic — equal-scoring capabilities are returned in
+`capability_id` order — and trips any deferred namespace loader whose
+prefix shares a token with the query.
+
 ## Granularity
 
 Each capability should map to a single, auditable action with clear side-effects.

docs/federation.md

@@ -0,0 +1,153 @@
+# Capability Federation — Marketplace Part 1
+
+> Issue [#52](https://github.com/dgenio/agent-kernel/issues/52) (manifest
+> format & local registry) is implemented here. Discovery over a network
+> (issue [#51](https://github.com/dgenio/agent-kernel/issues/51)) is **not**
+> part of this milestone — `agent-kernel` does not fetch manifests over
+> HTTP or sign them on your behalf yet. Bring your own transport for now.
+
+## What this gives you
+
+A single kernel can:
+
+1. **Advertise** its capabilities as a JSON-serialisable
+   [`CapabilityManifest`](../src/agent_kernel/models.py).
+2. **Import** another kernel's manifest, registering each capability locally
+   and routing invocations through a caller-supplied driver
+   (typically [`HTTPDriver`](integrations.md) or
+   [`MCPDriver`](integrations.md)).
+
+Every imported invocation still runs through the *local* policy → token →
+firewall pipeline. The remote endpoint is never trusted to authorise on the
+importing kernel's behalf. This keeps weaver-spec invariants intact for
+imported capabilities:
+
+| Invariant | How it's enforced for imports |
+|-----------|------------------------------|
+| **I-01** — Firewall on every result | The local `Firewall` runs on the driver's `RawResult` exactly as for native capabilities. |
+| **I-02** — Authorize + audit each call | The local `PolicyEngine` evaluates every request; the local `TraceStore` records every action. |
+| **I-06** — Tokens bind principal + capability + constraints | Tokens are signed with the importing kernel's HMAC secret. A token issued by Kernel A cannot be verified by Kernel B, because their secrets differ. |
+
+## Publishing a manifest
+
+```python
+from agent_kernel import (
+    Capability, CapabilityRegistry, HMACTokenProvider, Kernel,
+    SafetyClass, SensitivityTag,
+)
+
+registry = CapabilityRegistry()
+registry.register(
+    Capability(
+        capability_id="billing.invoices.list",
+        name="List Invoices",
+        description="List recent invoices",
+        safety_class=SafetyClass.READ,
+        sensitivity=SensitivityTag.PII,
+        tags=["billing", "invoices"],
+    )
+)
+
+kernel = Kernel(
+    registry=registry,
+    token_provider=HMACTokenProvider(secret="…"),
+    kernel_id="agent-b",
+)
+
+manifest = kernel.advertise(endpoint="https://agent-b.example/kernel")
+print(manifest.to_dict())
+# {
+#   "kernel_id": "agent-b",
+#   "version": "1",
+#   "endpoint": "https://agent-b.example/kernel",
+#   "trust_level": "unverified",
+#   "capabilities": [
+#     {
+#       "capability_id": "billing.invoices.list",
+#       "name": "List Invoices",
+#       …
+#     }
+#   ]
+# }
+```
+
+The manifest deliberately omits internal driver IDs, operation names,
+`parameters_model` Python references, and `tool_hints`. Only the
+[`CapabilityDescriptor`](../src/agent_kernel/models.py) projection of each
+capability is published.
+
+## Importing a manifest
+
+```python
+import json
+
+import httpx
+from agent_kernel import (
+    CapabilityManifest, CapabilityRegistry, HMACTokenProvider, Kernel,
+)
+from agent_kernel.drivers.http import HTTPDriver, HTTPEndpoint
+
+# 1. Fetch the manifest by whatever transport suits you.
+raw = httpx.get("https://agent-b.example/kernel/manifest").json()
+manifest = CapabilityManifest.from_dict(raw)
+
+# 2. Build a local driver pointing at the remote endpoint.
+remote = HTTPDriver(driver_id="agent-b")
+for cap in manifest.capabilities:
+    remote.register_endpoint(
+        cap.capability_id,
+        HTTPEndpoint(url=f"{manifest.endpoint}/invoke/{cap.capability_id}",
+                     method="POST"),
+    )
+
+# 3. Import. `import_remote` registers the driver and adds routes.
+kernel = Kernel(
+    registry=CapabilityRegistry(),
+    token_provider=HMACTokenProvider(secret="local-secret"),
+    kernel_id="agent-a",
+)
+kernel.import_remote(manifest, driver=remote, trust_policy="most_restrictive")
+
+# 4. Use imported capabilities exactly like local ones.
+for cap in kernel.list_capabilities():
+    print(cap.capability_id, "→", cap.impl.driver_id)
+```
+
+## Trust policies
+
+`import_remote(manifest, driver=..., trust_policy=...)` accepts three
+values for `trust_policy`:
+
+| Value | Sensitivity handling | When to use |
+|-------|---------------------|-------------|
+| `"most_restrictive"` *(default)* | Imported capability keeps the remote `SensitivityTag` verbatim — the local firewall will then redact accordingly. | Crossing trust boundaries — when you can't fully verify the remote's policy. |
+| `"local_only"` | Imported capability is registered with `SensitivityTag.NONE`; the importing kernel's policy is the only thing that gates the call. | You own both kernels and have a single canonical policy. |
+| `"remote_deferred"` | Same sensitivity handling as `most_restrictive` today. Reserved for part 2, when the importing kernel will be able to defer to a remote policy decision before applying its own. | Delegation patterns where the remote owns the authoritative policy. |
+
+`merge_sensitivity(local, remote)` is exported for callers that maintain
+their own capability records and want the canonical strictest-wins union.
+
+## What is *not* covered yet
+
+- **No network transport.** `agent-kernel` does not fetch, sign, or
+  authenticate manifests over HTTP — bring your own transport. Part 2
+  (issue #51) adds an opt-in manifest endpoint and a discovery protocol.
+- **No remote policy delegation.** `"remote_deferred"` currently behaves
+  identically to `"most_restrictive"`. The full "remote policy decides
+  first" semantics need part 2.
+- **No automatic re-import.** Manifests are imported once. If the publisher
+  adds capabilities, the importer must re-fetch and re-import.
+- **No identity verification.** `trust_level` is a publisher-declared hint;
+  it does not authenticate the publisher. Signature verification arrives
+  with part 2.
+
+## Reference
+
+- Models: [`CapabilityDescriptor`](../src/agent_kernel/models.py),
+  [`CapabilityManifest`](../src/agent_kernel/models.py).
+- Functions: [`build_manifest`](../src/agent_kernel/federation.py),
+  [`import_manifest`](../src/agent_kernel/federation.py),
+  [`merge_sensitivity`](../src/agent_kernel/federation.py).
+- Kernel methods: `Kernel.advertise()`, `Kernel.import_remote()`,
+  `Kernel.kernel_id`.
+- Errors: `FederationError`, `ManifestError`, `TrustPolicyError`.

src/agent_kernel/__init__.py

@@ -31,6 +31,11 @@
 
     from agent_kernel import OpenAIMiddleware, AnthropicMiddleware
 
+Federation (capability marketplace)::
+
+    from agent_kernel import CapabilityManifest, CapabilityDescriptor
+    from agent_kernel import build_manifest, import_manifest, TrustPolicy
+
 Errors::
 
     from agent_kernel import (
@@ -39,6 +44,7 @@
         PolicyDenied, PolicyConfigError, DriverError, FirewallError,
         BudgetExhausted, BudgetConfigError,
         CapabilityNotFound, HandleNotFound, HandleExpired,
+        NamespaceNotFound, FederationError, ManifestError, TrustPolicyError,
     )
 """
 
@@ -56,15 +62,26 @@
     CapabilityAlreadyRegistered,
     CapabilityNotFound,
     DriverError,
+    FederationError,
     FirewallError,
     HandleExpired,
     HandleNotFound,
+    ManifestError,
+    NamespaceNotFound,
     PolicyConfigError,
     PolicyDenied,
     TokenExpired,
     TokenInvalid,
     TokenRevoked,
     TokenScopeError,
+    TrustPolicyError,
+)
+from .federation import (
+    MANIFEST_VERSION,
+    TrustPolicy,
+    build_manifest,
+    import_manifest,
+    merge_sensitivity,
 )
 from .firewall.budget_manager import BudgetManager
 from .firewall.budgets import Budgets
@@ -75,14 +92,17 @@
 from .models import (
     ActionTrace,
     Capability,
+    CapabilityDescriptor,
     CapabilityGrant,
+    CapabilityManifest,
     CapabilityRequest,
     DenialExplanation,
     DryRunResult,
     FailedCondition,
     Frame,
     Handle,
     ImplementationRef,
+    NamespaceMetadata,
     PolicyDecision,
     PolicyDecisionTrace,
     PolicyTraceStep,
@@ -92,6 +112,7 @@
     ResponseMode,
     RoutePlan,
     ToolHints,
+    TrustLevel,
 )
 from .policy import DefaultPolicyEngine, ExplainingPolicyEngine, PolicyEngine
 from .policy_dsl import DeclarativePolicyEngine, PolicyMatch, PolicyRule
@@ -112,7 +133,9 @@
     "CapabilityRegistry",
     # models
     "Capability",
+    "CapabilityDescriptor",
     "CapabilityGrant",
+    "CapabilityManifest",
     "CapabilityRequest",
     "CapabilityToken",
     "DenialExplanation",
@@ -121,6 +144,7 @@
     "Frame",
     "Handle",
     "ImplementationRef",
+    "NamespaceMetadata",
     "PolicyDecision",
     "PolicyDecisionTrace",
     "PolicyTraceStep",
@@ -131,6 +155,7 @@
     "RoutePlan",
     "ActionTrace",
     "ToolHints",
+    "TrustLevel",
     # enums
     "SafetyClass",
     "SensitivityTag",
@@ -142,15 +167,25 @@
     "CapabilityAlreadyRegistered",
     "CapabilityNotFound",
     "DriverError",
+    "FederationError",
     "FirewallError",
     "HandleExpired",
     "HandleNotFound",
+    "ManifestError",
+    "NamespaceNotFound",
     "PolicyConfigError",
     "PolicyDenied",
     "TokenExpired",
     "TokenInvalid",
     "TokenRevoked",
     "TokenScopeError",
+    "TrustPolicyError",
+    # federation
+    "MANIFEST_VERSION",
+    "TrustPolicy",
+    "build_manifest",
+    "import_manifest",
+    "merge_sensitivity",
     # policy
     "AllowReason",
     "DefaultPolicyEngine",