Skip to content

Architecture: capability-driven host, provider, binding, and observability adapters #71

Description

@pacphi

Summary

Introduce a capability-driven abstraction for hosts, model providers/endpoints, configuration projections, and observability sources so adding OpenCode, OpenRouter, Ollama-backed models, and future providers does not require editing overlapping hardcoded lists throughout ak.

This is a follow-on architecture issue, not part of PR #67. It should provide the foundation for:

  • OpenCode and future agent CLIs as managed hosts;
  • OpenRouter and other gateways as model providers behind one or more hosts;
  • Ollama models served through both Claude and Codex compatibility integrations;
  • issue Usage scorecard: ingest OpenRouter session logs as a third provider #59's OpenRouter usage attribution without misclassifying OpenRouter as necessarily being a third execution host.

Related:

Why this is needed

The current provider/host implementation has grown incrementally and now represents several different concepts with adjacent arrays, maps, and conditionals:

  • src/lib/hosts.mjs
    • HOST_ADAPTERS
    • session-driver detection
    • auth and guidance capabilities
  • src/lib/providers.mjs
    • HOSTS
    • API_PROVIDERS
    • AQE_PROVIDER_TYPES
    • credential descriptors
    • install/detect/apply behavior
  • src/lib/routing.mjs
    • PRIMARY_HOSTS
    • routable/constructible providers
    • default host+model routes
  • command implementations
    • setup, status, sync, provider pick/off, verify, uninstall
  • dashboard/usage/live code
    • hardcoded host/provider ordering and accepted transcript hosts
    • provider-specific pricing, quota, session, and attribution behavior

Those are not all the same axis.

The four axes we need to distinguish

  1. Host / execution driver

    • The agent CLI or environment running the work.
    • Examples: Claude Code, Codex CLI, OpenCode.
  2. Model provider / endpoint

    • The service or local runtime serving inference.
    • Examples: Anthropic, OpenAI, OpenRouter, Ollama, Gemini, Bedrock, Azure OpenAI, ONNX.
    • A provider may be reached through more than one host.
  3. Configuration projection

    • The native configuration surface into which ak projects intent.
    • Examples: Claude settings/env, Codex TOML/profile, OpenCode JSON, ruflo router config, agentic-qe router config.
  4. Observability source

    • The transcript, usage, quota, or local-model evidence from which ak derives truth.
    • Examples: Claude JSONL, Codex rollout JSONL, OpenCode logs, OpenRouter response metadata, Ollama catalog/runtime APIs.

Today these concepts are easy to conflate. For example:

  • OpenCode is a host, but is not currently a primary/activity-routing target.
  • OpenRouter is a provider/gateway, but a request may still be executed and logged by Claude, Codex, OpenCode, ruflo, or AQE.
  • Ollama is a local provider/runtime, and ollama launch claude / ollama launch codex serve Ollama models through two different hosts and compatibility transports.
  • A transcript host is not enough to establish billing/provider provenance. A Claude transcript can represent Anthropic-hosted or Ollama-served inference.

Without a clearer abstraction, each new integration risks:

  • another hardcoded list that drifts from the others;
  • documentation that calls providers “hosts” or hosts “providers”;
  • invalid routing choices;
  • incorrect ownership/install behavior;
  • fabricated pricing, quota, or update claims;
  • dashboards grouping by the wrong dimension;
  • command paths that support only part of the lifecycle.

Goals

  1. Define one validated registry for built-in host adapters and one for built-in provider adapters.

  2. Make capabilities explicit so commands and UI derive behavior rather than checking ids.

  3. Represent a resolved execution as at least:

    host + provider + model + transport/endpoint + provenance
    
  4. Allow a provider to bind to multiple hosts where supported.

  5. Allow a host to expose multiple provider bindings without becoming a new provider itself.

  6. Preserve existing Claude/Codex routing behavior and existing kit.json configurations.

  7. Make setup/status/sync/verify/uninstall and the dashboard consume the same normalized facts.

  8. Support honest attribution for local and gateway-served models:

    • local vs subscription vs metered billing;
    • known vs inferred vs unknown provenance;
    • exact model id when observed;
    • no fabricated price/quota/cache fields.
  9. Keep the runtime zero-dependency and offline-first.

  10. Make future built-in integrations additive and testable without loading arbitrary third-party code.

Non-goals

  • Do not implement every provider in this issue.
  • Do not turn ak into a generic third-party plugin runtime.
  • Do not store API keys or OAuth credentials in kit.json.
  • Do not make every host a valid primary or activity-routing target.
  • Do not make OpenCode routable unless a separate, grounded design establishes that capability.
  • Do not fold Usage scorecard: ingest OpenRouter session logs as a third provider #59's full transcript/parser/pricing implementation into the initial refactor.
  • Do not claim provider provenance when only the host is known.
  • Do not add dashboard write/control behavior.

Proposed conceptual model

Names are illustrative; the ADR created by this issue may refine them.

1. Host adapter

A host adapter describes a driver and its native surfaces:

{
  id: 'claude',
  label: 'Claude Code',
  install: {
    bin: 'claude',
    npmPackage: '@anthropic-ai/claude-code',
    externalInstallPolicy: 'detect-never-overwrite'
  },
  capabilities: {
    canDriveSession: true,
    canBePrimary: true,
    canRouteActivities: true,
    commandStatusline: true,
    transcripts: true,
    usage: true,
    nativeMcpConfig: true,
    nativeGuidance: true
  },
  auth: { /* detection only; no secret persistence */ },
  configProjection: 'claude',
  observability: ['claude-transcripts', 'claude-statusline']
}

An OpenCode host adapter could set canBePrimary:false and
canRouteActivities:false while still supporting install, MCP, guidance,
agents, skills, status, sync, and teardown.

2. Provider adapter

A provider adapter describes inference service/runtime behavior independently of the host:

{
  id: 'ollama',
  label: 'Ollama',
  billing: 'local',
  credentials: { kind: 'none' },
  transports: ['openai-compatible', 'anthropic-compatible', 'native'],
  capabilities: {
    modelDiscovery: true,
    runtimeDiscovery: true,
    pricing: 'zero',
    quota: false,
    cacheAccounting: 'provider-dependent'
  },
  projections: ['ruflo', 'aqe', 'claude', 'codex'],
  observability: ['ollama-catalog', 'host-transcripts']
}

OpenRouter would be billing:'metered', use an environment credential descriptor,
support an OpenAI-compatible transport, and expose dated/offline pricing metadata
without being described as a host.

3. Provider binding

A binding connects a provider to a host/projection:

{
  id: 'ollama-via-claude',
  host: 'claude',
  provider: 'ollama',
  transport: 'anthropic-compatible',
  endpoint: 'http://127.0.0.1:11434',
  model: 'qwen3.6:latest',
  source: 'user',
  managedBy: 'agentic-kit'
}

and independently:

{
  id: 'ollama-via-codex',
  host: 'codex',
  provider: 'ollama',
  transport: 'openai-compatible',
  endpoint: 'http://127.0.0.1:11434/v1/',
  model: 'qwen3-coder:30b',
  source: 'user',
  managedBy: 'agentic-kit'
}

The same model/provider can therefore be reached through multiple hosts without
duplicating the provider definition.

4. Normalized runtime facts

Detection should return facts, not pre-rendered conclusions:

{
  hosts: {
    claude: {
      present: true,
      installMethod: 'npm',
      version: '…',
      enabled: true,
      authenticated: 'subscription',
      wired: true
    }
  },
  providers: {
    ollama: {
      configured: true,
      reachable: true,
      billing: 'local',
      credential: 'not-required'
    }
  },
  bindings: [
    {
      host: 'claude',
      provider: 'ollama',
      model: 'qwen3.6:latest',
      provenance: 'configured',
      reachable: true
    }
  ]
}

Status rows, dashboard cards, verification, and sync planning should project from
these facts. A missing fact must remain unknown rather than becoming a plausible default.

Capability-driven behavior

Commands should ask capabilities rather than compare ids:

  • Setup/sync install only adapters with an install capability.
  • Provider selection lists all managed host integrations, but primary-host selection lists only canBePrimary.
  • Activity routing accepts only canRouteActivities.
  • Guidance/statusline/transcript work runs only when the host exposes that surface.
  • Pricing/quota/cache views render only when the provider/observability source can support them.
  • Update drift applies only to packages ak actually owns (for example npm-managed host CLIs); external installs remain visible but unmanaged.
  • Verification runs a host/provider/binding-specific proof contract.

Configuration and migration

The design must preserve current configuration and provide a deterministic migration.

Questions for the ADR:

  1. Keep existing providers.hosts and add providers.bindings, or introduce a versioned top-level integrations model?
  2. Should existing dualRouting gain an optional provider field, or should provider resolution remain a separate binding lookup?
  3. How is an inferred legacy route represented without rewriting user intent?
  4. Which values are owned by ak, and how are {prior,written} guards generalized across JSON, TOML, env, and CLI-managed surfaces?

Minimum requirements:

  • Existing Claude-only and Claude+Codex kit.json files load unchanged.
  • Migration is additive, versioned, idempotent, and covered by fixtures.
  • Missing provider information remains unknown or is marked with explicit inferred provenance.
  • API keys remain environment-only.
  • Endpoint validation distinguishes trusted loopback local endpoints from remote HTTPS endpoints.
  • No adapter may silently overwrite externally managed host/provider configuration.

OpenRouter relationship (#59)

#59 should remain the delivery issue for usage-scorecard ingestion and pricing decisions.
This abstraction issue should give it a correct identity/provenance model.

Important distinction:

  • A Claude/Codex/OpenCode transcript is evidence of the host.
  • OpenRouter request/model metadata is evidence of the provider.
  • They may describe the same execution and should be joined where correlation is grounded.
  • If no client-side OpenRouter transcript exists, the scorecard must not invent a third host merely to represent provider usage.

The eventual scorecard should be able to answer separately:

  • Which host executed the session?
  • Which provider served the model?
  • Which model was observed?
  • What billing/pricing source applies?
  • Which claims are observed, configured, inferred, or unknown?

Ollama through Claude and Codex

The first concrete proving case for multi-host provider bindings should be Ollama:

  1. Configure/observe ollama launch claude.
  2. Configure/observe ollama launch codex.
  3. Preserve host-specific transcript parsing.
  4. Attribute the provider as Ollama only when configuration or runtime evidence supports it.
  5. Resolve the exact local model/digest from the bounded catalog approach in ADR-0011.
  6. Show billing as local/$0 without fabricating cache, quota, or exact token semantics.
  7. Keep both bindings independent: disabling one must not disable or rewrite the other.

This should use the validation evidence requested by docs/LOCAL-MODEL-VALIDATION.md
rather than assumptions about compatibility-layer behavior.

Suggested delivery phases

Phase 0 — ADR and inventory

  • Inventory every current host/provider hardcoded list and its consumer.
  • Define terminology and capability contracts.
  • Decide configuration/migration shape.
  • Define ownership and provenance vocabulary.
  • Record non-goals and backward-compatibility rules.

Deliverable: accepted ADR plus a test matrix; no behavior change.

Phase 1 — Registry extraction with behavior parity

  • Move existing Claude/Codex host metadata into the validated host registry.
  • Move current provider metadata/credentials/billing into the provider registry.
  • Derive existing exported lists for compatibility during migration.
  • Add registry validation for duplicate ids, missing required capabilities, invalid projections, and invalid billing/auth combinations.
  • Keep current CLI output and routing behavior byte-for-byte where practical.

Deliverable: internal refactor with no new provider behavior.

Phase 2 — Lifecycle/projection interfaces

  • Define detect/plan/apply/verify/undo contracts.
  • Move host-specific writes behind projection adapters.
  • Make setup/status/sync/provider/uninstall use normalized facts and lifecycle results.
  • Generalize ownership results without weakening the value-precise/no-clobber rules established by PR feat: OpenCode as a managed non-routable host adapter (ADR-0017) #67.

Deliverable: one lifecycle path consumed by all commands.

Phase 3 — Binding and provenance model

  • Add provider bindings and migration.
  • Teach routing/status/dashboard to distinguish host, provider, and model.
  • Preserve existing dualRouting semantics.
  • Add explicit provenance (observed, configured, inferred, unknown) to usage/model facts.

Deliverable: a Claude/Codex route can resolve through a named provider without conflation.

Phase 4 — Proving integrations

Deliverable: at least one provider bound to two hosts and one metered gateway represented behind an existing host.

Phase 5 — UI and documentation convergence

  • Derive CLI tables, dashboard grouping, filters, and provider ordering from registries/capabilities.
  • Update provider, usage, transcript, local-model, managed-tool, and upgrading documentation.
  • Remove compatibility exports/hardcoded lists only after all consumers migrate.

Test strategy

Registry contract tests

  • unique ids;
  • valid capabilities;
  • every projection/observability source resolves;
  • billing and credential descriptors are consistent;
  • no provider is accidentally accepted as a host or vice versa.

Adapter conformance tests

Every adapter must pass a shared contract:

  • detection is read-only;
  • planning is deterministic;
  • dry-run performs no writes;
  • apply is idempotent;
  • verify reports observed truth;
  • undo removes/restores only owned values;
  • external/user-owned configuration survives;
  • malformed/unavailable surfaces degrade honestly.

Matrix tests

At minimum:

Host Provider Expected
Claude Anthropic/subscription existing behavior preserved
Codex OpenAI/subscription existing behavior preserved
Claude Ollama local provider binding, Claude transcript host
Codex Ollama local provider binding, Codex transcript host
Claude or Codex OpenRouter metered provider behind existing host
OpenCode configured provider managed host lifecycle; routing capability respected

Command tests

  • setup/status/sync/provider/verify/uninstall consume the same adapter facts;
  • absent CLI, absent credential, unreachable local endpoint, and external install cases;
  • migration from existing configs;
  • dry-run and teardown ordering;
  • no real home/global writes.

Observability tests

Acceptance criteria

  • An ADR defines the four axes, capability contracts, configuration migration, ownership, and provenance.
  • Existing Claude/Codex behavior and configuration remain backward compatible.
  • Host, provider, binding, and observability registries have validation tests.
  • Setup/status/sync/provider/verify/uninstall derive behavior from adapter capabilities.
  • Primary/activity routing remains limited by capability, not a hardcoded incidental list.
  • One provider can bind to multiple hosts.
  • Ollama-through-Claude and Ollama-through-Codex are represented as two bindings to one provider.
  • OpenRouter is represented as a provider behind a host, not automatically as a third host.
  • Status/dashboard can display host, provider, model, billing, and provenance without conflation.
  • npm-managed versus external update ownership remains truthful.
  • API keys are never persisted.
  • Dry-run/idempotence/undo/no-clobber contracts are shared and tested.
  • Usage scorecard: ingest OpenRouter session logs as a third provider #59 can consume the abstraction without being subsumed by this issue.
  • Documentation uses the same vocabulary across README, providers, routing, usage, transcripts, local models, and upgrading.

Open design questions

  1. Should this remain internal-only, or should a stable JSON schema be documented for kit.json bindings?
  2. Should provider bindings be machine-wide, project-scoped, or support both with explicit precedence?
  3. Should routing policies name a provider directly, or resolve one through a host/model binding?
  4. How should multiple endpoints for the same provider be selected and displayed?
  5. What is the minimum grounded evidence required to upgrade provider provenance from configured/inferred to observed?
  6. How should host-native profiles created by external tools such as ollama launch be adopted, referenced, or left externally owned?
  7. Can ruflo and AQE share a normalized provider intent without pretending their native configuration schemas are identical?

Definition of done

This issue is complete when the architecture and compatibility layer are implemented,
the first multi-host provider binding is proven with Ollama, OpenRouter attribution has
a supported integration point for #59, and new built-in hosts/providers can be added
without modifying unrelated command, routing, dashboard, and usage hardcoded lists.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions