--- title: integrity-oracle created: 2026-07-07 updated: 2026-09-18 type: entity tags: [infrastructure, metrics, layer-2, tokenomics] confidence: high source_files: - integrity-oracle/scoring-core/src/lib.rs - integrity-oracle/backend/src/handlers.rs - integrity-oracle/backend/src/derive.rs - integrity-oracle/backend/src/otlp.rs - integrity-oracle/backend/src/chain.rs - integrity-oracle/backend/src/config.rs - integrity-oracle/backend/src/db.rs - integrity-oracle/backend/src/phi.rs - integrity-oracle/backend/src/vc.rs - integrity-oracle/backend/src/verification.rs - integrity-oracle/backend/src/attestation.rs - integrity-oracle/backend/src/kyc.rs - integrity-oracle/backend/src/crypto/mod.rs - integrity-oracle/backend/src/openapi.rs - integrity-oracle/backend/migrations/0001_init.sql - integrity-oracle/backend/migrations/0002_markets_and_judge.sql - integrity-oracle/backend/migrations/0003_agent_did_document.sql - integrity-oracle/backend/migrations/0021_otel_retention.sql --- The off-chain brain (Rust, Axum, Postgres, Redis, `alloy`): it ingests agent telemetry, computes the [Agent Integrity Score](ais.md), and independently verifies agents' on-chain state — including, as of this pass, the [market/application layer](agent-primitives.md) (§6.9) — so nothing downstream has to trust an agent's own word. Per §6.10 of the interface contract, this is the **only** backend that ever reads on-chain state. ## Table of contents - [Workspace](#workspace) - [HTTP API](#http-api) - [GET /v1/agent/{id}/telemetry, GET /v1/agent/{id}/traces](#get-v1-agent-id-telemetry-get-v1-agent-id-traces) - [GET /v1/markets, GET /v1/markets/{id} (§6.9)](#get-v1-markets-get-v1-markets-id-6-9) - [GET /v1/leaderboard](#get-v1-leaderboard) - [Contract-ownership + Integrity Health reads: GET /v1/agent/{id}/contracts, /baas](#contract-ownership-integrity-health-reads-get-v1-agent-id-contracts-baas) - [Verifiable Credentials: GET /v1/agent/{id}/vc](#verifiable-credentials-get-v1-agent-id-vc) - [Network benchmarks + protocol stats: GET /v1/benchmarks, /v1/stats](#network-benchmarks-protocol-stats-get-v1-benchmarks-v1-stats) - [XNS handle directory (Oracle-local, primary): GET /v1/xns/available, GET /v1/xns/handle/{handle}, POST /v1/xns/claim](#xns-handle-directory-oracle-local-primary-get-v1-xns-available-get-v1-xns-handle-handle-post-v1-xns-claim) - [On-chain XNS alias resolution (secondary): GET /v1/xns/resolve?handle=, GET /v1/agent/{id}/handle](#on-chain-xns-alias-resolution-secondary-get-v1-xns-resolve-handle-get-v1-agent-id-handle) - [Governance: GET /v1/governance/proposals](#governance-get-v1-governance-proposals) - [GET /v1/agent/{id}/wallet](#get-v1-agent-id-wallet) - [Server-side telemetry-signal re-derivation (derive.rs)](#server-side-telemetry-signal-re-derivation-derive-rs) - [Signed telemetry schema v3 (handlers.rs + spanschema.rs)](#signed-telemetry-schema-v3-handlers-rs-spanschema-rs) - [The OTLP/gRPC path (otlp.rs) — separate from telemetryevents, unauthenticated](#the-otlp-grpc-path-otlp-rs-separate-from-telemetryevents-unauthenticated) - [PHI backstop on POST /v1/telemetry/ingest](#phi-backstop-on-post-v1-telemetry-ingest) - [Judge evaluations (storage only — no judge implementation)](#judge-evaluations-storage-only-no-judge-implementation) - [On-chain client (chain.rs)](#on-chain-client-chain-rs) - [Anchoring](#anchoring) - [Canonical JSON signing — RFC 8785 JCS](#canonical-json-signing-rfc-8785-jcs) - [Finality-bearing agent directory](#finality-bearing-agent-directory) - [State](#state) ## Workspace - **`scoring-core`** — dependency-free; the **one** place the AIS formula is computed. Everyone else reads the HTTP API. - **`backend`** — Axum server, sqlx persistence, Redis rate limiting, `bb verify` ZK verification, Merkle building, and the `alloy` on-chain read client. ## HTTP API ``` POST /v1/agent/register GET /v1/agent/{id} GET /v1/agents GET /v1/agent/{id}/ais GET /v1/agent/{id}/ais/history GET /v1/agent/{id}/compliance GET /v1/agent/{id}/wallet GET /v1/agent/{id}/telemetry GET /v1/agent/{id}/telemetry/volume GET /v1/agent/{id}/otel/volume GET /v1/agent/{id}/traces GET /v1/agent/{id}/contracts GET /v1/agent/{id}/baas GET /v1/agent/{id}/vc GET /v1/agent/{id}/handle POST /v1/agent/{id}/verify/dns/challenge POST /v1/agent/{id}/verify/dns POST /v1/agent/{id}/verify/github/challenge POST /v1/agent/{id}/verify/github POST /v1/agent/{id}/verify/tee/challenge POST /v1/agent/{id}/verify/tee POST /v1/agent/{id}/verify/kyc/challenge POST /v1/agent/{id}/verify/kyc GET /v1/agent/{id}/verify POST /v1/agent/{id}/verify/{verification_id}/revoke/challenge POST /v1/agent/{id}/verify/{verification_id}/revoke GET /v1/traces/{trace_id} POST /v1/telemetry/ingest GET /v1/markets GET /v1/markets/{id} GET /v1/leaderboard GET /v1/benchmarks GET /v1/stats GET /v1/xns/resolve?handle= GET /v1/governance/proposals GET /v1/stream GET /v1/agent/{id}/stream GET /healthz ``` Full request-pipeline order for `POST /v1/telemetry/ingest` (PHI scan → agent lookup → rate limit → signature → server-side re-derivation → ZK → compliance → nonce replay → storage): [Telemetry Ingestion Pipeline](telemetry-ingestion.md). ### `GET /v1/agent/{id}/telemetry`, `GET /v1/agent/{id}/traces` Real history queries returning lists of past telemetry events and judge traces respectively, queried directly from Postgres (`telemetry_events` and `judge_evaluations` tables) and returned as structured DTO arrays. Used by the Dashboard UI to render live feeds and historical execution logs. **The honesty crux:** `POST /v1/agent/register` independently calls `XibalbaAgentRegistry.resolveDID` and rejects (`400`) if the client's claimed [7 primitives](agent-primitives.md) don't match on-chain state. This is what makes "the chain is the source of truth" real, not decorative. Request body (`RegisterAgentRequest`, `handlers.rs`): `did` (required — not `agent_id`), `did_document`, `primitives` (exactly the 7 `PrimitiveSetDto` fields), `ed25519_pubkey_hex`/`eth_address_hex` (`Option` each, but the handler 400s if *both* are absent), `verification_tier` (`i32`, defaults to `0`). Now documented in `docs/INTERFACE_CONTRACT.md` §6.3 — it was silent on this schema until 2026-07-09, which is how `integrity-sdk`'s client drifted from it undetected (see [integrity-sdk](integrity-sdk.md)'s "Fixed 2026-07-09" note). On 2026-07-16, we enhanced `chain.rs` (`resolve_primitives_by_did`) to log the actual RPC contract call error details when `resolveDID` fails instead of silently mapping them to `UnknownDid`. This resolves a major diagnostic visibility gap during stack startup and network resolution. As of 2026-08-31, every cached primitive read is explicitly scoped to the Oracle's configured chain. The database module intentionally exposes no unscoped primitive-read helper: direct agent reads, fleet enrichment, AIS and compliance calculation, leaderboard refresh, and reverse SovereignAgent-to-DID lookup all require `chain_id`. Cache rows from another network and legacy rows with `chain_id IS NULL` are treated as misses rather than evidence about the active chain; the underlying agent identity remains listable. Leaderboard snapshots and freshness markers are keyed by chain and each refresh atomically replaces that chain's derived rows, preventing stale entries from surviving a partial current-chain resolution. This is a cache-isolation property, not proof that a matching-chain cached row is fresh or that the chain itself is available. ### `GET /v1/markets`, `GET /v1/markets/{id}` (§6.9) Real reads of every `IntegrityMarket` clone via `MarketFactory`. Enumeration is concurrent (`futures::future::join_all`), not a serial loop, and cached in Postgres (`markets_cache` + a single-row `markets_index_sync` marker) behind a 30s staleness window (`handlers::MARKETS_CACHE_STALENESS_SECS`) — a documented tradeoff between RPC load and freshness, not silent staleness. On a cache miss/stale hit, `list_markets` re-enumerates `MarketFactory.allMarketsCount()` (not just refreshes existing rows) so a newly-created market is actually discovered. `GET /v1/markets/{id}` (`{id}` = the market's contract address) returns `question`, `outcomeCount`, `resolved`, `winningOutcome`, `resolveDeadline`, `totalStaked`, and `outcomeStaked` per outcome (the real pari-mutuel pool — cheap public-getter reads). An optional `?agent=0x...` query param adds a real, single `getPosition` read as `your_position`. **Documented gap:** enumerating every holder's position across a market requires indexing `PositionEntered` events, which this pass does not build — `positions_note` in the response says so explicitly rather than silently omitting it. Uint256 amounts (`min_ais_to_enter`, `total_staked`, `outcome_staked[i]`) are always decimal **strings** in both the DB (`TEXT`, not a numeric type sqlx isn't configured for) and the JSON DTOs — a `uint256` can exceed both `i64` and `f64`'s safe integer range. **Contract-vs-brief discrepancy found while building this:** `MarketFactory` has no `allMarkets() returns (address[])` getter — `address[] public allMarkets` only auto-generates an indexed `allMarkets(uint256) returns (address)` getter. Enumeration is `allMarketsCount()` + concurrent `allMarkets(i)` reads (`ChainClient::all_market_addresses`). By-creator listing uses the real `getMarketsByCreator(address) returns (address[])` function, not the `marketsByCreator` mapping's auto-getter (which also needs an index). ### `GET /v1/leaderboard` Ranks agents by real `ReputationRegistry.effectiveScore` (decimal string, same `ChainClient::effective_score` method `chain.rs` already had). **No fabricated P&L**: `realized_pnl` is always `null` — computing it for real would require indexing `IntegrityMarket` `PositionEntered`/`MarketResolved`/`PayoutClaimed` events across every market, out of scope for this pass. An honestly-incomplete ranking, not a silent mock. Only agents with a resolvable on-chain `PrimitiveSet` (cached, or live-resolved and cached on the fly) appear. ### Contract-ownership + Integrity Health reads: `GET /v1/agent/{id}/contracts`, `/baas` `/contracts` returns the `IntegrityMarket` clones an agent deployed and owns (`MarketFactory.getMarketsByCreator` on its `SovereignAgent`, then live per-market reads). `/baas` enumerates the `SmartBAA` escrows where the agent is the business associate, from `SmartBAAFactory.BAACreated` logs (no reverse index exists on-chain, so the event log is the only enumeration path) + each escrow's live status. Both are real on-chain reads; `/baas` returns a clean `MissingSingleton` (400) where the Integrity Health `SmartBAAFactory` isn't deployed. ### Verifiable Credentials: `GET /v1/agent/{id}/vc` Issues a signed **W3C Verifiable Credential** (`AgentIntegrityCredential`) for the agent's current AIS + verification tier — a real Ed25519 `Ed25519Signature2020` proof over the canonicalized credential, signed by the oracle's issuer key (`crate::vc`, `VC_ISSUER_SEED` env, `did:key` issuer). Not a mock: the returned JSON-LD credential is independently verifiable against the issuer's public key. Consumed by the dashboard's `DIDExplorer`. ### Network benchmarks + protocol stats: `GET /v1/benchmarks`, `/v1/stats` `/benchmarks` aggregates network-wide telemetry per model (`db::benchmark_by_model`, `GROUP BY payload->>'model' HAVING COUNT >= 3`) into a per-model/provider stability + grounding benchmark (a simulated AIS from the same entropy/grounding math scoring-core uses). `/stats` is the minimal protocol-wide singleton supplement the dashboard can't cheaply derive from its per-agent loop: marketplace volume (sum of cached market `total_staked`) + `A2ACapitalPool` escrow/release totals; `tvl` is composed client-side for a single source of truth. ### XNS handle directory (Oracle-local, primary): `GET /v1/xns/available`, `GET /v1/xns/handle/{handle}`, `POST /v1/xns/claim` As of 2026-09-26 (PRODUCTION_GAPS.md #71) this, not the on-chain resolver below, is the primary XNS system: a chat-app-style unique handle per agent, chosen by the user, stored in the oracle's own `xns_handles` Postgres table (`agent_id` PK, unique index on `lower(handle)`), independent of chain state entirely. `GET /v1/xns/available?handle=` checks availability and returns suggested alternatives when taken; `GET /v1/xns/handle/{handle}` resolves a handle → `agent_id`; `POST /v1/xns/claim` claims a handle for an agent (409 if taken). `POST /v1/agent/register` now takes a mandatory `handle` and claims it transactionally with primitive registration (`db::register_agent_with_handle`) — every agent registered from here on has a handle unconditionally; `list_agents` sources it via `db::get_xns_handles_for_agents`. Agents without a handle (registered before this change) are identified by DID. Consumed by `XNSSearchService` (dashboard) for search, availability, and claim. ### On-chain XNS alias resolution (secondary): `GET /v1/xns/resolve?handle=`, `GET /v1/agent/{id}/handle` Live reads of the `XibalbaNameService` singleton — now an optional, explicitly secondary *alias* listing, not the primary handle system (see above). `/xns/resolve` maps a human handle → `SovereignAgent` (and, best-effort via `db::did_by_sovereign_agent`, the reverse DID); `resolve_handle` gates on `handleExists` first since the contract's `resolve()` reverts on an unclaimed handle, so an unregistered handle returns a null address, not an error. `/agent/{id}/handle` is the reverse: an agent's `primaryHandle`. Both return `MissingSingleton` (**HTTP 400**) until XNS is deployed on the target network — an honest "not deployed", never a fabricated handle. Registering an on-chain alias for an Oracle-claimed handle is not yet built (`XNSRegisterForm.tsx`, which wrote directly to this contract, was removed 2026-09-26 — see PRODUCTION_GAPS.md #71 — because it bypassed the Oracle-local uniqueness table and could disagree with it about who holds a handle). ### Governance: `GET /v1/governance/proposals` Live enumeration of `IntegrityGovernance` proposals by index (`ChainClient::read_proposals`: `proposalCount()` then `getProposal(i)` + `state(i)` for `i in 1..=count`, 1-based, newest-first — an index loop, deliberately not log-scanning). Each DTO carries proposer, target, FOR/AGAINST tallies (decimal-string wei of ITK), the timelock ETA, and the derived `state` (`Active`/`Defeated`/`Succeeded`/`Queued`/`Executed`/`Expired`/`Canceled`). Returns `MissingSingleton` (**HTTP 400**) until `IntegrityGovernance` is deployed; the dashboard's `GovernancePanel`/`GuardianPilot` degrade to their honest "Not Yet Live" state on that (never a live-but-empty proposal list). See [Governance](governance.md). ### `GET /v1/agent/{id}/wallet` Real `IntegrityToken.balanceOf(sovereignAgent)` read (decimal string), plus open positions: a real `getPosition` read against every cached market for that agent's `SovereignAgent` address, filtered to `amount > 0 && !claimed`. **Documented gap:** `transaction_history` is always `null` — transfer/stake/payout history requires indexing on-chain events (`Transfer`, `PositionEntered`, `PayoutClaimed`, ...), not built this pass. ### Server-side telemetry-signal re-derivation (`derive.rs`) `POST /v1/telemetry/ingest` does not trust a client's self-reported `derived_signals`. It independently recomputes entropy/grounding from the same signed request's raw `otel_spans` content and accepts sacrifice/compliance only from independently admissible evidence. Token-derived sacrifice and client compliance remain audit-only proxies and fail closed in authoritative AIS when their evidence is absent. Only the oracle's own admissible values feed `telemetry_events`/[AIS](ais.md) — the client's claim is stored purely as an audit-trail comparison. Two polarity/calibration bugs were fixed at this exact call site in the same pass: `performance_variance` was receiving the SDK's stability-score polarity (1.0=best) into a column `scoring-core` treats as a true variance (0.0=best) — backwards for every agent until fixed; and `gpu_hours_verified` now receives an hours-equivalent proxy rather than a pre-normalized `[0,1]` index, removing a double-log-compression that capped max-sacrifice agents around ~100/1000 instead of ~1000. Full pipeline-order writeup (this is step 5 of an 11-step ordered handler sequence — PHI scan, agent lookup, rate limit, and signature verification all run first): [Telemetry Ingestion Pipeline](telemetry-ingestion.md). Formula and trust-model detail: [AIS](ais.md). ### Signed telemetry schema v3 (`handlers.rs` + `span_schema.rs`) The signed `POST /v1/telemetry/ingest` envelope currently accepts schema versions 1 through 3. Version 3 is the first version whose `otel_spans` array is structurally validated before the PHI scan and persistence path: the allowlisted shape, metadata/property counts, text and batch bounds, numeric ranges, and covered-entity address format are checked. Unknown or oversized shapes fail closed with a client-visible bad request. The validator does not apply to the separate unauthenticated OTLP receiver and does not make OTLP data eligible for AIS. The SDK and oracle version constants must move together. Historical envelopes without a `schema_version` remain valid, while future versions are rejected until the oracle is upgraded. See `docs/INTERFACE_CONTRACT.md` §4.2a for the wire-level contract. ### The OTLP/gRPC path (`otlp.rs`) — separate from telemetry_events, unauthenticated Lights up the SDK's already-real `OTLPSpanExporter`/`OTLPMetricExporter` (`telemetry/core.py::init_telemetry`, gRPC `localhost:4317`), which previously exported into a void — nothing listened on that port. Real spans arrive with **no Ed25519/secp256k1 signature envelope** (unlike `POST /v1/telemetry/ingest`), so this deliberately does NOT touch `telemetry_events`/AIS — feeding unauthenticated spans into scoring would let anyone move an agent's score. Trace export is fully implemented (PHI-scanned via the same `crate::phi` backstop, persisted to a separate `otel_spans` table, broadcast over SSE); metrics export is accepted (so the SDK's metric exporter gets a real gRPC response) but not yet parsed or persisted. ### PHI backstop on `POST /v1/telemetry/ingest` `src/phi.rs` mirrors `integrity-sdk/integrity_sdk/security/redactor.py`'s regex categories (`PRIVATE_KEY`, `API_KEY`, `SSN`, `CREDIT_CARD`, `EMAIL`, `PHONE`, `MRN`) as a server-side defense-in-depth backstop: it recursively scans every JSON **string** leaf in `otel_spans` (and an optional `judge_evaluation`), and rejects (`400`, `AppError::PhiDetected`) if any raw, unredacted pattern is found — belt-and-suspenders for a buggy/bypassed client, never a replacement for the SDK's own client-side redaction. Runs before any DB/RPC work in `ingest_telemetry`, so it fails fast. Only string values are scanned (numeric/structural fields are skipped) — the SDK's redactor never touches those either, so scanning them here would just produce false positives on values nobody was ever going to redact. ### Judge evaluations (storage only — no judge implementation) `TelemetryIngestRequest.judge_evaluation` (optional) persists into the new `judge_evaluations` table (`run_id`, `judge_model`, `verdict`, `score`, `rationale_summary`, linked to the telemetry event). **Deliberately NOT part of the signed envelope** — `ingest_telemetry`'s `signable` JSON (what `crypto::verify_agent_signature` checks) does not include this field, so adding a judge evaluation never requires a client to re-sign, and no existing client's signature breaks. It rides along as an unauthenticated sidecar on an otherwise-authenticated request. No judge/rubric implementation exists anywhere in this codebase — this is plumbing only, per the task scope (a Xibalba Solutions product decision not yet made). ## On-chain client (`chain.rs`) Read-only via `alloy` (stored as `DynProvider` so `sol!` bindings work): resolves an agent's `PrimitiveSet`, reads `ReputationRegistry.effectiveScore`/`isZkBoosted`, `ComplianceGate.vertical`/`isHealthcareCompliant`, `MarketFactory` enumeration, per-market `IntegrityMarket` view state, `getPosition`, `IntegrityToken.balanceOf`, `SmartBAAFactory`/`SmartBAA` (Integrity Health), and — new — `XibalbaNameService` (`resolve_handle`/`primary_handle`) and `IntegrityGovernance` (`read_proposals`). Optional singleton addresses (`MarketFactory`, `IntegrityToken`, `A2ACapitalPool`, `SmartBAAFactory`, `XibalbaNameService`, `IntegrityGovernance`) are `Option
` in the deployments-file parse (`Singletons`), not required: a deployments file that predates any of them still parses, and handlers needing an absent one return a clean `ChainError::MissingSingleton` — mapped to **HTTP 400** in `error.rs` (a deployment-shape fact, not a transient RPC failure), shared across the market/health/XNS/governance endpoints — instead of this client failing to even connect. ## Anchoring Telemetry leaves batched into a keccak256 [Merkle tree](merkle-batching.md); because `StateAnchor` is per-agent, the same epoch root is submitted to each participating agent's own `StateAnchor` clone — a documented gas tradeoff. ## Canonical JSON signing — RFC 8785 JCS `crypto::canonical_json_bytes`, the byte representation checked for signed telemetry, uses `serde_jcs`. Python producers use the `jcs` package. This shared RFC 8785 rule replaces the earlier custom ASCII formatter and pins both ECMAScript number serialization and raw non-ASCII UTF-8 across runtimes. ## Finality-bearing agent directory `GET /v1/agents/snapshot` is the directory interface consumed by Cortex. Its response includes `chain_id`, `block_number`, `finalized_block_number`, `finalized_block_hash`, `snapshot_id`, and `finalized`. The `finalized` value is the conjunction of the operator approval setting `AGENT_DIRECTORY_FINALIZED` and the execution client's finalized cursor covering the directory cursor. When approval is enabled, the handler anchors `block_number` to the execution client's finalized block before evaluating the conjunction; it does not label the moving latest head as finalized. The Cortex sync worker refuses any snapshot whose `finalized` value is false, whose finalized cursor is missing or malformed, or whose finalized cursor does not cover the directory cursor. The setting must therefore remain disabled until registration state has been independently checked at a finalized block. On 2026-09-14, read-only Base Sepolia historical calls at block `46830231` resolved all five directory agents with `exists=true`; the Oracle was then restarted with `AGENT_DIRECTORY_FINALIZED=true` and returned a finalized snapshot at that same block with five agents. ## State **80 backend + scoring-core lib tests** (confirmed via a real run — 72 backend, 8 scoring-core; up from 54 with the `derive.rs` re-derivation module's own parity-with-`derive.py` unit tests among the additions), including `src/phi.rs` unit tests covering every PHI category, the already-redacted-marker non-reflag case, and the numeric-vs-string-field scan boundary), plus a real full-stack e2e (`tests/e2e.rs`, **9 tests**, opt-in via `ORACLE_E2E=1`) that stands up live anvil + `Deploy.s.sol` (which now deploys the market layer as part of genesis) + a real SDK-registered agent + Postgres + Redis + the HTTP server, and asserts accept-correct / reject-fabricated primitives, AIS scoring, the live compliance read, a real (empty, pre-any-market) `GET /v1/markets`, a real one-entry `GET /v1/leaderboard`, a real `GET /v1/agent/{id}/wallet` balance read, the PHI backstop's real HTTP-level 400, and — new — `oracle_e2e_recomputed_grounding_overrides_inflated_client_claim`, proving an agent that claims an inflated grounding score while its own signed `otel_spans` contain hallucination markers gets the oracle's real, low recomputation stored and scored, never the client's claim. **Documented follow-up, not built this pass:** a full market lifecycle e2e (`enterPosition`/`resolve`/`claimPayout` through a second real registered agent) — out of scope for this task; the light e2e above proves the real binding/parse/handler path without that heavier setup. Related: [Telemetry Ingestion Pipeline](telemetry-ingestion.md), [AIS](ais.md), [agent primitives](agent-primitives.md), [Interface Contract](INTERFACE_CONTRACT.md).