Skip to content

Oracle Service Testing and Adversarial Scenarios

Jump to bottom
RunTimeAdmin edited this page Nov 3, 2026 · 33 revisions

Oracle Service Testing & Adversarial Scenarios

**Referenced Files in This Document** - [README.md](https://github.com/RunTimeAdmin/Sigvara/blob/main/README.md) - [adversarial.test.js](https://github.com/RunTimeAdmin/Sigvara/blob/main/oracle/adversarial.test.js) - [scoring.test.js](https://github.com/RunTimeAdmin/Sigvara/blob/main/oracle/scoring.test.js) - [payments.test.js](https://github.com/RunTimeAdmin/Sigvara/blob/main/oracle/payments.test.js) - [metrics.test.js](https://github.com/RunTimeAdmin/Sigvara/blob/main/oracle/metrics.test.js) - [epoch-policy.test.js](https://github.com/RunTimeAdmin/Sigvara/blob/main/oracle/epoch-policy.test.js) - [payment-scan.test.js](https://github.com/RunTimeAdmin/Sigvara/blob/main/oracle/payment-scan.test.js) - [config.test.js](https://github.com/RunTimeAdmin/Sigvara/blob/main/oracle/config.test.js) - [secrets.test.js](https://github.com/RunTimeAdmin/Sigvara/blob/main/oracle/secrets.test.js) - [store.test.js](https://github.com/RunTimeAdmin/Sigvara/blob/main/oracle/store.test.js) - [state-serialize.test.js](https://github.com/RunTimeAdmin/Sigvara/blob/main/oracle/state-serialize.test.js) - [chain-safety.test.js](https://github.com/RunTimeAdmin/Sigvara/blob/main/oracle/chain-safety.test.js) - [attest.test.js](https://github.com/RunTimeAdmin/Sigvara/blob/main/oracle/attest.test.js) - [http.test.js](https://github.com/RunTimeAdmin/Sigvara/blob/main/oracle/http.test.js) - [robustness.test.js](https://github.com/RunTimeAdmin/Sigvara/blob/main/oracle/robustness.test.js) - [scoring.js](https://github.com/RunTimeAdmin/Sigvara/blob/main/oracle/scoring.js) - [payments.js](https://github.com/RunTimeAdmin/Sigvara/blob/main/oracle/payments.js) - [metrics.js](https://github.com/RunTimeAdmin/Sigvara/blob/main/oracle/metrics.js) - [epoch-policy.js](https://github.com/RunTimeAdmin/Sigvara/blob/main/oracle/epoch-policy.js) - [payment-scan.js](https://github.com/RunTimeAdmin/Sigvara/blob/main/oracle/payment-scan.js)

Table of Contents

  1. Introduction
  2. Project Structure
  3. Core Components
  4. Architecture Overview
  5. Detailed Component Analysis
  6. Dependency Analysis
  7. Performance Considerations
  8. Troubleshooting Guide
  9. Conclusion
  10. Appendices

Introduction

This document explains how the Sigvara oracle tests validate reputation scoring, payment verification, metrics collection, epoch scheduling, and chain-scanning behavior. It focuses on adversarial scenarios that simulate malicious actors, sybil rings, wash trading, manipulation attempts against community and propagation factors, and operator disagreement. It also provides guidance for performance testing epoch processing, score computation, API response times, load handling, monitoring coverage, environment setup, and validation of decentralization properties.

The repository includes:

  • Contract-level tests under test/ and Foundry configuration: 739 tests as of 3 November 2026, including 21 invariants. Without a fork URL 727 run and 12 skip.
  • Oracle Node.js tests under oracle/: 902 tests, no network access required.
  • SDK tests under packages/sdk/test/: 300 tests, 8 of which skip without a live deployment.
  • Documentation describing protocol architecture, reputation model, and deployment status.

Section sources

Project Structure

The oracle test suite is organized by feature:

  • Adversarial scoring tests assert economic resistance to wash rings and sybil bootstrap attacks.
  • Scoring tests validate each factor and determinism across operators.
  • Payment tests verify settlement integrity, decay, diversity, self-payment detection, and outcome semantics.
  • Metrics tests ensure every counter/gauge is declared, exposed, and structurally valid.
  • Epoch policy tests enforce proposal/finalize timing, checker divergence rules, and safe concurrency.
  • Payment scan tests implement ADR 0003 pull scanning with pacing, batching, canary checks, and checkpoint safety.
  • Payer-signed outcome tests (payer-outcome.test.js) check the EIP-712 signature by recovery and through ERC-1271 against the shared vectors in test/vectors/payment-outcome.json, the refusal codes, precedence over a token holder's outcome, GET /outcomes, and the import from other operators, including forged items, a peer dating an item ahead and the import grace. evidence-as-of.test.js covers the checker reading the evidence as it stood at a proposal.
  • Startup and state tests (config.test.js, secrets.test.js, store.test.js, state-serialize.test.js) cover the refusal of unusable settings and malformed keys, state loading with .bak recovery, and the state file written in pieces.
  • Chain safety tests (chain-safety.test.js) cover a transaction that never lands, an absurd fee, an RPC on the wrong chain and a proposal the contract would refuse, using real ethers objects against a fake JSON-RPC endpoint.
  • End-to-end tests (attest.test.js, http.test.js, robustness.test.js, payer-reads.test.js, scan-checkpoint.test.js) run the real index.js and watcher.js as child processes against a stubbed chain from test-support/, so startup refusals, exit codes, request handling and the order in which an epoch charges and proposes are tested as they happen.
graph TB
subgraph "Oracle Test Suite"
ADV["Adversarial Tests"]
SCORING["Scoring Tests"]
PAYMENTS["Payment Verification Tests"]
METRICS["Metrics Tests"]
EPOCH["Epoch Policy Tests"]
SCAN["Payment Scan Tests"]
end
ADV --> SCORING
ADV --> PAYMENTS
SCORING --> PAYMENTS
METRICS --> EPOCH
SCAN --> PAYMENTS
Loading

Diagram sources

Section sources

Core Components

This section maps the core components exercised by the tests and their responsibilities.

  • Scoring pipeline: pure functions compute fee activity, success rate, tenure, external trust, community flags, and propagation. The total is capped at 100. Deterministic time input prevents operator drift.
  • Payment verification: validates ERC-20 Transfer logs, enforces minimum amount, confirmations, asset matching, payer provenance, self-payment refusal, and outcome semantics.
  • Metrics registry: in-memory counters and gauges exported as Prometheus text; tests assert completeness, family grouping, TYPE declarations, and evidence cache exposure.
  • Epoch policy: decides propose/skip/finalize actions, validates timer intervals, tolerance bounds, divergence comparison, and chunked mapping concurrency, and holds the clock-skew check and the stale-scoring rule that /health uses.
  • Payment scan (ADR 0003): pulls Transfer logs, plans credits, batches recipients, paces RPC calls, verifies query mechanism via canary, and advances checkpoints safely.

Section sources

Architecture Overview

The oracle computes a normalized reputation score from payment-backed attestations and external signals, proposes it on-chain, and exposes it through APIs. A second bonded operator runs in checker mode to detect divergence.

sequenceDiagram
participant Client as "Client / Counterparty"
participant Oracle as "Sigvara Oracle"
participant Chain as "Arc Testnet"
participant Reputation as "SigvaraReputation"
participant Checker as "Checker Operator"
Client->>Oracle : Attestation + settlement txHash
Oracle->>Chain : Verify transfer logs and timestamps
Chain-->>Oracle : Receipt, block timestamp
Oracle->>Oracle : Compute six-factor score
Oracle->>Reputation : proposeReputation(didHash, data, root)
Reputation-->>Oracle : ScoreProposed(proposedAt)
Note over Reputation : Challenge window open
alt Committee rejects
Committee->>Reputation : rejectReputation(didHash)
else Window elapses unchallenged
Client->>Reputation : finalizeReputation(didHash)
Reputation-->>Client : ReputationUpdated(totalScore)
end
Checker->>Oracle : Re-score pending proposal
Checker->>Reputation : finalize only if agreed
Loading

Diagram sources

Section sources

Detailed Component Analysis

Adversarial Scoring Tests

These tests feed attacker-crafted payment sets into the real scoring pipeline without stubs, ensuring changes cannot weaken caps silently. Key scenarios:

  • Single-payer cap: one wallet cannot buy more than maxPerPayer points regardless of volume or frequency.
  • Wash ring ceiling: multiple wallets controlled by an attacker reach a bounded total; the design intentionally reduces fee/success weight and increases tenure/external trust to make rings expensive.
  • Sybil bootstrap: zero-scored counterparties raise no caps and grant no propagation; farmed standing does not launder into targets.
  • Time and decay: tenure rewards elapsed paid activity; abandoned farms decay rather than holding scores.
  • Failure reporting: hiding failures is bounded by prior; a handful of perfect outcomes cannot outrank sustained evidence.
  • Flags: community damage is bounded; five flags cost exactly the community factor.
flowchart TD
Start(["Adversarial Scenario"]) --> BuildEvents["Build wash ring events<br/>by wallet count and span"]
BuildEvents --> ScorePipeline["Run payments.byPayer / diversifiedVolume / diversifiedAttestations / propagationScore / activityWindow"]
ScorePipeline --> ComputeScore["computeScore with registeredAt, flags, externalScore"]
ComputeScore --> AssertBounds{"Total and factors within expected bounds?"}
AssertBounds --> |Yes| Pass["Test passes"]
AssertBounds --> |No| Fail["Test fails — caps weakened"]
Loading

Diagram sources

Section sources

Scoring Algorithm Tests

Unit tests cover each factor and the composite score:

  • Fee score: proxy when payment verification is off; measured volume when required.
  • Success score: Bayesian prior prevents single observations from maxing out; decaying records fade toward zero.
  • Age/tenure: logarithmic curve over paid activity span; recency weights recent activity; calendar fallback preserved when verification is off.
  • Community score: flags reduce points; floor at zero.
  • External and propagation: clamped to contract caps; propagation uses HARD standing only.
  • Determinism: same inputs and chain clock produce identical scores across hosts.
classDiagram
class Scoring {
+feeScore(attestationTotal)
+successScore(successful, total, prior)
+ageCurve(days)
+ageScore(registeredAtSeconds, activity, nowMs)
+communityScore(unresolvedFlags)
+computeScore(opts)
}
class Payments {
+decayedVolume(events, halfLifeMs, now)
+decayedAttestations(events, halfLifeMs, now)
+activityWindow(events, halfLifeMs, now)
+propagationScore(events, payerScores)
}
Scoring --> Payments : "uses measured factors"
Loading

Diagram sources

Section sources

Payment Verification and Integrity Tests

Tests validate:

  • Config parsing and defaults.
  • Log matching and multi-leg transactions.
  • Settlement verification: payer provenance, reverted/unmined transactions, minimum amount, confirmations, RPC errors, malformed hashes, timestamp attribution.
  • Decay and attestations: age-weighted volume and success ratios; pulled payments carry success: null, excluded from both sides of the ratio.
  • Self-payment detection and case-insensitive address matching.
  • Payer diversity: per-payer caps preserve ratios and distinct counts.
  • Web of trust: HARD standing scaling and propagation scoring; ring bootstrap blocked.
  • Clock determinism: recency and decay depend on explicit now; two operators agree when given the same chain clock.
  • Intake path consistency: attested and scanned paths attribute split payments identically.
flowchart TD
Input(["verifyPayment(txHash, payTo)"]) --> ValidateTx["Validate txHash format"]
ValidateTx --> FetchReceipt["Fetch receipt"]
FetchReceipt --> CheckStatus{"status == 1?"}
CheckStatus --> |No| ErrorReverted["Throw 'reverted'"]
CheckStatus --> |Yes| Confirmations["Check confirmations >= minConfirmations"]
Confirmations --> MatchTransfer["Match Transfer logs to asset/payTo"]
MatchTransfer --> SumLegs["Sum legs by sender"]
SumLegs --> MinAmount{"amount >= minAmount?"}
MinAmount --> |No| ErrorBelowMin["Throw 'below_minimum'"]
MinAmount --> |Yes| BlockTimestamp["Read block timestamp"]
BlockTimestamp --> TimestampValid{"timestamp finite?"}
TimestampValid --> |No| ErrorRpcTs["Throw 'rpc_error' timestamp unavailable"]
TimestampValid --> |Yes| ReturnVerified["Return verified payment with settledAt, legs, payer"]
Loading

Diagram sources

Section sources

Metrics Collection Tests

Tests ensure:

  • Counters increment by default or specified amount; unknown names are ignored.
  • Gauges set values; unknown names are ignored.
  • Prometheus exposition includes HELP/TYPE comments, metric families grouped, and all declared metrics rendered.
  • Evidence cache stats are conditionally included.
  • Every metric name used in source files is declared and exported; no silent no-ops.
flowchart TD
Inc["metrics.inc(name, amount)"] --> Registry["In-memory counters/gauges"]
Set["metrics.set(name, value)"] --> Registry
ToProm["metrics.toPrometheusText(extra)"] --> Render["Render HELP/TYPE families"]
Registry --> Render
Extra["extra.evidenceCache"] --> Render
Render --> Output["Prometheus text output"]
Loading

Diagram sources

Section sources

Epoch Policy and Concurrency Tests

Tests validate:

  • Proposal lifecycle: propose, skip during challenge window, finalize-then-propose after window.
  • Timer interval validation: rejects NaN, Infinity, too short, too long, and below minimum.
  • Checker mode: never finalize or overwrite a disputed score; small skew tolerated; offsetting factor errors still diverge.
  • Divergence comparison: reports differing factors and totals; incomparable factors fail closed.
  • Chunked mapping: preserves order, respects concurrency limits, handles empty lists, rejects invalid sizes, propagates mapper errors.
  • Clock skew: a chain clock far ahead of or behind the host's is refused and the reason says which; a timestamp that is not a number is refused rather than compared.
  • Stale scoring: after a success the limit is two epoch intervals from it, and a process that has never finished an epoch is given two intervals from its start.
stateDiagram-v2
[*] --> Idle
Idle --> Propose : "no pending"
Propose --> Pending : "proposal created"
Pending --> Skip : "challenge window open"
Skip --> Skip : "still within window"
Skip --> FinalizeThenPropose : "window elapsed"
FinalizeThenPropose --> Pending : "new proposal"
Pending --> Finalize : "checker agrees and the primary is a grace late"
Pending --> Diverged : "checker disagrees"
Diverged --> Pending : "committee action"
Loading

Diagram sources

Section sources

Payment Scan (ADR 0003) Tests

Tests validate:

  • Decoding Transfer logs and ignoring non-Transfers.
  • Crediting payments to registered agents with success: null.
  • Grouping multi-leg transactions per sender and summing amounts.
  • Refusals: self-payment, operator self-payment, below minimum, already credited; all reasons named.
  • Safe head and checkpoint logic: avoids reorg stranding, never commits past unresolved blocks, and holds the checkpoint after a failed canary so the range is scanned again.
  • Canary verification: proves query mechanism works even when scans return nothing.
  • Pacing: waits between calls except first/last; scales across recipient batches; default pace derived from measured limiter behavior.
flowchart TD
Logs["Batch getLogs(recipients, blocks)"] --> Decode["decodeTransfer()"]
Decode --> Plan["planCredits(decoded, opts)"]
Plan --> Credits["credits[]"]
Plan --> Skipped["skipped[] with reason"]
Credits --> Store["Store credits"]
Skipped --> Audit["Audit skipped reasons"]
Store --> Checkpoint["checkpointAfter(unresolved, from, to)"]
Checkpoint --> NextRange["Advance range safely"]
Loading

Diagram sources

Section sources

Dependency Analysis

The oracle modules have clear dependencies:

  • Adversarial tests depend on payments and scoring to run the production pipeline.
  • Scoring depends on payments for measured factors and propagation.
  • Payment scan shares Transfer topic constants and decoding patterns with payments.
  • Metrics is independent but consumed by other modules; tests statically scan usage.
  • Epoch policy is pure and tested independently; mapChunked supports epoch concurrency.
graph LR
ADV["adversarial.test.js"] --> PAY["payments.js"]
ADV --> SCR["scoring.js"]
SCR --> PAY
PS["payment-scan.js"] --> PAY
MET["metrics.js"] --> |exposed| ALL["All modules"]
EP["epoch-policy.js"] --> |pure| EPTEST["epoch-policy.test.js"]
Loading

Diagram sources

Section sources

Performance Considerations

Recommended strategies aligned with the test suite:

  • Epoch processing:

    • Use mapChunked to bound concurrency and preserve order; test peak in-flight operations and rejection of invalid chunk sizes.
    • Validate epoch intervals with epochIntervalError to avoid continuous loops due to misconfigured timers.
    • Measure propose/finalize attempt rates and throttling via metrics counters.

  • Score computation:

    • Ensure deterministic scoring by passing chain-derived now; tests assert identical outputs across hosts.
    • Validate decay curves and recency sensitivity; measure impact of half-life settings on score stability.

  • API response times:

    • Track HTTP requests and badge cache hit ratios; monitor evidence cache hits/misses/evictions/bytes.
    • Separate upstream RPC errors from internal errors using dedicated counters.

  • Load testing:

    • Simulate wash rings and high-frequency attestations; assert caps hold under volume.
    • Stress payment scan with large recipient sets and block ranges; verify pacing and batch sizing.
    • Exercise checker mode with many agents; ensure divergence detection remains fast and accurate.

  • Monitoring coverage:

    • Run static scans to ensure every metrics.inc/metrics.set name is declared and exposed.
    • Validate Prometheus exposition structure: HELP/TYPE present, families contiguous, samples typed.

Troubleshooting Guide

Common issues and diagnostics:

  • Payment verification failures:

    • Malformed tx hash: rejected before RPC calls.
    • Not found/reverted: transaction not mined or failed.
    • Unconfirmed: below minimum confirmations.
    • Below minimum: amount less than configured threshold.
    • RPC errors: provider down or unreadable block timestamp; never fall back to current time.

  • Score divergence:

    • Incomparable factors or NaN tolerance cause “diverged” verdict; fail closed to prevent silent agreement.
    • Offset factor errors still trigger divergence even if totals match.

  • Metrics silence:

    • Unknown metric names are ignored; static tests catch undeclared names and missing exposition.
    • Families must be contiguous; TYPE declarations must precede samples.

  • Epoch loop runaway:

    • Invalid EPOCH_HOURS parses to NaN or exceeds timer limit; validator rejects these cases explicitly.

  • A test that fails only under load:

    • "a rate-limited receipt lookup does not undo a transaction that was mined" in chain-safety.test.js failed in 15 to 20 percent of runs when eight copies of the suite ran at once, and passes alone every time. If it is the only red test, run it again before looking for a cause.

  • Payment scan empty results:

    • Canary verifies query mechanism; if canary fails, report “failed” instead of “passed”.
    • Checkpoints do not advance past unresolved blocks; next scan retries safely.

Section sources

Conclusion

The oracle test suite rigorously validates scoring economics, payment integrity, metrics observability, epoch scheduling, and decentralized operation. Adversarial tests pin economic ceilings against wash rings and sybil bootstrap. Payment tests ensure provenance, decay, diversity, and outcome semantics. Metrics tests guarantee complete and parseable exposition. Epoch policy tests protect against timer traps and unsafe finalization. Payment scan tests provide robust, paced, and verifiable chain indexing. Together, they support production-grade reliability, decentralization, and security.

Appendices

Example Test Scenarios and Edge Cases

  • Scoring edge cases:

    • New agent with no activity receives only community points.
    • Total never exceeds 100; external score clamped to 0–25.
    • Slashed-like scenario with high flags yields zero community contribution.
    • Tenure requires active span; idle old agents score zero.

  • Payment validation failures:

    • Split payments summed per sender; largest contributor names attester identity.
    • Timestamps come from block timestamps; NaN timestamps refused.
    • Pull scanner credits with success: null; outcomes arrive by push, and payer-signed ones then pass between operators by import.

  • Data integrity checks:

    • Both intake paths attribute multi-sender payments identically.
    • Distinct payers counted correctly; case-insensitive matching.
    • Propagation uses HARD standing only; farmed counterparties grant nothing.

Section sources

Environment Setup and Production Mimicry

  • Use Arc testnet deployment addresses and USDC gas for realistic conditions.
  • Enable payment verification and scanning in test environments to mirror production scoring behavior.
  • Configure epoch intervals and divergence tolerances within validated ranges.
  • Run full suites offline where possible; use mock providers for RPC-dependent tests.
  • Validate decentralization by running primary and checker modes with separate hosts and RPC providers.

Section sources

Home

Overview

Protocol Architecture

Smart Contracts Reference

Oracle Service

Reputation System Design

TypeScript SDK

Integration Guides

Deployment and Operations

Development and Testing

Advanced Topics

Clone this wiki locally