Skip to content

Testing Framework and Methodologies

Jump to bottom
RunTimeAdmin edited this page Nov 3, 2026 · 18 revisions

Testing Framework & Methodologies

**Referenced Files in This Document** - [foundry.toml](https://github.com/RunTimeAdmin/Sigvara/blob/main/foundry.toml) - [E2E.t.sol](https://github.com/RunTimeAdmin/Sigvara/blob/main/test/E2E.t.sol) - [StakingInvariant.t.sol](https://github.com/RunTimeAdmin/Sigvara/blob/main/test/StakingInvariant.t.sol) - [RegistrationHelper.sol](https://github.com/RunTimeAdmin/Sigvara/blob/main/test/helpers/RegistrationHelper.sol) - [vitest.config.ts](https://github.com/RunTimeAdmin/Sigvara/blob/main/packages/sdk/vitest.config.ts) - [integration.test.ts](https://github.com/RunTimeAdmin/Sigvara/blob/main/packages/sdk/test/integration.test.ts) - [package.json](https://github.com/RunTimeAdmin/Sigvara/blob/main/oracle/package.json) - [adversarial.test.js](https://github.com/RunTimeAdmin/Sigvara/blob/main/oracle/adversarial.test.js)

Table of Contents

  1. Introduction
  2. Project Structure
  3. Core Components
  4. Architecture Overview
  5. Detailed Component Analysis
  6. Dependency Analysis
  7. Performance Considerations
  8. Troubleshooting Guide
  9. Conclusion
  10. Appendices

Introduction

This document explains Sigvara’s multi-layered testing approach across Solidity smart contracts, the TypeScript SDK, and the Oracle service. It covers:

  • Foundry-based unit, integration, end-to-end, fuzz, and invariant tests for on-chain logic
  • Vitest-based TypeScript SDK testing with environment-driven live chain integration
  • Oracle service testing including adversarial scenarios and performance validation patterns
  • Test organization, naming conventions, fixtures, helper utilities, and best practices

The goal is to make the testing strategy accessible while preserving technical depth for maintainers and contributors.

Project Structure

Sigvara organizes tests by layer and runtime:

  • Solidity tests under test/ using Foundry (Forge), including helpers and invariants
  • TypeScript SDK tests under packages/sdk/test/ using Vitest
  • Oracle service tests under oracle/ using Node’s built-in test runner

As of 3 November 2026 the contract suite has 724 tests (each of the 19 invariants counts once), the oracle suite 852 and the SDK suite 300. Without a fork URL, 712 of the contract tests run and 12 skip, and 8 of the SDK tests, the integration tests, skip without a live deployment.

graph TB
subgraph "Solidity Tests"
E2E["E2E Integration<br/>test/E2E.t.sol"]
INV["Fuzz + Invariants<br/>test/StakingInvariant.t.sol"]
HELP["Helpers<br/>test/helpers/RegistrationHelper.sol"]
end
subgraph "TypeScript SDK Tests"
VCFG["Vitest Config<br/>packages/sdk/vitest.config.ts"]
INT["Integration Tests<br/>packages/sdk/test/integration.test.ts"]
end
subgraph "Oracle Service Tests"
PKG["Node Test Runner<br/>oracle/package.json"]
ADV["Adversarial Scoring<br/>oracle/adversarial.test.js"]
end
E2E --> HELP
INV --> HELP
INT --> VCFG
ADV --> PKG
Loading

Diagram sources

Section sources

Core Components

  • Foundry configuration defines source/output directories, remappings, optimizer settings, the EVM version (cancun, set explicitly), and fuzz profiles. It also configures RPC endpoints for Arc chains used by scripts and tests.
  • End-to-end Solidity tests cover the full agent lifecycle: registration, staking, reputation proposal/finalization, slashing, unbonding, and terminal slash outcomes.
  • Fuzz and invariant tests drive a handler contract that exercises all reachable transitions and asserts global properties like token conservation and collateralization.
  • TypeScript SDK integration tests run against a live EVM deployment when environment variables are present; otherwise they skip automatically.
  • Oracle adversarial tests validate scoring robustness against wash rings, sybil behavior, time decay, failure hiding, and flagging impacts.

Section sources

Architecture Overview

The testing architecture spans three layers:

sequenceDiagram
participant Forge as "Foundry (forge test)"
participant Contracts as "On-chain Contracts"
participant Handler as "StakingHandler (fuzz)"
participant TS as "Vitest (SDK tests)"
participant Chain as "Live EVM (Arc testnet)"
participant Node as "Node test runner (Oracle)"
Note over Forge,Contracts : Unit, integration, E2E, fuzz, invariants
Forge->>Contracts : Deploy proxies, roles, mock SVR
Forge->>Contracts : Register, stake, propose, finalize, slash
Forge->>Handler : Execute randomized transitions
Handler-->>Forge : Assert invariants after each step
Note over TS,Chain : Live chain integration
TS->>Chain : Read identity, reputation, verify signatures
Chain-->>TS : On-chain state and events
Note over Node : Adversarial scoring
Node->>Node : Run scoring pipeline with attacker-crafted inputs
Node-->>Node : Validate caps, decay, community impact
Loading

Diagram sources

Detailed Component Analysis

Foundry Smart Contract Testing Infrastructure

  • Configuration and profiles:
    • Default profile sets source, output, libraries, remappings, optimizer, solc version, EVM version, and fuzz runs (1,000).
    • CI profile increases fuzz runs to 5,000 for stricter coverage.
    • RPC endpoints for Arc testnet/mainnet are defined for script/test usage.
    • Two suites fork Arc testnet. SlashDrillFork.t.sol forks it on every run, so it needs the arc_testnet endpoint to answer. SplitAuthorityFork.t.sol skips its twelve tests unless --fork-url https://rpc.testnet.arc.io is passed.
  • End-to-end tests:
    • Deploy mock ERC20 token and proxy-backed implementations for Identity, Reputation, and Staking.
    • Wire roles between contracts and fund operators.
    • Exercise full lifecycle: register → stake → propose/finalize reputation → slash → claim proceeds → zero reputation.
    • Cover dispute flow, unbonding period enforcement, slash sweeping of queued withdrawals, multiple epochs, and multiple agents.
    • Enforce terminal slash semantics: slashed agents cannot be reactivated or rescored.
  • Fuzz and invariants:
    • A handler contract exposes every externally reachable transition with bounded random inputs.
    • The fuzzer targets the handler and asserts invariants such as custody matching obligations, token conservation, slashed agents fully zeroed, burn address monotonicity, disputed agents staying bonded, and active agents being collateralized. The handler rules on disputes as the admin and also drives expireSlash, so the lapse path is reachable.
    • OracleBondInvariantTest is a second campaign over the operator bond registry. Its five invariants say that signer mappings stay symmetric, a signer is never also an operator, standing comes only through a bonded Active operator, nobody is left signing for nobody, and the active count and custody still add up. Together the staking and operator bond campaigns hold 11 of the 19 invariants; the score store, the fee registry and the identity registry have campaigns of their own.
flowchart TD
Start(["setUp: deploy mocks, proxies, roles"]) --> Reg["Register agent"]
Reg --> Stake["Deposit stake"]
Stake --> Propose["Propose reputation"]
Propose --> Finalize["Finalize after challenge window"]
Finalize --> Slash["Initiate slash"]
Slash --> Dispute{"Disputed?"}
Dispute --> |Yes| Resolve["Resolve dispute"]
Dispute --> |No| Execute["Execute slash"]
Resolve --> Reinstated["Agent reinstated if resolved"]
Execute --> ZeroRep["Reputation zeroed, stake swept"]
ZeroRep --> End(["End"])
Reinstated --> End
Loading

Diagram sources

Section sources

Helper Utilities and Fixtures

  • RegistrationHelper abstracts signing and registration so tests can create agents without repeating signature ceremony code.
  • It computes the registration digest, signs it with the agent key, and submits the registration call on behalf of the operator.
  • StackFixture (test/helpers/StackFixture.sol) deploys the three registries behind proxies wired the way the deploy scripts wire them, a funded operator, and a SigvaraReader pointed at them. SigvaraReader.t.sol, GatedMarketplace.t.sol, InflowGuard.t.sol, OperatorCustody.t.sol and SmartWalletAgent.t.sol use it instead of each carrying its own copy of that setup.
sequenceDiagram
participant Test as "Test Contract"
participant Helper as "RegistrationHelper"
participant Identity as "SigvaraIdentity"
Test->>Helper : registerSigned(identity, operator, agentPk, pubKey)
Helper->>Helper : signRegistration(...)
Helper->>Identity : registerAgent(agentAddress, pubKey, sig)
Identity-->>Helper : didHash
Helper-->>Test : didHash
Loading

Diagram sources

Section sources

TypeScript SDK Testing Strategy with Vitest

  • Vitest configuration enables globals and node environment for tests.
  • The unit suites need no chain: the verifier's and the gate's contract reads are stubbed, and ethers.Contract is replaced by a fake for registerAgent and depositStake. The 8 integration tests are the only ones that need a live deployment.
  • Integration tests:
    • Require environment variables for RPC URL, addresses, and operator private key.
    • Skip automatically when required env vars are missing, preventing CI blockage.
    • Initialize verifier with chain ID and contract addresses.
    • Register an agent once per suite and then exercise identity queries, DID document building, challenge-response verification, and reputation checks.
sequenceDiagram
participant VT as "Vitest"
participant SDK as "@sigvara/protocol-sdk"
participant Provider as "ethers JsonRpcProvider"
participant Chain as "Arc Testnet"
VT->>SDK : new SigvaraVerifier({ rpcUrl, addresses, chainId })
VT->>Provider : connect to RPC_URL
VT->>Chain : registerAgent(operator, agentAddress, pubKey)
VT->>SDK : verifySignature(agentDid, payload, signature, maxAge, audience)
Chain-->>SDK : on-chain verification result
SDK-->>VT : boolean
Loading

Diagram sources

Section sources

Oracle Service Testing Approach

  • Node test runner is configured via package.json scripts (npm test runs node --test).
  • Adversarial tests:
    • Feed attacker-crafted payment sets into the real scoring pipeline without stubs.
    • Validate per-payer caps, wash ring ceilings, sybil bootstrap resistance, tenure decay, abandonment decay, failure reporting bounds, and flagging damage limits.
    • Measure how many wallets and what durations affect scores, ensuring economic assumptions hold.
flowchart TD
Inputs["Attacker-crafted payments"] --> Pipeline["payments.* + computeScore"]
Pipeline --> Metrics["feeScore, successScore, ageScore,<br/>communityScore, propagationScore, total"]
Metrics --> Checks{"Caps and invariants hold?"}
Checks --> |Yes| Pass["Pass"]
Checks --> |No| Fail["Fail regression"]
Loading

Diagram sources

Section sources

Dependency Analysis

Testing dependencies and relationships:

  • Solidity tests depend on Foundry stdlib, OpenZeppelin contracts, and project contracts.
  • E2E tests depend on RegistrationHelper for signing and registration.
  • Invariant tests depend on a handler contract that encapsulates all transitions and accounting.
  • TypeScript SDK tests depend on ethers and tweetnacl for provider interaction and cryptography.
  • Oracle tests depend on internal modules for payments, scoring, and configuration.
graph LR
Forge["Foundry"] --> E2E["E2E.t.sol"]
Forge --> INV["StakingInvariant.t.sol"]
E2E --> HELP["RegistrationHelper.sol"]
INV --> HELP
TS["Vitest"] --> INT["integration.test.ts"]
Node["Node test runner"] --> ADV["adversarial.test.js"]
Loading

Diagram sources

Section sources

Performance Considerations

  • Foundry fuzz profiles:
    • Default fuzz runs set to a moderate level for local iteration.
    • CI fuzz runs increased to strengthen coverage and catch edge cases.
  • Oracle adversarial tests:
    • Use realistic payment volumes and time spans to measure scoring behavior under stress.
    • Ensure caps and decay functions remain efficient and bounded.

Recommendations:

  • Keep fuzz runs proportional to CI budget; use targeted fuzz for hot paths.
  • For Oracle scoring, batch payment processing and avoid unnecessary recomputation in hot loops.
  • Profile SDK integration tests to ensure RPC calls do not dominate runtime; cache static reads where safe.

Troubleshooting Guide

Common issues and resolutions:

  • Foundry setup:
    • Verify remappings and solc version match foundry.toml.
    • Ensure deployments directory permissions allow read-write for scripts/tests.
  • E2E tests:
    • If role grants fail, confirm admin prank context and correct role constants.
    • If slash disputes behave unexpectedly, check challenge windows and warp timestamps.
  • TypeScript SDK integration tests:
    • Missing environment variables cause tests to skip; set SIGVARA_RPC_URL, addresses, and operator key.
    • Network errors: verify RPC_URL and chain ID; adjust timeouts if necessary.
  • Oracle adversarial tests:
    • If scoring caps appear violated, inspect payerScores and externalScore inputs.
    • Ensure half-life and feeUnit configuration matches expected units.

Section sources

Conclusion

Sigvara’s testing framework combines rigorous Foundry-based Solidity tests (unit, integration, E2E, fuzz, invariants), environment-driven TypeScript SDK integration tests, and adversarial Oracle scoring tests. Together, they validate functional correctness, economic security, and resilience against attack vectors. Following the documented organization, naming conventions, and helper patterns will keep suites maintainable and effective.

Appendices

Writing Effective Tests: Patterns and Examples

  • Solidity E2E pattern:
    • Deploy mock token and proxy-backed contracts.
    • Wire roles and fund operators.
    • Exercise lifecycle steps with explicit assertions at each phase.
    • Cover dispute, unbonding, slash sweep, multiple epochs, and multiple agents.
  • Invariant pattern:
    • Build a handler exposing all transitions.
    • Seed initial state deterministically.
    • Assert conservation, collateralization, and terminal-state properties after fuzzing.
  • TypeScript SDK pattern:
    • Gate tests behind environment variables.
    • Initialize verifier and provider once per suite.
    • Test identity, challenge-response, DID document, and reputation thresholds.
  • Oracle adversarial pattern:
    • Construct attacker-crafted payment sets mirroring production pipelines.
    • Validate caps, decay, community flags, and bootstrap resistance.

Section sources

Test Organization and Naming Conventions

  • Solidity:
    • Place E2E tests in test/E2E.t.sol and invariants in test/StakingInvariant.t.sol.
    • Share common signing and registration logic in test/helpers/.
    • Name test functions with descriptive prefixes indicating scenario and outcome.
  • TypeScript SDK:
    • Group related tests in dedicated files under packages/sdk/test/.
    • Use describe blocks for feature areas (identity, challenge-response, DID, reputation).
  • Oracle:
    • Keep adversarial scenarios in oracle/adversarial.test.js with clear comments describing economic assumptions.

Section sources

Best Practices for Maintainable Test Suites

  • Isolate setup and teardown; reuse helpers for repeated operations.
  • Prefer deterministic fixtures and bounded fuzz ranges.
  • Gate external dependencies behind environment variables and skip gracefully.
  • Document economic assumptions and expected bounds in adversarial tests.
  • Keep assertions granular to pinpoint failures quickly.

Home

Overview

Protocol Architecture

Smart Contracts Reference

Oracle Service

Reputation System Design

TypeScript SDK

Integration Guides

Deployment and Operations

Development and Testing

Advanced Topics

Clone this wiki locally