API Evidentia Eval

evidentia-eval API

Auto-generated page. This public-surface index is generated from the package source by scripts/wiki/sync_api_docs.py. Do not edit it by hand; change the package and re-run the generator (uv run python scripts/wiki/sync_api_docs.py).

DFAH (Decision-Faithfulness Assessment Harness) determinism + faithfulness eval harness for Evidentia — dev-time AI-output quality gates

Install: pip install evidentia-eval. Import root: evidentia_eval.

Public API

The names exported by evidentia_eval.__all__ (re-exported at the package root):

Symbol	Summary
CLAIM_EXTRACTION_PROMPT	—
DEFAULT_FAITHFULNESS_THRESHOLD	—
DEFAULT_SEMANTIC_MODEL	—
DEFAULT_SEMANTIC_THRESHOLD	—
DFAHarness	Run determinism + replay checks against a generator function.
DeterminismResult	One prompt's determinism outcome over N samples.
EvalResult	Top-level harness output covering all prompts in one run.
EvalSample	One prompt's inputs (immutable; audit-trail-stable).
FaithfulnessResult	One claim's faithfulness outcome.
PromptFaithfulnessResult	Per-prompt faithfulness aggregation (v0.8.4 P1 wiring).
ReplayResult	One prompt's replay-equivalence outcome.
SemanticFaithfulnessNotAvailableError	Raised when sentence-transformers is not installed.
determinism_score	Compute :class:DeterminismResult from raw sample outputs.
extract_claims	Extract atomic claims from generated text via LLM.
faithfulness_score	Compute :class:FaithfulnessResult for one claim.
faithfulness_score_semantic	Compute :class:FaithfulnessResult via sentence-embeddings.
hash_output	SHA-256 hex digest of the normalized output.
normalize_for_determinism	Apply the canonical normalization for determinism comparison.
replay_equivalent	Build a :class:ReplayResult from two raw outputs.
sign_eval_result	Write an EvalResult to disk + produce its Sigstore bundle.
verify_eval_result	Verify a signed EvalResult JSON against its Sigstore bundle.

Submodules

Public submodules + subpackages of evidentia_eval (import directly, e.g. from evidentia_eval.claim_extraction import ...):

• evidentia_eval.claim_extraction
• evidentia_eval.faithfulness
• evidentia_eval.faithfulness_semantic
• evidentia_eval.harness
• evidentia_eval.metrics
• evidentia_eval.seeds
• evidentia_eval.signing

Full signatures

This page is a public-surface index. For full, always-current type signatures and rendered docstrings, see the live API reference (built from the source with mkdocstrings) or read the source.