Skip to content

ContextTrace v0.9.0

Choose a tag to compare

View all tags
@samarth1412 samarth1412 released this 05 Jun 16:46
v0.9.0
82066bb

ContextTrace v0.9.0

ContextTrace v0.9.0 adds local-first claim grounding upgrades for RAG and agent systems: span-scoped verification, optional local NLI, calibration tooling, and source trust/freshness scoring.

Highlights

  • Separated grounding from truth with support_status, truth_status, and source_status.
  • Span-scoped judge verification so local LLM judges see selected evidence spans with offsets, not broad retrieved chunks or fluent answer prose.
  • Optional local NLI mode with --mode nli, contexttrace[nli], and contexttrace[nli-onnx].
  • Local NLI calibration with contexttrace nli-calibrate for exact match, entailment precision, contradiction recall, dangerous false-green rate, citation match, and latency.
  • Source trust and freshness scoring from local trace metadata: source_authority, source_timestamp, source_version, canonical, and canonical_source.
  • Source-aware statuses including grounded_but_stale, grounded_but_conflicted, grounded_by_low_authority_source, and supported_by_canonical_source.
  • Verification reports now include Source Trust & Freshness details.

Local NLI Calibration Note

The bundled calibration command was tested locally with MoritzLaurer/DeBERTa-v3-base-mnli-fever-anli downloaded to a local model directory.

Observed calibration on the bundled all case set:

  • exact match rate: 0.667
  • entailment precision: 1.000
  • entailment recall: 0.571
  • contradiction precision: 0.909
  • contradiction recall: 1.000
  • dangerous miss rate: 0.000
  • unsupported marked supported: 0
  • p95 case latency on CPU: 1644.758 ms

Interpretation: this model was strict and avoided false green supported labels, but it did not pass ContextTrace's default exact-match gate. Treat --mode nli as an optional local verifier that should be calibrated against your data before CI gating.

Example

pip install "contexttrace[nli]"
set CONTEXTTRACE_NLI_MODEL_PATH=C:/models/contexttrace-nli

contexttrace verify trace.json --mode nli --report
contexttrace nli-calibrate --case-set all --report

Validation

Validated locally before release with:

pytest packages/contexttrace/tests -q
pytest apps/api/tests -q
python -m contexttrace.cli verify-benchmark --mode semantic --case-set all --json
python -m contexttrace.cli verify-benchmark --mode local_ml --case-set all --json
powershell -ExecutionPolicy Bypass -File scripts/release_check.ps1

ContextTrace remains a local diagnostic and regression tool. Grounded means supported by selected evidence; it does not mean independently true, current, or authoritative.

Assets 4
Loading