Skip to content

Add RAGAS DeepEval and Python OTLP hooks#43

Merged
haasonsaas merged 1 commit into
mainfrom
codex/eval2otel-framework-python-iteration
May 18, 2026
Merged

Add RAGAS DeepEval and Python OTLP hooks#43
haasonsaas merged 1 commit into
mainfrom
codex/eval2otel-framework-python-iteration

Conversation

@haasonsaas
Copy link
Copy Markdown
Contributor

Summary

  • Add RAGAS and DeepEval conversion adapters with provenance, evidence hashes, RAG metrics, failed-metric warnings, and root exports.
  • Upgrade the Python package from contract-only scaffold to an optional OTLP emitter with content events, env-driven sampling/PII redaction, and provider instrumentation handles.
  • Refresh README and contract docs for the expanded framework and Python surfaces.

Test plan

  • npm run lint
  • npm run build
  • npx tsc --noEmit
  • npm test -- --coverage
  • npm audit --omit=dev
  • PYTHONPATH=python python3 -m unittest discover -s python/tests
  • npm pack --dry-run
  • CI=true OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf bash test-harness/run-tests.sh

Closes #4
Refs #2

@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 18, 2026

Code Coverage

Package Line Rate Branch Rate Complexity Health
src 96% 83% 0
src.integrations 95% 75% 0
Summary 96% (1122 / 1171) 81% (900 / 1107) 0

@haasonsaas haasonsaas requested a review from EvalOpsBot May 18, 2026 22:01
@socket-security
Copy link
Copy Markdown

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff Package Supply Chain
Security
Vulnerability Quality Maintenance License
Addedpypi/​anthropic@​0.102.093100100100100
Addedpypi/​openai@​2.37.096100100100100
Addedpypi/​opentelemetry-api@​1.41.198100100100100
Addedpypi/​cohere@​6.1.099100100100100
Addedpypi/​opentelemetry-sdk@​1.41.199100100100100
Addedpypi/​opentelemetry-exporter-otlp@​1.41.1100100100100100

View full report

@haasonsaas haasonsaas merged commit 95b4e2b into main May 18, 2026
5 checks passed
@haasonsaas haasonsaas deleted the codex/eval2otel-framework-python-iteration branch May 18, 2026 22:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: Integrate evaluation frameworks (RAGAS, DeepEval, Promptfoo)

1 participant