deep-eval-web

Run deepeval in Node.js via Pyodide. Browser support is planned; Node is the first target.

What this does

• Vendors deepeval v4.0.7 on npm install (GitHub tarball + patches) and applies Pyodide-oriented patches (removes grpcio / OpenTelemetry deps, hardens telemetry imports, skips threading-heavy paths).
• Ships no-op opentelemetry + grpc stub wheels so deepeval imports cleanly in WASM.
• Loads Pyodide in Node, installs patched wheels with micropip, patches HTTP via pyodide-http, and runs a real evaluate() smoke test against an OpenAI-compatible endpoint.

Prerequisites

• Node.js 18+ (uses --experimental-wasm-stack-switching for HTTP in Node < 24)
• Python 3.9+ (host only, for building wheels)
• A local OpenAI-compatible API (default: http://deezr:4000/v1)

Quick start

npm install
npm run build:wheels
OPENAI_BASE_URL=http://deezr:4000/v1 OPENAI_API_KEY=local npm run smoke

Environment variables

Variable	Default	Description
OPENAI_BASE_URL	http://deezr:4000/v1	OpenAI-compatible API base URL
OPENAI_API_KEY	local	API key (most local servers accept any value)
OPENAI_MODEL_NAME	(auto)	Model id; if unset, probes GET /v1/models and uses the first model

Pyodide bootstrap also sets:

• DEEPEVAL_TELEMETRY_OPT_OUT=1
• DEEPEVAL_DISABLE_DOTENV=1
• DEEPEVAL_DISABLE_TIMEOUTS=1 (threading is unavailable in WASM)

Project layout

vendor/deepeval/              # downloaded on npm install (deepeval v4.0.7)
patches/deepeval/             # ordered patches applied before wheel build
packages/pyodide-otel-stubs/  # no-op opentelemetry + grpc for WASM
scripts/build-wheels.sh       # build dist/*.whl
src/node/                     # Node loader + smoke entrypoint
src/pyodide/                  # Python bootstrap + smoke eval
dist/                         # built wheels (gitignored)

Scripts

Script	Description
npm install	Download deepeval source and apply Pyodide patches (postinstall)
npm run build:wheels	Build stub + deepeval wheels into dist/
npm run smoke	End-to-end evaluate() via Pyodide in Node
npm run lock:pyodide	Generate a micropip lock file (experimental; not used by default loader)

Known limitations (Node MVP)

• No OTEL export — tracing/Confident OTEL integrations are stubbed.
• No pytest CLI — use programmatic evaluate() / metrics API.
• No threading — timeouts and rich live progress are disabled or bypassed.
• portalocker — unavailable in WASM; disk cache locking is skipped (warning only).

Browser (later)

The same wheels and Python modules are intended to work in the browser with Vite/static hosting and COOP/COEP headers for streaming HTTP. Node is validated first.