[OAI] Support braintrust >=0.13 wrapping (fix Python CI)

[Kenny Wong (wong-codaio)]

Summary

[OAI] Support braintrust >=0.13 wrapping (fix Python CI)

• braintrust 0.13 changed wrap_openai to patch methods in place (wrapt) instead of returning a NamedWrapper, and dropped the *Wrapper classes — breaking is_wrapped detection and test_oai.py's imports (collection abort → red Python CI on every PR).
• fix-forward (no pin): detect wrapping version-agnostically — NamedWrapper (<0.13) or wrapt wrapper on create() (>=0.13).
• behavior change: >=0.13 only instruments the v1 SDK, so v0 clients are no longer traced. tests updated.

PTAL:
FYI:

Test plan

• pytest py/autoevals/ → 68 passed on braintrust 0.23 (was: collection ImportError)
• manual sanity check below — verifies detection + a real classifier completing through the wrapped client (so the injected braintrust span metadata doesn't break the call):

braintrust 0.23.0
OK: wrapping detection works (structural)
OK: live classifier through wrapped client -> score=1

check_braintrust_wrapping.py (run it yourself)

"""Sanity check: autoevals still works with the installed braintrust.

braintrust periodically changes how wrap_openai instruments OpenAI clients
(e.g. 0.13 moved from a NamedWrapper proxy to in-place wrapt patching). If
detection regresses, scorer spans silently lose their `purpose` -- and the
injected span metadata could even break the wrapped call. Run this after a
braintrust upgrade to confirm nothing regressed.

  # structural checks only (no network/key):
  python check_braintrust_wrapping.py

  # also run a real classifier through the wrapped client:
  BRAINTRUST_API_KEY=... python check_braintrust_wrapping.py
"""

import importlib.metadata as md
import os

import openai
from braintrust.oai import wrap_openai

from autoevals import Factuality, init
from autoevals.oai import LLMClient, get_openai_wrappers, openai_client_is_wrapped, prepare_openai

print("braintrust", md.version("braintrust"))
NamedWrapper, _ = get_openai_wrappers()

# 1. helper detects a wrapped client and does not false-positive on a plain one.
wrapped = wrap_openai(openai.OpenAI(api_key="x"))
plain = openai.OpenAI(api_key="x")
assert openai_client_is_wrapped(wrapped, NamedWrapper), "wrapped client not detected"
assert not openai_client_is_wrapped(plain, NamedWrapper), "plain client falsely detected"

# 2. init() -> prepare_openai() wraps the default client and reports is_wrapped.
init(openai.OpenAI(api_key="x"))
assert prepare_openai().is_wrapped, "prepare_openai() did not wrap"

# 3. a client with custom methods opts out of wrapping.
init(None)
custom = openai.OpenAI(api_key="x")
assert not LLMClient(openai=custom, complete=custom.chat.completions.create).is_wrapped
print("OK: wrapping detection works (structural)")

# 4. live: run a real classifier through the wrapped client. The wrapped call
#    must accept the injected braintrust span metadata and return a score.
if os.environ.get("OPENAI_API_KEY") or os.environ.get("BRAINTRUST_API_KEY"):
    init(None)  # default client -> braintrust proxy + BRAINTRUST_API_KEY
    assert prepare_openai().is_wrapped
    r = Factuality(model="gpt-4o-mini").eval(
        output="Paris", expected="Paris", input="What is the capital of France?"
    )
    assert r.error is None, r.error
    assert r.score == 1, r.score
    print(f"OK: live classifier through wrapped client -> score={r.score}")
else:
    print("SKIP live classifier (set BRAINTRUST_API_KEY or OPENAI_API_KEY to run it)")

Other Notes

• supersedes [Deps] Pin braintrust <0.13 to unbreak Python CI #192 (pin); surfaced while diagnosing [OAI] Allow forcing Responses API for non-gpt-5 model names #190.