Skip to content

v0.5.1

Choose a tag to compare

View all tags
@ashishodu2023 ashishodu2023 released this 10 Oct 03:39
v0.5.1
b4cb923
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Added

  • Prompt version tracking — tracer.set_prompt(name, version), tracer.clear_prompt(), and the tracer.use_prompt(name, version) context manager tag every trace with prompt_name / prompt_version. The hosted API stores them, GET /v1/traces filters by prompt_name / prompt_version, and the dashboard shows the prompt on each trace.
  • pytest plugin for eval gates — @cortexops.eval_gate("golden_v1.yaml", fail_on="task_completion < 0.90") turns a test that returns an agent into a golden-dataset gate. Auto-registered via the pytest11 entry point; adds the cortexops_eval marker and an eval summary to the pytest report. Install with pip install "cortexops[pytest]".
  • Sampling configuration — CORTEXOPS_SAMPLE_RATE environment variable and a sample_errors constructor flag. Each trace records its sample_rate and whether it was sampled.
  • Automatic Pro key email (hosted API) — Stripe and PayPal checkouts email the new key automatically (Resend, falling back to SendGrid), with every outcome in the auth audit log. Admins can rotate and re-send a key with POST /v1/admin/keys/{id}/resend-email.

Fixed

  • Unsampled failed runs (sample_rate < 1.0) raised ValueError: "Trace" object has no field "metadata" instead of the agent's own exception, and were never sent to the hosted API.
  • Traces now include the tracer's environment; previously every hosted trace was recorded as development.
  • fail_on expressions with trailing text (e.g. "task_completion < 0.9 or else") are now rejected instead of being silently truncated.
  • Hosted API: Stripe webhook retries could create duplicate Pro keys.

Fixed after review

  • Hosted API: trace metadata, prompt_name, and prompt_version now go through PII redaction; the prompt_name / prompt_version trace filters match only the top-level prompt fields.
  • Hosted API: admin key resend commits the rotated key before emailing it and rejects non-Pro and expired keys.
  • Hosted API: eval ingest commits a new prompt version when the model or temperature changes.
  • SDK: prompt name/version lengths are validated in set_prompt, and async OpenAI Agents / Agno calls are traced.
  • Hosted API: /health reports version 0.5.1.

Full changelog: v0.5.0...v0.5.1 · PR #26

Assets 2
Loading