v0.5.50

Fix

Model modal on runs launched before a model's card was set. A run started after model cards existed (v0.5.46) but before a model's card was filled in under Connections recorded an empty identity. Its model modal then showed no model card and no benchmark for that model, even after the card was set later.

Such runs now show the model's current card and leaderboard, the same as runs from before model cards existed. A run that recorded a real card still shows the card it ran with. The agent's base model on agent runs follows the same rule.

Upgrade note: no migrations.

v0.5.49

Highlights

Open WebUI v0.12.4.12 (subpath fork). The embedded chat (Playground and agents) now uses v0.12.4.12-subpath, up from v0.12.4.10-subpath.

• New: an optional site-wide script via WEBUI_CUSTOM_JS_URL, plus the fix that lets Open WebUI start with it.
• Embedded and dev modes rebuild their managed Open WebUI environment automatically on first start.
• Docker Compose: the default image is now ghcr.io/sushantgautam/open-webui:v0.12.4.12-subpath (override with OPEN_WEBUI_IMAGE).

Upgrade note: no migrations. On the first start after upgrading, the embedded chat takes a minute or two longer while it installs the new Open WebUI.

v0.5.48

Highlights

Large runs no longer stall (#82). Scenarios waiting in the queue for a free worker slot no longer expire. Hatchet's default 5-minute scheduling timeout used to fail every scenario of a large run that had not started within 5 minutes of submission, so runs with more scenarios than worker slots stopped part-way with no error. The scenario task now waits as long as needed; a resubmitted scenario still never runs twice at once.

Failed spans show why they failed (#80). With SimpleAudit 0.6, Studio stores each span's status message and exception events. For a failed tool call the judge now sees status: ERROR and exception: <type>: <message>, and the run timeline's Error block shows the same.

• At every capture level only the exception type and message are kept; stack traces and other span events need full capture.
• Spans stored before this release keep showing their error as before.

Upgrade note: includes the migration model_registry.0022 (two new columns on stored spans, no data changes). Restart the worker together with the web process.

v0.5.47

Highlights

Model modal on run pages. Click a target, auditor, judge or agent base model on a run page to open a model view:

• Model card: opens the Hugging Face or vendor card.
• Open in Connections: jumps to the model's connection.
• Benchmark: the EuroEval leaderboard shown right in the modal, with this model highlighted; also opens in a new tab.

It fills the screen on phones. Runs created before model cards existed show the model's current card; runs that recorded one keep their own.

Benchmark leaderboard links.

• Admin Settings → Overview → Benchmark leaderboard link: a URL template where {model} is replaced with the model's name. Default: the EuroEval Norwegian generative leaderboard with the model highlighted. Leave it empty to hide the links.
• Leaderboard name per model: the model edit form in Connections has an optional field for when the leaderboard names the model differently (e.g. gpt-4.1-2025-04-14). Otherwise the model card identifier is used.
• Connections: each model's chip links to its model card and its leaderboard.

SimpleAudit engine 0.6.0. Studio now requires simpleaudit>=0.6.0,<0.7.0. The engine brings judge calibration, parallel repetitions, richer tracing evidence for the judge, broader credential redaction and sturdier verdict parsing. See the engine release notes.

Upgrade note: no migrations. Restart the worker together with the web process so it loads the new engine.

v0.5.46

Highlights

Model cards: record what each model is.

• In Connections, each model's edit form has a new Model card row: a type (Hugging Face, Commercial or RAG / system), an identifier such as Qwen/Qwen3-30B-A3B or gpt-4.1, and an optional card URL. For Hugging Face models the card link is derived from the identifier; a pasted huggingface.co/... URL is trimmed to owner/model.
• The model list shows a chip with a link to the card.
• The identity is frozen into each new run's snapshot, and the run page's Frozen Manifest shows it under Target, Auditor and Judge. Remapping a model later never changes what past runs show; runs created before this release show no chip.
• Agents need nothing extra: an agent run shows its base model's identity.

Extensions: companion tools next to Studio (#83).

SIMPLEAUDIT_EXTENSIONS (inline JSON or a path to a JSON file) adds, without forking templates:

• Sidebar menu items, filtered by role on the server, optionally grouped under a section heading.
• Stylesheets and scripts on Studio pages (defer, optional integrity, optional pages globs such as /runs/*).
• window.Studio for scripts (user, role, workspace, CSRF token, version, onPage() helper) and a studio:ready event.
• Embed pages at /x/<slug>/ that show a same-origin app inside the Studio shell.
• GET /ext/authz?role=…, a forward-auth endpoint with the same contract as /chat/authz, so a companion app behind a proxy can reuse Studio's login.

Off by default: with the setting unset nothing changes. The manifest is operator code (its scripts run with full same-origin privileges), so it is read only from configuration. See docs/extensions.md, with an example in docs/examples/extensions.json.

Upgrade note: includes the migration model_registry.0021 (adds model_identity; no data changes). Restart the worker together with the web process.

v0.5.45

Highlights

Experiments are now editable collections of runs.

• Add and remove runs after launch. On an experiment page, + Add runs opens a dialog that searches the workspace's runs (by name, #id, scenario set or target). Remove selected takes runs out. A run page has Add to experiment. Runs and their frozen inputs never change; only which experiments they belong to does.
• A run can belong to several experiments. Adding a run to another experiment keeps it in the first one. Run pages and the runs dashboard show every experiment a run is in.
• Factors follow the runs. The compared inputs ("compares target × max turns") are worked out from the runs currently in the experiment, so they update when runs are added or removed.
• Delete an experiment. Only the grouping is removed. Runs, results and monitors are kept; monitors simply stop being tied to it.
• Permissions: workspace admins and auditors can edit or delete a private experiment. A public experiment needs an admin and an explicit confirmation, because adding a run publishes it (transcripts included). A run is publicly visible when any experiment it belongs to is public.

Timeline: no duplicate flag when there is no captured input, and a retrieval comes first when it starts at the same time as another step.

Upgrade note: includes the migration audits.0010, which copies existing experiment links into the new table. Restart the worker together with the web process.

v0.5.44

Highlights

OTLP traces sent as JSON with base64 ids now link to their audit request.

• The OTLP/JSON spec encodes trace and span ids as hex. Some exporters send protobuf's JSON mapping instead, with base64 ids. Studio stored those ids as they arrived, so the spans never matched the request's traceparent: the connection page's Test OTLP reported "no trace reached Studio" even though spans were arriving, and audits could not pick up the target's spans. The receiver now converts base64 ids to hex; hex ids are unchanged.
• Copy env vars on an OTLP credential now sets OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf (was http/json). It is the OpenTelemetry default, and the Python and JavaScript SDKs export it without a custom exporter. The receiver still accepts both formats, so existing targets need no change.

v0.5.43

Highlights

The trajectory timeline shows a retrieval and the model call it makes in the order they ran.

• A model call nested in a retrieval span (for example rewriting the user's question into a search query) is now shown before its retrieval row, tagged for retrieval, followed by the retrieval with its query and documents, then the answer. In v0.5.42 the two were folded into one row and the retrieval appeared first.
• The 👤 marker (the model call that received the user's message) appears once per turn, on the turn's first model call. A RAG app that sends the question to both the query rewrite and the answer no longer gets two markers per turn.

Rules and judges are unchanged: a model call inside a retrieval span is still never taken as the agent's answer.

v0.5.42

Highlights

A model call inside a retrieval span is part of that retrieval, not the agent's answer. In the OpenTelemetry GenAI conventions a retrieval span is a retrieval pipeline; a model call the pipeline makes (for example rewriting the user's question into a search query) is its child. Studio now reads that structure, for any target.

Rules and judges

• Model calls nested in a retrieval span are auxiliary: answer rules and the semantic agentic judge's conversation turns never take them as the reply. Before, a query rewrite could be read as an answer (for example a rewrite that returned "No").

Trajectory timeline

• The model call is shown inside its retrieval step, tagged + model call: its input and output first, then the retrieval query and documents, so the step reads in the order it ran. Before, the retrieval row came first and the rewrite looked like a reply.
• Finding and judge links to the model call open that retrieval step.

Verified with a traced RAG target (Helsedirektoratet RAG API emitting invoke_workflow › retrieval › chat) across an 8-scenario audit.

v0.5.41

Highlights

A retrieval inside a tool call is now one step, in the timeline and for the judges. With Open WebUI v0.12.4.10, every content-reading tool emits execute_tool with a retrieval span for the same call inside it (the GenAI tracing standard). Studio recognises that pair by span structure alone, so the same applies to any framework that nests retrieval in a tool call.

Trajectory timeline

• One row per lookup: the tool's arguments plus the retrieved documents. A 5-turn agent run went from 22 rows to 16.
• Finding and judge links to the retrieval span open that row.
• A retrieval outside any tool call (for example memory recall) keeps its own row.

Judges

• Main judge: when the nested retrieval has documents, the tool result is replaced by a pointer to them, so the same content is not sent twice. A tool result with no documents (for example "No matches") is kept.
• Semantic agentic judge:
  • The 25-step evidence cap counts tool steps only. Before, model calls, retrievals and workflow spans counted too, so a long run could drop its later tool calls and be marked truncated.
  • Retrieved sources are listed by name, without also listing the same file by ID.