Repository navigation
v1.75.0
·
121 commits
to master
since this release
New Features
d61e7a7- gooddata-eval: extend detail.latency_breakdown to 5 remaining kinds (commit by @Tomkess)79a3d1d- gooddata-eval: register knowledge_question evaluator (commit by @Tomkess)6b9d690- gooddata-eval: add Langfuse OTLP/experiment foundations and LANGFUSE_BASE_URL (commit by @tychtjan)89d8a89- gooddata-eval: read Langfuse traces via v2 observations and move the client to core/langfuse (commit by @tychtjan)b12210b- gooddata-eval: link agentic runs as Langfuse experiments via OTLP root spans (commit by @tychtjan)b47cf1a- gooddata-eval: pass question, conversation and output into experiment spans from every agentic kind (commit by @tychtjan)6488fe0- gooddata-eval: write single-shot results as Langfuse experiments over OTLP (commit by @tychtjan)334faff- gooddata-eval: make the K verdict a gate, pass@K or pass^K (commit by @myhoai)1c1ae20- gooddata-api-client: add the CREATE_METRIC workspace permission (commit by @Vojtasii)fa82a6a- gooddata-eval: record the turns and steps each agentic run took (commit by @myhoai)
Bug Fixes
9aa8d86- gooddata-eval: return a copy of activated_skills from TurnResult.detail() (commit by @Tomkess)7f04d92- address review findings on judge faults, run counts and trace linking (commit by @tychtjan)3f9b3f8- gooddata-eval: persist skill_routing credit across conversation turns (commit by @Tomkess)26d4f52- gooddata-eval: track set_skills as replacing, not accumulating (commit by @Tomkess)2d64613- gooddata-eval: keep only the final set_skills declaration per turn (commit by @Tomkess)3e1d84d- gooddata-eval: count skills replaced within a turn toward coverage (commit by @Tomkess)3bd2a29- gooddata-eval: assert alert group-by attributes (commit by @tychtjan)57aa25f- gooddata-eval: three false-negative sources in eval scoring (commit by @myhoai)f5703ce- gooddata-eval: accept a string-encoded rejectedSpans count in OTLP export responses (commit by @tychtjan)18073f8- gooddata-eval: fall back to the default score retry delay for negative or NaN Retry-After (commit by @tychtjan)a7ca20b- gooddata-eval: restore the dataset_run_items compat shim and harden the Langfuse score path (commit by @tychtjan)76470ba- gooddata-eval: keep gate out of the positional signature and in the merged report (commit by @myhoai)454286c- gooddata-eval: report the gate verdict without redefining pass@K (commit by @myhoai)
Performance Improvements
1f9babd- take Langfuse trace linking off the eval item critical path (commit by @tychtjan)8125426- cut gooddata_sdk import cost by importing only what the SDK uses (commit by @hkad98)
Refactors
0aa2b77- gooddata-eval: derive TurnResult.detail() from the model, not a key list (commit by @Tomkess)55e22cd- gooddata-api-client: split LDM object permissions per family (commit by @jeskepetr)dc79848- gooddata-eval: remove the legacy Langfuse ingestion and dataset-run code paths (commit by @tychtjan)99935f9- gooddata-eval: count turns and steps as int, not float (commit by @myhoai)
Tests
316c88a- gooddata-eval: use the real set_skills argument key in conversation tests (commit by @Tomkess)bbe1187- gooddata-eval: add an in-process fake Langfuse server for wire-level tests (commit by @tychtjan)7d8155e- gooddata-eval: cover the Langfuse v4 request choreography against a fake server (commit by @tychtjan)