0.4.0

CI

• (deps) bump the ai-dial-ci group with 4 updates (#54)
• (deps) bump the ai-dial-ci group with 4 updates (#58)
• add daily SCA scan via epam/ai-dial-ci 4.13.2 (#53)

Other

• bump black from 26.3.1 to 26.10.1 (#51)
• bump filelock from 3.20.3 to 4.2.1 (#57)
• bump langgraph from 1.2.6 to 1.2.14 (#56)
• bump setuptools from 83.0.0 to 84.1.0 (#55)
• bump virtualenv from 21.14.5 to 21.14.13 (#52)
• update aidial-rag-eval to 0.7.0 (#50)
• update dependencies highlighted by trivy (#45)

0.4.0-rc.0

CI

• (deps) bump the ai-dial-ci group with 4 updates (#54)
• (deps) bump the ai-dial-ci group with 4 updates (#58)
• add daily SCA scan via epam/ai-dial-ci 4.13.2 (#53)

Other

• bump black from 26.3.1 to 26.10.1 (#51)
• bump filelock from 3.20.3 to 4.2.1 (#57)
• bump langgraph from 1.2.6 to 1.2.14 (#56)
• bump setuptools from 83.0.0 to 84.1.0 (#55)
• bump virtualenv from 21.14.5 to 21.14.13 (#52)
• update aidial-rag-eval to 0.7.0 (#50)
• update dependencies highlighted by trivy (#45)

0.3.0

Features

• add structured logging support (#32)

Fixes

• avoid downloading cl100k_base.tiktoken on runtime (#37)

CI

• (deps) bump epam/ai-dial-ci from 4.8.0 to 4.11.0 (#40)
• (deps) bump the ai-dial-ci group with 4 updates (#31)

Other

• anyio update to 4.x (#43)
• bump aiohttp from 3.14.1 to 3.14.3 (#34)
• bump nltk from 3.10.0 to 3.10.3 (#39)
• bump setuptools from 82.0.1 to 83.0.0 (#27)
• remove pip from docker image (#41)

Deployment Changes

Structured logs support

This release adds support for structured (JSON) logging, implemented by the aidial-sdk. Logging format is now configurable via new environment variables:

Variable	Default	Description
DIAL_SDK_LOG_FORMAT	text	Logging format. Set to text or json. Implemented by the DIAL SDK.
DIAL_SDK_TEXT_LOG_FORMAT	'%(levelprefix)s | %(asctime)s | %(name)s | %(process)d | %(message)s'	Logging format for text logs. Implemented by the DIAL SDK.
DIAL_SDK_JSON_LOG_FORMAT	'{"level": "%(levelname)s", "time": "%(asctime)s", "logger": "%(name)s", "process": "%(process)d", "message": "%(message)s"}'	Logging format for JSON logs. Implemented by the DIAL SDK.

See the full logging documentation for details on logging configuration and available logging modes.

0.3.0-rc.1

Other

• anyio update to 4.x (#43)

0.3.0-rc.0

Features

• add structured logging support (#32)

Fixes

• avoid downloading cl100k_base.tiktoken on runtime (#37)

CI

• (deps) bump epam/ai-dial-ci from 4.8.0 to 4.11.0 (#40)
• (deps) bump the ai-dial-ci group with 4 updates (#31)

Other

• bump aiohttp from 3.14.1 to 3.14.3 (#34)
• bump nltk from 3.10.0 to 3.10.3 (#39)
• bump setuptools from 82.0.1 to 83.0.0 (#27)
• remove pip from docker image (#41)

Deployment Changes

Structured logs support

This release adds support for structured (JSON) logging, implemented by the aidial-sdk. Logging format is now configurable via new environment variables:

Variable	Default	Description
DIAL_SDK_LOG_FORMAT	text	Logging format. Set to text or json. Implemented by the DIAL SDK.
DIAL_SDK_TEXT_LOG_FORMAT	'%(levelprefix)s | %(asctime)s | %(name)s | %(process)d | %(message)s'	Logging format for text logs. Implemented by the DIAL SDK.
DIAL_SDK_JSON_LOG_FORMAT	'{"level": "%(levelname)s", "time": "%(asctime)s", "logger": "%(name)s", "process": "%(process)d", "message": "%(message)s"}'	Logging format for JSON logs. Implemented by the DIAL SDK.

See the full logging documentation for details on logging configuration and available logging modes.

0.1.1

Features

• add structured logging support (#32) (#33)

0.1.0

Features

• Initial release of the DIAL Admin evaluation-metrics service — a stateless FastAPI endpoint that scores model outputs against ground-truth data, exposing deterministic metrics (exact_match, regex_match, set comparison, deepdiff) alongside LLM-backed metric families from DeepEval, Ragas, and aidial-rag-eval, with the available metrics discoverable at runtime via the API (27af3b5)

Fixes

• Upgrade nltk to 3.10.0 to address a dependency security advisory (Dependabot alert) (#26)

Deployment Changes

New environment variables

Variable	Default	Description
LOG_LEVEL	INFO	Log level. Allowed values: CRITICAL, ERROR, WARNING, INFO, DEBUG.
LOG_LEVEL_OVERRIDE	{}	Per-module log-level overrides as a JSON dict (e.g. {"uvicorn.access":"INFO"}).
DIAL_URL	Required	Base URL of the DIAL API endpoint, used for all LLM and embeddings calls. Startup fails if unset.
DIAL_API_KEY	Empty	API key for the DIAL instance. Required for any LLM- or embeddings-based metric; without it only non-LLM metrics (exact_match, regex_match, deepdiff) work.
EVAL__APP__MAX_CONCURRENT_EVALUATIONS	10	Maximum evaluation requests processed concurrently; requests beyond the limit are queued.
EVAL__APP__MAX_QUEUE_BACKLOG	100	Maximum requests allowed to wait in the queue; requests arriving when full are rejected.
EVAL__METRICS__COMMON__SUPPORTED_MODELS	gemini-2.5-flash-lite, gpt-5-nano-2025-08-07, anthropic.claude-haiku-4-5-20251001-v1:0, gemini-3.1-flash-lite-preview, gpt-5-mini-2025-08-07, gpt-5.4-mini-2026-03-17	Default DIAL LLM deployments offered across all LLM-based metric groups; shown in the DIAL Admin metrics-configuration UI.
EVAL__METRICS__COMMON__DEFAULT_MODEL	gemini-3.1-flash-lite-preview	Default LLM deployment when none is specified in a request; must be in the supported-models list.
EVAL__METRICS__DEEPEVAL__SUPPORTED_MODELS	from common	Overrides the supported-models list for DeepEval metrics only.
EVAL__METRICS__DEEPEVAL__DEFAULT_MODEL	from common	Overrides the default model for DeepEval metrics only.
EVAL__METRICS__DEEPEVAL__VERBOSE_MODE	false	When enabled, DeepEval prints intermediate evaluation steps to stdout; does not affect API responses.
EVAL__METRICS__AIDIAL_RAG_EVAL__SUPPORTED_MODELS	from common	Overrides the supported-models list for aidial-rag-eval metrics only.
EVAL__METRICS__AIDIAL_RAG_EVAL__DEFAULT_MODEL	from common	Overrides the default model for aidial-rag-eval metrics only.
EVAL__METRICS__AIDIAL_RAG_EVAL__MAX_CONCURRENCY	8	Maximum parallel internal LLM calls made by aidial-rag-eval metrics during a single evaluation.
EVAL__METRICS__RAGAS__SUPPORTED_MODELS	from common	Overrides the supported-models list for Ragas metrics only.
EVAL__METRICS__RAGAS__DEFAULT_MODEL	from common	Overrides the default model for Ragas metrics only.
EVAL__METRICS__RAGAS__EMBEDDINGS_MODEL	text-embedding-ada-002	DIAL embeddings deployment used by Ragas metrics that need semantic similarity (e.g. ragas.answer_relevancy).

Note

OpenTelemetry configuration is supported via the standard OTEL_* environment variables. See the OpenTelemetry SDK configuration docs.

0.1.0-rc.0

Features

• Initial release of the DIAL Admin evaluation-metrics service — a stateless FastAPI endpoint that scores model outputs against ground-truth data, exposing deterministic metrics (exact_match, regex_match, set comparison, deepdiff) alongside LLM-backed metric families from DeepEval, Ragas, and aidial-rag-eval, with the available metrics discoverable at runtime via the API (27af3b5)

Fixes

• Upgrade nltk to 3.10.0 to address a dependency security advisory (Dependabot alert) (#26)

Deployment Changes

New environment variables

Variable	Default	Description
LOG_LEVEL	INFO	Log level. Allowed values: CRITICAL, ERROR, WARNING, INFO, DEBUG.
LOG_LEVEL_OVERRIDE	{}	Per-module log-level overrides as a JSON dict (e.g. {"uvicorn.access":"INFO"}).
DIAL_URL	Required	Base URL of the DIAL API endpoint, used for all LLM and embeddings calls. Startup fails if unset.
DIAL_API_KEY	Empty	API key for the DIAL instance. Required for any LLM- or embeddings-based metric; without it only non-LLM metrics (exact_match, regex_match, deepdiff) work.
EVAL__APP__MAX_CONCURRENT_EVALUATIONS	10	Maximum evaluation requests processed concurrently; requests beyond the limit are queued.
EVAL__APP__MAX_QUEUE_BACKLOG	100	Maximum requests allowed to wait in the queue; requests arriving when full are rejected.
EVAL__METRICS__COMMON__SUPPORTED_MODELS	gemini-2.5-flash-lite, gpt-5-nano-2025-08-07, anthropic.claude-haiku-4-5-20251001-v1:0, gemini-3.1-flash-lite-preview, gpt-5-mini-2025-08-07, gpt-5.4-mini-2026-03-17	Default DIAL LLM deployments offered across all LLM-based metric groups; shown in the DIAL Admin metrics-configuration UI.
EVAL__METRICS__COMMON__DEFAULT_MODEL	gemini-3.1-flash-lite-preview	Default LLM deployment when none is specified in a request; must be in the supported-models list.
EVAL__METRICS__DEEPEVAL__SUPPORTED_MODELS	from common	Overrides the supported-models list for DeepEval metrics only.
EVAL__METRICS__DEEPEVAL__DEFAULT_MODEL	from common	Overrides the default model for DeepEval metrics only.
EVAL__METRICS__DEEPEVAL__VERBOSE_MODE	false	When enabled, DeepEval prints intermediate evaluation steps to stdout; does not affect API responses.
EVAL__METRICS__AIDIAL_RAG_EVAL__SUPPORTED_MODELS	from common	Overrides the supported-models list for aidial-rag-eval metrics only.
EVAL__METRICS__AIDIAL_RAG_EVAL__DEFAULT_MODEL	from common	Overrides the default model for aidial-rag-eval metrics only.
EVAL__METRICS__AIDIAL_RAG_EVAL__MAX_CONCURRENCY	8	Maximum parallel internal LLM calls made by aidial-rag-eval metrics during a single evaluation.
EVAL__METRICS__RAGAS__SUPPORTED_MODELS	from common	Overrides the supported-models list for Ragas metrics only.
EVAL__METRICS__RAGAS__DEFAULT_MODEL	from common	Overrides the default model for Ragas metrics only.
EVAL__METRICS__RAGAS__EMBEDDINGS_MODEL	text-embedding-ada-002	DIAL embeddings deployment used by Ragas metrics that need semantic similarity (e.g. ragas.answer_relevancy).

Note

OpenTelemetry configuration is supported via the standard OTEL_* environment variables. See the OpenTelemetry SDK configuration docs.