0.23.0

BREAKING CHANGES

• Images are published to a new registry. Update the image repository of ADAS and the Enrichment Runner in the deployment values; the 0.23 Helm chart already points at it (498f65e)
• Deploy together with the matching Enrichment Runner: the runner-only endpoints now require Adas-Contract-Version: 4 (498f65e)
• Grant the writer SELECT on system.parts and system.data_skipping_indices before deploying: existing row-grain enrichments are relaid out to the aligned layout, with no rollback to 0.3 once a table is aligned (498f65e)
• #27 A writer granted per table needs CREATE TABLE, SELECT and INSERT on _dial_internal_failed_ingest, or startup fails at V17 and ingest stops at the first failed blob (#37)
• Writes to an aligned enrichment change: POST /v1/tables/{name}/rows must carry the source row's request_time, and a pipeline onto it must read its source with a row trigger (498f65e)
• #28 Before deploying the matching runner, set enabled: false in ADAS on every pipeline paused in the runner with origin: operator, since enabled is now the only run control (#36)
• Ingest now cuts a payload body truncated at 4 MiB back to before its half inline file. Set SYNC_INLINE_FILES_DROP_TRUNCATED=false to store bodies as sent (#3)
• #26 The pipeline preview response is reshaped: sample_statement becomes statement, now ADAS SQL instead of ClickHouse SQL, and the definition echoes are gone. Update preview clients and handle the statement_not_portable note (#31)
• #40 SAVED_QUERY_WRITE and SAVED_QUERY_PUBLISH now imply SAVED_QUERY_READ, and a personal saved query is visible to its owner only, FULL_ADMIN included. Review roles-mapping entries that grant write without read (#38)
• /v1/evaluators* is removed and answers 404. Dump the evaluator archive tables before upgrading to keep the pre-fold definitions (498f65e)
• The OpenAPI document and Swagger UI need a credential in oidc mode. Set DISABLE_SWAGGER_AUTHORIZATION=true to serve them without one (498f65e)
• Telemetry service.name and the Flight SQL server name are now ai-dial-analytics. Set OTEL_SERVICE_NAME=analytics-data-access-service to keep existing dashboards and alerts matching (498f65e)

Features

• #40 Store dashboards of query widgets under /v1/dashboards, with private and public visibility (#38)
• #61 Name the table holding session rows and bind its fields to the session roles with GET/PUT /v1/settings/sessions-source, under the new SETTINGS_READ and SETTINGS_WRITE privileges (#60, #65)
• #27 Record every blob or day that fails to ingest, retry it on a sweep (SYNC_RETRY_*), and list or requeue it under /v1/data-sync/failures (#37)
• #39 Store chat request bodies as per-turn deltas with SYNC_REQUEST_BODY_MODE=delta (opt-in, default whole) (#4)
• #39 Rebuild delta-stored request bodies on read in the projection-only fields request_body_full and request_body_rebuild, bounded by QUERY_REQUEST_BODY_REBUILD_MAX_ROUNDS and QUERY_REQUEST_BODY_REBUILD_MAX_ANCESTORS (#35, #65)
• Partition row-grain enrichments like their source (layout: aligned), and relay out existing ones automatically after deploy (498f65e)
• #64 Read usage logs from Google Cloud Storage with BLOB_PROVIDER=google-cloud-storage, by service-account key or keyless through Application Default Credentials (#63, #65)
• Run the PostgreSQL catalog on Cloud SQL with IAM database authentication, through connector parameters in POSTGRES_OPS (#63, #65)
• Add a projection_only flag to every entity schema-discovery field, true for the two computed request-body fields (#35, #65)
• Fetch projection-only heavy enrichment columns for the returned page only, and push enrichment-only filters and scalar calls into the enrichment join (498f65e)
• Late-materialize projection-only columns of partition-aligned enrichments, so a row page no longer joins the enrichment over its whole window (#50)
• Bound a window-only join to an aligned enrichment by the time window instead of key sets (498f65e)
• Trim inline files from payload bodies at ingest with SYNC_INLINE_FILES_DROP_TRUNCATED (default true) and SYNC_INLINE_FILES_MAX_SIZE (#3)
• #59 Keep the columns the catalog flags sensitive out of ingest with SYNC_SENSITIVE_COLUMNS_ENABLED=false (opt-in, default true) (#58, #65)
• Store the route that served a request in route_name and route_type on dial_usage_log (#5)
• Set ingest-table retention with SYNC_USAGE_LOG_TTL_DAYS and SYNC_PAYLOAD_TTL_DAYS. Unset leaves the live TTL alone (#32)
• Skip a decompressed log line longer than SYNC_MAX_LINE_BYTES (default 67108864) as malformed (498f65e)
• Declare an aggregate pipeline as one INSERT … SELECT statement in the new sql field (498f65e)
• Return one flat pipeline preview shape with columns, notes and an aggregate total_rows (498f65e)
• #26 Return the pipeline preview statement as portable ADAS SQL, and the same query as structured-query DSL in query (#31)
• #28 Make enabled a pipeline's only run control, so stopping and restarting a pipeline no longer bumps generation (#36)
• Lower the AGGREGATE_PREVIEW_ROWS default from 20 to 5 (498f65e)
• Ship a recommended catalog bundle, docs/recommended-catalog/adas-catalog.json, with client-identity, session and baggage tables and pipelines (498f65e)
• Label Claude Desktop traffic as claudedesktop in usage_client_identity (#34)
• #46 Label DIAL CLI traffic as dialcli in usage_client_identity, with new dial_cli_version and dial_cli_harness_name columns (#45)
• Require a credential for the OpenAPI document and Swagger UI in oidc mode, unless DISABLE_SWAGGER_AUTHORIZATION=true (498f65e)
• Filter the /v1/activities feed by initiatedEmail (498f65e)
• Serve the actuator on its own port when MANAGEMENT_SERVER_PORT is set. Unset keeps it on the API port (#2, #43)
• Report PostgreSQL in /actuator/health/readiness, and serve the liveness and readiness groups outside Kubernetes too (498f65e)

Fixes

• Key enrichment joins by the filter's source-only conjuncts, so a filter on an enrichment column no longer unkeys every join (498f65e)
• Prune unused JSON paths from keyed enrichment joins, and bound each composite join operand by its own conjuncts (498f65e)
• Report an unknown bare select item as an unknown column instead of a self-referencing alias (#42)
• Keep a declared read source on a PATCH without inputs when that source is the target enrichment's parent (498f65e)
• Record table grant and row-policy changes in table revisions, and refuse renaming an enrichment's provenance column with 422 (498f65e)
• Accept a source with granularity: year, now rendered as toYear (498f65e)
• #27 Store an unreadable long, double or bool value as absent instead of 0 or false, and count it on sync.values.coercion.failures (#37)
• Keep a request on its delta chain when a message changes between a one-text-block array and a plain string; a later turn's request_cache_control gains a text_blocks entry (affects pre-release users only) (#62, #65)
• Upgrade vulnerable dependencies and clear Gradle build warnings (498f65e)

Docs

• Ship the upgrade guide as docs/release-0.23-upgrade.md, with the new settings, privileges, migrations and rollback steps (#53, #58, #65)
• Add an architecture overview with a diagram, docs/architecture.md (#44)
• Add docs/gcs-access.md and docs/cloud-sql-access.md for running on Google Cloud (#63, #65)
• Document the full ClickHouse writer grant set in docs/clickhouse-server.md (498f65e)

CI

• Bump the ai-dial-ci workflow group from 4.12.0 to 4.13.2 (#33, #47)
• Bump step-security/harden-runner from 2.22.2 to 2.22.4 and gradle/actions/dependency-submission from 6.4.0 to 6.4.1 (#48, #57, #65)

Other

• Move to open source (498f65e)
• Remove the evaluator archive API and tables (Postgres V48), and rename the OpenAPI component EvaluatorVarDto to TransformVarDto (498f65e)
• Rename the service to ai-dial-analytics in the telemetry service.name default and the Flight SQL server name (498f65e)
• Add google-auth-library-oauth2-http 1.55.0 and the Cloud SQL socket factory 1.30.2, and hold every gRPC module at 1.78.0 for Arrow Flight (#63, #65)
• Add the analytics-release-notes and pr-ready Claude Code skills, and review rules in CLAUDE.md and openspec/config.yaml (#51, #55, #65)

Deployment Changes

This release includes changes that need operator action. Please review the upgrade guide before proceeding.

Upgrade together with Enrichment Runner 0.23.0: the ADAS ↔ Enrichment Runner contract version rises from 3 to 4, and neither build works with the other's previous release. See the runner upgrade guide.

0.23.0-rc.2

Features

• #61 Name the table holding session rows and bind its fields to the session roles with GET/PUT /v1/settings/sessions-source, under the new SETTINGS_READ and SETTINGS_WRITE privileges (#60, #65)
• #39 Rebuild delta-stored request bodies on read in the projection-only fields request_body_full and request_body_rebuild, bounded by QUERY_REQUEST_BODY_REBUILD_MAX_ROUNDS and QUERY_REQUEST_BODY_REBUILD_MAX_ANCESTORS (#35, #65)
• Add a projection_only flag to every entity schema-discovery field, true for the two computed request-body fields (#35, #65)
• #64 Read usage logs from Google Cloud Storage with BLOB_PROVIDER=google-cloud-storage, by service-account key or keyless through Application Default Credentials (#63, #65)
• #59 Keep the columns the catalog flags sensitive out of ingest with SYNC_SENSITIVE_COLUMNS_ENABLED=false (opt-in, default true) (#58, #65)
• Run the PostgreSQL catalog on Cloud SQL with IAM database authentication, through connector parameters in POSTGRES_OPS (#63, #65)

Fixes

• Keep a request on its delta chain when a message changes between a one-text-block array and a plain string; a later turn's request_cache_control gains a text_blocks entry (affects pre-release users only) (#62, #65)

Docs

• Rename the upgrade guide to docs/release-0.23-upgrade.md and add the new settings, privileges, migrations and rollback steps to it (#58, #65)
• Add docs/gcs-access.md and docs/cloud-sql-access.md for running on Google Cloud (#63, #65)

CI

• Bump step-security/harden-runner from 2.22.3 to 2.22.4 (#57, #65)

Other

• Add google-auth-library-oauth2-http 1.55.0 and the Cloud SQL socket factory 1.30.2, and hold every gRPC module at 1.78.0 for Arrow Flight (#63, #65)
• Add the analytics-release-notes and pr-ready Claude Code skills, and review rules in CLAUDE.md and openspec/config.yaml (#51, #55, #65)

Deployment Changes

This release includes changes that need operator action. Please review the upgrade guide before proceeding.

0.23.0-rc.1

Docs

• Rename release-0.4 upgrade guide to 0.23.0 (#53)

0.23.0-rc.0

BREAKING CHANGES

• Deploy together with the matching Enrichment Runner: the runner-only endpoints now require Adas-Contract-Version: 4
• Grant the writer SELECT on system.parts and system.data_skipping_indices before deploying: existing row-grain enrichments are relaid out to the aligned layout, with no rollback to 0.3 once a table is aligned
• Writes to an aligned enrichment change: POST /v1/tables/{name}/rows must carry the source row's request_time, and a pipeline onto it must read its source with a row trigger
• #27 A writer granted per table needs CREATE TABLE, SELECT and INSERT on _dial_internal_failed_ingest, or startup fails at V17 and ingest stops at the first failed blob (#37)
• Ingest now cuts a payload body truncated at 4 MiB back to before its half inline file. Set SYNC_INLINE_FILES_DROP_TRUNCATED=false to store bodies as sent (#3)
• #28 Before deploying the matching runner, set enabled: false in ADAS on every pipeline paused in the runner with origin: operator, since enabled is now the only run control (#36)
• #40 SAVED_QUERY_WRITE and SAVED_QUERY_PUBLISH now imply SAVED_QUERY_READ, and a personal saved query is visible to its owner only, FULL_ADMIN included. Review roles-mapping entries that grant write without read (#38)
• #26 The pipeline preview response is reshaped: sample_statement becomes statement, now ADAS SQL instead of ClickHouse SQL, and the definition echoes are gone. Update preview clients and handle the statement_not_portable note (#31)
• /v1/evaluators* is removed and answers 404. Dump the evaluator archive tables before upgrading to keep the pre-fold definitions
• The OpenAPI document and Swagger UI need a credential in oidc mode. Set DISABLE_SWAGGER_AUTHORIZATION=true to serve them without one
• Telemetry service.name and the Flight SQL server name are now ai-dial-analytics. Set OTEL_SERVICE_NAME=analytics-data-access-service to keep existing dashboards and alerts matching

Features

• #40 Store dashboards of query widgets under /v1/dashboards, with private and public visibility (#38)
• Partition row-grain enrichments like their source (layout: aligned), and relay out existing ones automatically after deploy
• #27 Record every blob or day that fails to ingest, retry it on a sweep (SYNC_RETRY_*), and list or requeue it under /v1/data-sync/failures (#37)
• #39 Store chat request bodies as per-turn deltas with SYNC_REQUEST_BODY_MODE=delta (opt-in, default whole) (#4)
• Declare an aggregate pipeline as one INSERT … SELECT statement in the new sql field
• Return one flat pipeline preview shape with columns, notes and an aggregate total_rows
• #26 Return the pipeline preview statement as portable ADAS SQL, and the same query as structured-query DSL in query (#31)
• Lower the AGGREGATE_PREVIEW_ROWS default from 20 to 5
• #28 Make enabled a pipeline's only run control, so stopping and restarting a pipeline no longer bumps generation (#36)
• Fetch projection-only heavy enrichment columns for the returned page only, and push enrichment-only filters and scalar calls into the enrichment join
• Late-materialize projection-only columns of partition-aligned enrichments, so a row page no longer joins the enrichment over its whole window (#50)
• Bound a window-only join to an aligned enrichment by the time window instead of key sets
• Trim inline files from payload bodies at ingest with SYNC_INLINE_FILES_DROP_TRUNCATED (default true) and SYNC_INLINE_FILES_MAX_SIZE (#3)
• Store the route that served a request in route_name and route_type on dial_usage_log (#5)
• Set ingest-table retention with SYNC_USAGE_LOG_TTL_DAYS and SYNC_PAYLOAD_TTL_DAYS. Unset leaves the live TTL alone (#32)
• Skip a decompressed log line longer than SYNC_MAX_LINE_BYTES (default 67108864) as malformed
• Ship a recommended catalog bundle, docs/recommended-catalog/adas-catalog.json, with client-identity, session and baggage tables and pipelines
• Label Claude Desktop traffic as claudedesktop in usage_client_identity (#34)
• #46 Label DIAL CLI traffic as dialcli in usage_client_identity, with new dial_cli_version and dial_cli_harness_name columns (#45)
• Filter the /v1/activities feed by initiatedEmail
• Require a credential for the OpenAPI document and Swagger UI in oidc mode, unless DISABLE_SWAGGER_AUTHORIZATION=true
• Serve the actuator on its own port when MANAGEMENT_SERVER_PORT is set. Unset keeps it on the API port (#2, #43)
• Report PostgreSQL in /actuator/health/readiness, and serve the liveness and readiness groups outside Kubernetes too

Fixes

• Key enrichment joins by the filter's source-only conjuncts, so a filter on an enrichment column no longer unkeys every join
• Prune unused JSON paths from keyed enrichment joins, and bound each composite join operand by its own conjuncts
• Report an unknown bare select item as an unknown column instead of a self-referencing alias (#42)
• Keep a declared read source on a PATCH without inputs when that source is the target enrichment's parent
• Record table grant and row-policy changes in table revisions, and refuse renaming an enrichment's provenance column with 422
• Accept a source with granularity: year, now rendered as toYear
• #27 Store an unreadable long, double or bool value as absent instead of 0 or false, and count it on sync.values.coercion.failures (#37)
• Upgrade vulnerable dependencies and clear Gradle build warnings

Docs

• Add an architecture overview with a diagram, docs/architecture.md (#44)
• Document the full ClickHouse writer grant set in docs/clickhouse-server.md

Other

• Remove the evaluator archive API and tables (Postgres V48), and rename the OpenAPI component EvaluatorVarDto to TransformVarDto
• Rename the service to ai-dial-analytics in the telemetry service.name default and the Flight SQL server name
• Move to open source

Deployment Changes

This release includes changes that need operator action. Please review the upgrade guide before proceeding.

Upgrade together with Enrichment Runner 0.23.0-rc.0: the ADAS ↔ Enrichment Runner contract version rises from 3 to 4, and neither build works with the other's previous release. See the runner upgrade guide.