You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
New server type duckdb to test the tables inside a DuckDB database file (the file is opened read-only)
--dry-run flag for datacontract dbt sync that reports the same plan as a real sync, but writes nothing to disk (#1513@q-maze)
Test results report how many rows a check found bad out of the rows it read, plus the data quality dimension and the quality rule a check comes from (#1526)
postgresql is accepted as the ODCS synonym of the postgres server type
datacontract import pydantic-model reads the contract description from the module docstring (#1507@OGsiji)
Changed
The engine field of test results is now always one of datacontract-cli, dbt or jsonschema; the values datacontract, ibis and dbt-sync no longer occur (#1505)
datacontract test and datacontract ci report a failed publish of the test results on the console and exit with code 1, instead of silently succeeding
datacontract api:
only publishes test results to the Entropy Data platform or the host configured via ENTROPY_DATA_HOST; other publish_url targets are refused (#1541)
no longer sends permissive Access-Control-Allow-Origin: * headers; it serves no CORS headers at all, since the only browser client is the same-origin Swagger UI
no longer reloads on file changes by default; pass --reload to enable it (development only)
A JSON Schema check that could not be run is reported as skipped, not info
Security
datacontract export html and datacontract catalog now HTML-escape data contract field values, closing a stored cross-site scripting hole
The Entropy Data API key is only sent to the Entropy Data host, no longer to any host a data contract URL or --publish URL points at; set ENTROPY_DATA_HOST for a self-hosted deployment
A quality.type: sql rule must be a read-only query; DDL, DML, COPY, ATTACH and the like are reported as a failed check instead of being executed, for every data source
datacontract api:
refuses servers[].type: local, so a posted data contract cannot read the files of the server running it; set DATACONTRACT_CLI_API_ALLOW_LOCAL_FILES=true to allow it
confines the DuckDB connection of a file-based server to the data locations the posted data contract declares
refuses to send an environment-held data source credential to a host named by the posted contract, preventing credential exfiltration through a crafted servers section
compares the x-api-key header in constant time to avoid a timing side channel
reports the URL of a failed authoritativeDefinitions lookup instead of echoing the server-side error reason, which is now logged instead
Fixed
A quality.type: sql rule is read in the SQL dialect of its server type, so dialect-specific syntax (BigQuery backticks, Snowflake SAMPLE, SQL Server TOP) is no longer mistaken for an invalid query
Configuration options that override a server's location (DATACONTRACT_BIGQUERY_PROJECT, DATACONTRACT_POSTGRES_SCHEMA, and the like) now apply to the whole test run, not just to the connection
Test results published to Entropy Data no longer drop the quality rule id, category, rule definition and failed samples
datacontract test and datacontract export sodacl freshness and retention checks now honor the schema object's and property's physicalName (#1488@erikgrip2)
datacontract test --publish no longer fails for runs with skipped checks (skipped checks are omitted from the published test results)
BigQuery: DATACONTRACT_BIGQUERY_BILLING_PROJECT no longer overrides the server's project, so tables are still read from the data project (#1358)
datacontract export pydantic-model exports the ODCS timestamp and time logical types as datetime.datetime and datetime.time instead of guessing from the physical type, which turned a TIMESTAMPTZ column into a str (#1507@OGsiji)
datacontract export pydantic-model exports the ODCS date logical type as datetime.date instead of datetime.datetime (#1507@OGsiji)
datacontract import pydantic-model maps datetime.datetime to the timestamp logical type and datetime.time to time (#1507@OGsiji)
datacontract export dcs exports a pii custom property as a boolean instead of the string 'True'