Repository navigation
Releases: Se7enquick/eliza-dq
Releases · Se7enquick/eliza-dq
Release list
v0.2.1
Fixes
- MySQL/ClickHouse CAST:
CAST(AS VARCHAR)→CAST(AS CHAR)for MySQL,CAST(AS STRING)for BigQuery/Databricks,CAST(AS String)for ClickHouse. Affected checks:not_missing,min_length,max_length. - Regex backslash on MySQL/ClickHouse:
\d,\w,\.patterns were silently stripped by string parser. Now doubled for dialects that consume backslashes. referencecheck in SQL runner: result was silently dropped (missing parser). Now returnsorphan_countcorrectly.agg_rowcrash: defensive handling when aggregation query returns empty or unexpected result.
CI
Integration tests against 4 real engines on every push:
| Engine | Tests | How |
|---|---|---|
| PostgreSQL | 18 | Service container |
| MySQL | 16 | Service container |
| DuckDB | 17 | In-process |
| Polars | 19 | DataFrame/parquet/CSV |
All tests use the same 10-row dataset with exact fail count assertions.
workflow_dispatch dropdown to run one specific dialect when fixing a bug.
v0.2.0
New
- [Feature] Lazy Polars import -- SQL path cold start 20ms instead of 176ms (Polars loads only when DataFrames used)
- [Feature] Default sample queries select only checked columns instead of
SELECT *-- cheaper on BQ/Athena with no configuration - [Feature]
sample_idandsample_columnsYAML options for full control over sample query cost - [Feature] Benchmark script supports 259M rows via
--full
Fixes
- [Fix] SQL errors report
status=error,exit_code=2(was silently passing) - [Fix] Regex backslash preservation in all 8 SQL dialects
- [Fix] Connector dialect correctly routed to SQL generation
- [Fix]
HAVING COUNT(*) > 1for Postgres/Redshift/Snowflake unique check - [Fix]
COUNT(CASE...THEN 1 END)aggregation (26% faster thanSUM)
Verified benchmarks
- Athena: 2x faster than Soda Core (179M-492M rows, parallel samples + 20ms cold start)
- BigQuery: similar speed to Soda (slot scheduling dominates), up to 92% cheaper with
sample_idon wide tables - DataFrames: 16x faster than Pandera on 126M rows (both check everything)
- Streaming: 259M rows from disk in 1.1s, under 1 GB RAM
pip install eliza-dq --upgrade
v0.1.3
- [Feature]
sample_idandsample_columnsfor cheaper DWH sample queries (up to 92% cost reduction on wide BQ tables) - [Feature] Parallel sample collection via ThreadPoolExecutor
- [Fix] SQL errors now report
status=error,exit_code=2 - [Fix] Regex backslash preservation in all SQL dialects
- [Fix] Connector dialect correctly routed to SQL generation
- [Fix]
HAVING COUNT(*) > 1for Postgres/Redshift/Snowflake unique check - [Fix]
COUNT(CASE...THEN 1 END)optimized aggregation (26% faster) - [Docs] Verified benchmarks on 10 fresh tables (BQ + Athena, cache disabled)
pip install eliza-dq --upgrade