BenchBox 0.4.2

Before you upgrade

• BREAKING: Runs now fail on errors they used to hide. A run fails if any
  query fails validation, including warm-up queries, or if no table loads any
  rows. DataFrame platforms refuse --phases throughput; use --phases power
  or a SQL platform. FlightData at scale factor 0.1 and above fails when a
  month can't be downloaded. To use synthetic rows for missing months, set the
  allow_synthetic_fallback option.
• Some results can't be compared with 0.4.1. TPC-DS runs at scale factors
  other than 1 now use query parameters for that scale, which changes up to 12
  queries. DataFrame runs use the same parameters as SQL runs, and TPC-H
  DataFrame runs now follow --seed. Power-phase times, and the cost per hour
  based on them, now include the time between queries.
• Some answers change because they were wrong before. SQLite TPC-H Q6,
  write primitives on Snowflake and Databricks, TPC-Havoc Q16-Q18 on
  DataFrame platforms, NYC Taxi weekdays on Polars, and many DataFusion
  DataFrame queries now return correct answers.
• Databricks turns off the result cache. BenchBox checks that the
  warehouse confirms the setting and stops if it doesn't. To measure with the
  cache, set the disable_result_cache option to false. When BenchBox
  rebuilds a schema, it now empties tables instead of dropping them, so
  repeated runs no longer use up the Unity Catalog table quota.
• Regenerate some data. TPC-H data generated on macOS by earlier versions
  has wrong supplier and customer addresses. FlightData downloads and
  generates its data again on the next run.
• Stricter submissions. A TPC-Havoc submission must include every variant.
  A result in which every query was skipped no longer passes integrity
  certification. Nine incorrect TPC-H results from early BigQuery, Databricks
  and Snowflake runs are removed from Results Explorer.

Added

• ParadeDB and Citus platforms (experimental). Use --platform paradedb
  or --platform citus. On Citus, the distribution_column option spreads
  tables across workers.
• More data sources. DuckLake can store data on Google Cloud Storage and
  Azure as well as S3. NYC Taxi can load For-Hire Vehicle trips and run three
  queries on them. The synthetic Join Order Benchmark has all 113 queries, up
  from 13, each with pandas and Polars versions.
• More tuning templates. BenchBox ships Snowflake templates for TPC-H and
  TPC-DS. SSB can now be tuned.
• Choose TPC-DS query values. In Python,
  DSQGenBinary.generate_with_parameters now applies the values you pass, and
  DSQGenBinary.generate_parameter_log lists the values used for a seed and
  scale factor. BenchBox includes the qualification values from the TPC-DS
  specification.

Changed

• Vector Search is stable. It remains SQL-only.
• Faster loads and queries. Loading is faster on BigQuery, ClickHouse
  Cloud and PostgreSQL. Snowflake no longer pauses after each query. pandas
  runs TPC-DS Q78 in about 0.3 seconds at scale factor 0.1, down from about
  14 seconds.
• TPC-Havoc runs on more engines. Variants that were skipped now run on
  BigQuery, Databricks, Snowflake, PostgreSQL-family engines, ClickHouse and
  DataFusion. --dry-run saves the same SQL a real run would use.
• Result files always record tuning status. Every result now includes
  platform.tuning.validation_status, including runs without tuning. No
  existing field changes.

Fixed

• DataFrame answers match SQL. All 103 TPC-DS queries return the same
  answers as SQL on Polars, pandas and DataFusion. TPC-H Q1, Q3, Q5, Q10, Q14
  and Q15 are fixed on DataFusion. TPC-H Q17 returns the right answer on empty
  input, and Dask sorts NULLs like the other platforms.
• DataFrame runs no longer crash when two runs share the data cache.
• Cloud platforms. TPC-DS One Big Table on Snowflake and BigQuery no longer
  runs against an empty table. Generated data for eight more benchmarks now
  reaches cloud storage. Data Vault, TSBS DevOps and the read, write,
  transaction and metadata primitives run on Snowflake, BigQuery and
  Databricks, and skip unsupported queries with a reason. Exasol, Presto and
  Trino load data correctly.
• Tuning. The tuning wizard saves the table layouts you choose. DuckDB
  tuned runs no longer report false index mismatches, and TPC-DS tuned runs
  load on engines that enforce primary keys. Runs without tuning reuse their
  database. The console reports only tuning work that succeeded. DuckLake no
  longer deletes its data partway through a run.
• Command line. --show-plans prints query plans. Selectors such as
  --platform clickhouse:local work, and an unknown deployment lists the
  ones that exist.
• Data generation. TPC-DS generation on Windows uses the scale factor you
  ask for. TPC-H Skew at scale factor 10 fits in 16 GB of memory. Read
  Primitives data loads on the first run.
• Other fixes. Seeded TPC-H runs no longer report false failures on Q11,
  Q16, Q18 and Q20. Anonymized results hide more cloud account details.
  SQLite throughput streams no longer mix up rows. TPC-DS One Big Table works
  from an installed package. Results Explorer recovers from startup errors and
  no longer stalls on links to missing results.