Skip to content

v1.1.0

Latest

Choose a tag to compare

@gmr gmr released this 09 Feb 21:20
d473897

What's Changed

Parallel Execution with Connection Pooling

  • Replaced single-connection-per-node model with per-node connection pools, enabling true parallel query execution across cluster nodes
  • Connection pool size is automatically calculated from the concurrency setting and node count
  • Added cleanup for partially-opened connections if a node becomes unreachable during setup

Improved Progress Display

  • Integrated Rich console into logging with RichHandler for better terminal output
  • Added QueryProgress.__rich__() protocol support for dynamic Rich Live rendering
  • Progress table now shows per-query node, offset, progress bar, rows, memory usage, and elapsed time
  • Fixed ETA calculation to properly account for concurrency (remaining * avg_time / concurrency) instead of relying on Rich's TimeRemainingColumn which did not understand parallel execution

Enhanced Progress Metrics

  • Switched from tracking bytes read/written to displaying query memory usage, which is more useful for monitoring resource consumption
  • Progress bar correctly scales for write-heavy queries using max(total, rows, 1)

Better Error Handling

  • Added UnexpectedPacketFromServerError to _cancel_in_flight exception handler for consistency with all other ClickHouse operations
  • Broadened exception handling in connect() with proper cleanup of partial connections
  • Added LOGGER.debug('Fatal error', exc_info=True) in CLI exception handler for --verbose traceback visibility
  • User-friendly validation error messages for settings

Centralized Progress Polling

  • Replaced per-query poll tasks with a single background _poll_all_progress task using persistent poll connections
  • Reduces connection overhead and simplifies lifecycle management

Other Improvements

  • Set client_name to clickhouse-query-runner to identify the app in ClickHouse server logs
  • Use public cursor.set_query_id() API instead of setting _query_id directly
  • Added column mapping comments for PROGRESS_QUERY positional indices
  • Downgraded parser final log from INFO to DEBUG
  • Version bumped from 1.0.0 to 1.1.0

Testing

  • 157 tests passing with 96.90% coverage (threshold: 90%)
  • Added tests for ETA calculation, UnexpectedPacketFromServerError handling, connection pooling, and progress metrics