Skip to content

Conversation

zhicwu
Copy link
Contributor

@zhicwu zhicwu commented Jan 10, 2023

  • introduce ClickHousePassThruStream and make ClickHouseFile a subclass
  • enable gRPC request chunking
  • enable gRPC request compression - only lz4 works
  • bump 3rd party libs to unblock another PR

@zhicwu zhicwu marked this pull request as draft January 10, 2023 16:08
@zhicwu zhicwu linked an issue Jan 10, 2023 that may be closed by this pull request
@zhicwu zhicwu changed the title Pass-thru stream and gRPC request chunking and compression Extract clickhouse-data from clickhouse-client Jan 16, 2023
@zhicwu zhicwu changed the title Extract clickhouse-data from clickhouse-client introduce ClickHousePassThruStream and enable broken gRPC request chunking and compression Jan 18, 2023
@zhicwu zhicwu marked this pull request as ready for review January 18, 2023 07:03
@zhicwu
Copy link
Contributor Author

zhicwu commented Jan 18, 2023

Merge with errors - will raise another PR to get them fixed.

@zhicwu zhicwu merged commit 6c9880a into ClickHouse:master Jan 18, 2023
@zhicwu
Copy link
Contributor Author

zhicwu commented Jan 19, 2023

0.4.0 (zstd)

Benchmark                            (client)  (connection)  (statement)   (type)   Mode  Cnt    Score   Error  Units
Query.selectArrayOfUInt16  ClickhouseHttpJdbc         reuse     prepared  default  thrpt   20    3.163 ± 0.240  ops/s
Query.selectArrayOfUInt16  ClickHouseGrpcJdbc         reuse     prepared  default  thrpt   20    3.177 ± 0.234  ops/s
Query.selectDateTime32     ClickhouseHttpJdbc         reuse     prepared  default  thrpt   20   22.116 ± 0.807  ops/s
Query.selectDateTime32     ClickHouseGrpcJdbc         reuse     prepared  default  thrpt   20   21.014 ± 0.747  ops/s
Query.selectDateTime64     ClickhouseHttpJdbc         reuse     prepared  default  thrpt   20   21.182 ± 0.841  ops/s
Query.selectDateTime64     ClickHouseGrpcJdbc         reuse     prepared  default  thrpt   20   20.220 ± 0.915  ops/s
Query.selectDecimal64      ClickhouseHttpJdbc         reuse     prepared  default  thrpt   20   39.425 ± 1.642  ops/s
Query.selectDecimal64      ClickHouseGrpcJdbc         reuse     prepared  default  thrpt   20   37.520 ± 1.678  ops/s
Query.selectInt32          ClickhouseHttpJdbc         reuse     prepared  default  thrpt   20   74.724 ± 3.755  ops/s
Query.selectInt32          ClickHouseGrpcJdbc         reuse     prepared  default  thrpt   20   68.913 ± 3.694  ops/s
Query.selectInt8           ClickhouseHttpJdbc         reuse     prepared  default  thrpt   20  111.786 ± 9.284  ops/s
Query.selectInt8           ClickHouseGrpcJdbc         reuse     prepared  default  thrpt   20  102.959 ± 6.767  ops/s
Query.selectMapOfInt32     ClickhouseHttpJdbc         reuse     prepared  default  thrpt   20    2.009 ± 0.114  ops/s
Query.selectMapOfInt32     ClickHouseGrpcJdbc         reuse     prepared  default  thrpt   20    1.909 ± 0.085  ops/s
Query.selectString         ClickhouseHttpJdbc         reuse     prepared  default  thrpt   20   32.933 ± 1.898  ops/s
Query.selectString         ClickHouseGrpcJdbc         reuse     prepared  default  thrpt   20   30.254 ± 1.443  ops/s
Query.selectTupleOfInt16   ClickhouseHttpJdbc         reuse     prepared  default  thrpt   20    3.132 ± 0.219  ops/s
Query.selectTupleOfInt16   ClickHouseGrpcJdbc         reuse     prepared  default  thrpt   20    2.995 ± 0.213  ops/s
Query.selectUInt32         ClickhouseHttpJdbc         reuse     prepared  default  thrpt   20   72.778 ± 2.955  ops/s
Query.selectUInt32         ClickHouseGrpcJdbc         reuse     prepared  default  thrpt   20   68.278 ± 3.217  ops/s
Query.selectUInt64         ClickhouseHttpJdbc         reuse     prepared  default  thrpt   20   45.184 ± 1.581  ops/s
Query.selectUInt64         ClickHouseGrpcJdbc         reuse     prepared  default  thrpt   20   42.694 ± 1.434  ops/s
Query.selectUInt8          ClickhouseHttpJdbc         reuse     prepared  default  thrpt   20  114.584 ± 5.771  ops/s
Query.selectUInt8          ClickHouseGrpcJdbc         reuse     prepared  default  thrpt   20   97.367 ± 5.193  ops/s
Query.selectUuid           ClickhouseHttpJdbc         reuse     prepared  default  thrpt   20   37.054 ± 1.498  ops/s
Query.selectUuid           ClickHouseGrpcJdbc         reuse     prepared  default  thrpt   20   35.317 ± 1.298  ops/s

0.4.0 (lz4)

Benchmark                            (client)  (connection)  (statement)   (type)   Mode  Cnt    Score   Error  Units
Query.selectArrayOfUInt16  ClickhouseHttpJdbc         reuse     prepared  default  thrpt   20    3.424 ± 0.245  ops/s
Query.selectDateTime32     ClickhouseHttpJdbc         reuse     prepared  default  thrpt   20   21.675 ± 0.935  ops/s
Query.selectDateTime64     ClickhouseHttpJdbc         reuse     prepared  default  thrpt   20   21.150 ± 1.191  ops/s
Query.selectDecimal64      ClickhouseHttpJdbc         reuse     prepared  default  thrpt   20   53.933 ± 3.034  ops/s
Query.selectInt32          ClickhouseHttpJdbc         reuse     prepared  default  thrpt   20   84.223 ± 4.595  ops/s
Query.selectInt8           ClickhouseHttpJdbc         reuse     prepared  default  thrpt   20  107.816 ± 5.543  ops/s
Query.selectMapOfInt32     ClickhouseHttpJdbc         reuse     prepared  default  thrpt   20    1.790 ± 0.090  ops/s
Query.selectString         ClickhouseHttpJdbc         reuse     prepared  default  thrpt   20   33.445 ± 1.625  ops/s
Query.selectTupleOfInt16   ClickhouseHttpJdbc         reuse     prepared  default  thrpt   20    3.216 ± 0.189  ops/s
Query.selectUInt32         ClickhouseHttpJdbc         reuse     prepared  default  thrpt   20   88.385 ± 4.619  ops/s
Query.selectUInt64         ClickhouseHttpJdbc         reuse     prepared  default  thrpt   20   64.603 ± 2.510  ops/s
Query.selectUInt8          ClickhouseHttpJdbc         reuse     prepared  default  thrpt   20  106.578 ± 4.594  ops/s
Query.selectUuid           ClickhouseHttpJdbc         reuse     prepared  default  thrpt   20   42.622 ± 1.667  ops/s

Note: gRPC is excluded from the test because it's too slow to run.

0.3.2-patch11 (lz4)

Benchmark                            (client)  (connection)  (statement)   (type)   Mode  Cnt    Score   Error  Units
Query.selectArrayOfUInt16  ClickhouseHttpJdbc         reuse     prepared  default  thrpt   20    1.502 ± 0.073  ops/s
Query.selectDateTime32     ClickhouseHttpJdbc         reuse     prepared  default  thrpt   20   21.463 ± 1.020  ops/s
Query.selectDateTime64     ClickhouseHttpJdbc         reuse     prepared  default  thrpt   20   21.468 ± 0.963  ops/s
Query.selectDecimal64      ClickhouseHttpJdbc         reuse     prepared  default  thrpt   20   50.547 ± 2.513  ops/s
Query.selectInt32          ClickhouseHttpJdbc         reuse     prepared  default  thrpt   20   79.476 ± 3.068  ops/s
Query.selectInt8           ClickhouseHttpJdbc         reuse     prepared  default  thrpt   20   95.295 ± 3.891  ops/s
Query.selectMapOfInt32     ClickhouseHttpJdbc         reuse     prepared  default  thrpt   20    1.591 ± 0.072  ops/s
Query.selectString         ClickhouseHttpJdbc         reuse     prepared  default  thrpt   20   33.653 ± 1.018  ops/s
Query.selectTupleOfInt16   ClickhouseHttpJdbc         reuse     prepared  default  thrpt   20    3.293 ± 0.329  ops/s
Query.selectUInt64         ClickhouseHttpJdbc         reuse     prepared  default  thrpt   20   62.790 ± 3.058  ops/s
Query.selectUInt8          ClickhouseHttpJdbc         reuse     prepared  default  thrpt   20  104.199 ± 4.738  ops/s
Query.selectUuid           ClickhouseHttpJdbc         reuse     prepared  default  thrpt   20   40.743 ± 1.850  ops/s

Note: gRPC is excluded from the test because it's too slow to run.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

ZSTD compression
1 participant