conbench-apache-arrow / Conbench performance report
failed
May 10, 2024 in 0s
Found 60 regressions
Thanks for your patience. Conbench analyzed the 7 benchmarking runs that have been run so far on PR commit 223951e.
Benchmarks with performance regressions
There were 13 possible performance regressions, according to the lookback z-score method.
Benchmarks with regressions:
- Pull Request Run on
arm64-m6g-linux-compute
at 2024-05-10 08:36:38Ztpch
(R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-17, scale_factor=10tpch
(R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-14, scale_factor=10tpch
(R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-12, scale_factor=10tpch
(R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-05, scale_factor=10tpch
(R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-18, scale_factor=10tpch
(R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-07, scale_factor=10tpch
(R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-10, scale_factor=10tpch
(R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-08, scale_factor=10tpch
(R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-20, scale_factor=10tpch
(R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-22, scale_factor=10tpch
(R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-09, scale_factor=10tpch
(R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-04, scale_factor=10tpch
(R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-12, scale_factor=1
All benchmark runs analyzed:
- Pull Request Run on
ursa-i9-9960x
at 2024-05-10 08:32:11Z - Pull Request Run on
ec2-t3-xlarge-us-east-2
at 2024-05-10 09:19:36Z - Pull Request Run on
test-mac-arm
at 2024-05-10 09:19:32Z - Pull Request Run on
arm64-t4g-linux-compute
at 2024-05-10 09:09:21Z - Pull Request Run on
ursa-thinkcentre-m75q
at 2024-05-10 09:38:39Z - Pull Request Run on
ec2-m5-4xlarge-us-east-2
at 2024-05-10 08:56:40Z - Pull Request Run on
arm64-m6g-linux-compute
at 2024-05-10 08:36:38Z
Unstable benchmarks with performance regressions
The following benchmark results indicate a possible performance regression, but are known to sometimes produce false positives when applying the lookback z-score analysis.
-
Pull Request Run on
test-mac-arm
at 2024-05-10 09:19:32ZCopyEmptyVector
(C++) with params=<SMALL_VECTOR(std::string)>, source=cpp-micro, suite=arrow-small-vector-benchmarkBM_RleDecoding
(C++) with params=65536/16, source=cpp-micro, suite=parquet-column-io-benchmarkstd_copy_converting
(C++) with source=cpp-micro, suite=arrow-range-benchmarkShortVectorInsertAtEnd
(C++) with params=<STD_VECTOR(int)>, source=cpp-micro, suite=arrow-small-vector-benchmarklazy_copy
(C++) with source=cpp-micro, suite=arrow-range-benchmarkBM_RleDecoding
(C++) with params=32768/16, source=cpp-micro, suite=parquet-column-io-benchmarkBitBlockCounterFourWordsSumWithOffset
(C++) with params=64, source=cpp-micro, suite=arrow-bit-block-counter-benchmarkMoveEmptyVector
(C++) with params=<SMALL_VECTOR(std::string)>, source=cpp-micro, suite=arrow-small-vector-benchmarkBuildChunkedBinaryArray
(C++) with source=cpp-micro, suite=arrow-builder-benchmarkBM_RleDecoding
(C++) with params=4096/16, source=cpp-micro, suite=parquet-column-io-benchmarkGreaterArrayArrayString
(C++) with params=65536/1, source=cpp-micro, suite=arrow-compute-scalar-compare-benchmarkBatchToTensorSimple
(C++) with params=/size:65536/num_columns:3, source=cpp-micro, suite=arrow-tensor-benchmarkReadCachedFile
(C++) with params=num_cols:8/is_partial:0/real_time, source=cpp-micro, suite=arrow-ipc-read-write-benchmarkArrayScalarKernel
(C++) with params=<MultiplyChecked, Int8Type>/size:4194304/inverse_null_proportion:100, source=cpp-micro, suite=arrow-compute-scalar-arithmetic-benchmarkBM_DecodeArrowBooleanRle
(C++) with params=DecodeArrowNonNull/65536, source=cpp-micro, suite=parquet-encoding-benchmarkArrayScalarKernel
(C++) with params=<Add, UInt64Type>/size:4194304/inverse_null_proportion:100, source=cpp-micro, suite=arrow-compute-scalar-arithmetic-benchmarkAllocateTouchDeallocate
(C++) with params=/size:65536/real_time/threads:4, source=cpp-micro, suite=arrow-memory-pool-benchmarkArrayScalarKernel
(C++) with params=<MultiplyChecked, Int8Type>/size:4194304/inverse_null_proportion:0, source=cpp-micro, suite=arrow-compute-scalar-arithmetic-benchmarkGreaterArrayScalarString
(C++) with params=65536/1, source=cpp-micro, suite=arrow-compute-scalar-compare-benchmarkMoveShortVector
(C++) with params=<SMALL_VECTOR(std::string)>, source=cpp-micro, suite=arrow-small-vector-benchmarkReadCachedFile
(C++) with params=num_cols:1/is_partial:0/real_time, source=cpp-micro, suite=arrow-ipc-read-write-benchmarkBM_DecodeArrowBooleanRle
(C++) with params=DecodeArrow/65536, source=cpp-micro, suite=parquet-encoding-benchmarkBM_DecodeArrowBooleanPlain
(C++) with params=DecodeArrowWithNull/num_values:4096/null_in_ten_thousand:1000, source=cpp-micro, suite=parquet-encoding-benchmarkAllocateTouchDeallocate
(C++) with params=/size:1048576/real_time/threads:1, source=cpp-micro, suite=arrow-memory-pool-benchmarkBM_RleDecoding
(C++) with params=1024/16, source=cpp-micro, suite=parquet-column-io-benchmarkHexParsing
(C++) with params=, source=cpp-micro, suite=arrow-value-parsing-benchmarkConstructFromEmptyStdVector
(C++) with params=<SMALL_VECTOR(std::string)>, source=cpp-micro, suite=arrow-small-vector-benchmarklazy_postinc
(C++) with source=cpp-micro, suite=arrow-range-benchmarklazy_copy_converting
(C++) with source=cpp-micro, suite=arrow-range-benchmarkReadCachedFile
(C++) with params=num_cols:1/is_partial:1/real_time, source=cpp-micro, suite=arrow-ipc-read-write-benchmarkTouchArea
(C++) with params=size:65536/real_time/threads:4, source=cpp-micro, suite=arrow-memory-pool-benchmarkArrayScalarKernel
(C++) with params=<Add, UInt64Type>/size:4194304/inverse_null_proportion:0, source=cpp-micro, suite=arrow-compute-scalar-arithmetic-benchmarkCopyEmptyVector
(C++) with params=<STATIC_VECTOR(std::string)>, source=cpp-micro, suite=arrow-small-vector-benchmark
-
Pull Request Run on
arm64-t4g-linux-compute
at 2024-05-10 09:09:21ZValidateSmallAscii
(C++) with source=cpp-micro, suite=arrow-utf8-util-benchmarkBM_ReadInt64Column
(C++) with params=<Repetition::OPTIONAL, Compression::ZSTD>/65536/1024, source=cpp-micro, suite=parquet-column-io-benchmarkHexParsing
(C++) with params=, source=cpp-micro, suite=arrow-value-parsing-benchmarkValidateLargeAscii
(C++) with source=cpp-micro, suite=arrow-utf8-util-benchmarkInlinedTrieLookupFound
(C++) with source=cpp-micro, suite=arrow-trie-benchmark
-
Pull Request Run on
ursa-thinkcentre-m75q
at 2024-05-10 09:38:39ZValidateSmallAscii
(C++) with source=cpp-micro, suite=arrow-utf8-util-benchmarkBM_ReadInt64Column
(C++) with params=<Repetition::REPEATED, Compression::ZSTD>/65536/1024, source=cpp-micro, suite=parquet-column-io-benchmarkAsOfJoinOverhead
(C++) with params=left_freq:400/left_cols:20/left_ids:500/batch_size:4000/num_right_tables:1/right_freq:400/right_cols:20/right_ids:500/real_time, source=cpp-micro, suite=arrow-acero-asof-join-benchmarkBM_ReadInt64Column
(C++) with params=<Repetition::REQUIRED, Compression::ZSTD>/65536/1024, source=cpp-micro, suite=parquet-column-io-benchmarkConstructFromShortStdVector
(C++) with params=<STATIC_VECTOR(std::string)>, source=cpp-micro, suite=arrow-small-vector-benchmarkBM_ReadInt64Column
(C++) with params=<Repetition::OPTIONAL, Compression::ZSTD>/65536/1024, source=cpp-micro, suite=parquet-column-io-benchmarkAsOfJoinOverhead
(C++) with params=left_freq:400/left_cols:20/left_ids:500/batch_size:4000/num_right_tables:10/right_freq:400/right_cols:20/right_ids:500/real_time, source=cpp-micro, suite=arrow-acero-asof-join-benchmarkAsOfJoinOverhead
(C++) with params=left_freq:200/left_cols:20/left_ids:500/batch_size:4000/num_right_tables:1/right_freq:200/right_cols:20/right_ids:500/real_time, source=cpp-micro, suite=arrow-acero-asof-join-benchmarkAsOfJoinOverhead
(C++) with params=left_freq:400/left_cols:100/left_ids:500/batch_size:4000/num_right_tables:1/right_freq:400/right_cols:100/right_ids:500/real_time, source=cpp-micro, suite=arrow-acero-asof-join-benchmark
Details
This report was generated using the lookback z-score method with a z-score threshold of 30.0.
Loading