Use double locking
conbench-apache-arrow / Conbench performance report
failed
May 10, 2024 in 0s
Found 62 regressions
Thanks for your patience. Conbench analyzed the 7 benchmarking runs that have been run so far on PR commit ade727e.
Benchmarks with performance regressions
There were 13 possible performance regressions, according to the lookback z-score method.
Benchmarks with regressions:
- Pull Request Run on
arm64-m6g-linux-computeat 2024-05-10 16:13:53Ztpch(R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-17, scale_factor=10tpch(R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-14, scale_factor=10tpch(R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-12, scale_factor=10tpch(R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-05, scale_factor=10tpch(R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-18, scale_factor=10tpch(R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-07, scale_factor=10tpch(R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-10, scale_factor=10tpch(R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-08, scale_factor=10tpch(R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-20, scale_factor=10tpch(R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-22, scale_factor=10tpch(R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-09, scale_factor=10tpch(R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-04, scale_factor=10tpch(R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-12, scale_factor=1
All benchmark runs analyzed:
- Pull Request Run on
ursa-thinkcentre-m75qat 2024-05-10 17:58:53Z - Pull Request Run on
ec2-t3-xlarge-us-east-2at 2024-05-10 17:32:04Z - Pull Request Run on
test-mac-armat 2024-05-10 17:33:16Z - Pull Request Run on
ursa-i9-9960xat 2024-05-10 16:11:41Z - Pull Request Run on
arm64-m6g-linux-computeat 2024-05-10 16:13:53Z - Pull Request Run on
arm64-t4g-linux-computeat 2024-05-10 18:30:34Z - Pull Request Run on
ec2-m5-4xlarge-us-east-2at 2024-05-10 17:17:25Z
Unstable benchmarks with performance regressions
The following benchmark results indicate a possible performance regression, but are known to sometimes produce false positives when applying the lookback z-score analysis.
-
Pull Request Run on
ursa-thinkcentre-m75qat 2024-05-10 17:58:53ZValidateSmallAscii(C++) with source=cpp-micro, suite=arrow-utf8-util-benchmarkAsOfJoinOverhead(C++) with params=left_freq:400/left_cols:20/left_ids:100/batch_size:4000/num_right_tables:1/right_freq:400/right_cols:20/right_ids:100/real_time, source=cpp-micro, suite=arrow-acero-asof-join-benchmarkBM_ReadInt64Column(C++) with params=<Repetition::REPEATED, Compression::ZSTD>/65536/1024, source=cpp-micro, suite=parquet-column-io-benchmarkAsOfJoinOverhead(C++) with params=left_freq:400/left_cols:10/left_ids:500/batch_size:4000/num_right_tables:1/right_freq:400/right_cols:10/right_ids:500/real_time, source=cpp-micro, suite=arrow-acero-asof-join-benchmarkAsOfJoinOverhead(C++) with params=left_freq:400/left_cols:20/left_ids:500/batch_size:4000/num_right_tables:1/right_freq:400/right_cols:20/right_ids:500/real_time, source=cpp-micro, suite=arrow-acero-asof-join-benchmarkBM_DeltaBitPackingEncode_Int64_Wide(C++) with params=32768, source=cpp-micro, suite=parquet-encoding-benchmarkCopyEmptyVector(C++) with params=<STD_VECTOR(std::shared_ptr)>, source=cpp-micro, suite=arrow-small-vector-benchmarkBM_ReadInt64Column(C++) with params=<Repetition::REQUIRED, Compression::ZSTD>/65536/1024, source=cpp-micro, suite=parquet-column-io-benchmarkAsOfJoinOverhead(C++) with params=left_freq:1000/left_cols:20/left_ids:500/batch_size:4000/num_right_tables:1/right_freq:1000/right_cols:20/right_ids:500/real_time, source=cpp-micro, suite=arrow-acero-asof-join-benchmarkAllocateDeallocate(C++) with params=/size:4096/real_time/threads:1, source=cpp-micro, suite=arrow-memory-pool-benchmarkBM_ReadInt64Column(C++) with params=<Repetition::OPTIONAL, Compression::ZSTD>/65536/1024, source=cpp-micro, suite=parquet-column-io-benchmarkAsOfJoinOverhead(C++) with params=left_freq:400/left_cols:20/left_ids:500/batch_size:4000/num_right_tables:10/right_freq:400/right_cols:20/right_ids:500/real_time, source=cpp-micro, suite=arrow-acero-asof-join-benchmarkAsOfJoinOverhead(C++) with params=left_freq:200/left_cols:20/left_ids:500/batch_size:4000/num_right_tables:1/right_freq:200/right_cols:20/right_ids:500/real_time, source=cpp-micro, suite=arrow-acero-asof-join-benchmarkAsOfJoinOverhead(C++) with params=left_freq:400/left_cols:20/left_ids:1000/batch_size:4000/num_right_tables:1/right_freq:400/right_cols:20/right_ids:1000/real_time, source=cpp-micro, suite=arrow-acero-asof-join-benchmarkAsOfJoinOverhead(C++) with params=left_freq:400/left_cols:100/left_ids:500/batch_size:4000/num_right_tables:1/right_freq:400/right_cols:100/right_ids:500/real_time, source=cpp-micro, suite=arrow-acero-asof-join-benchmark
-
Pull Request Run on
test-mac-armat 2024-05-10 17:33:16ZBM_RleDecoding(C++) with params=65536/16, source=cpp-micro, suite=parquet-column-io-benchmarkShortVectorInsertAtEnd(C++) with params=<STD_VECTOR(int)>, source=cpp-micro, suite=arrow-small-vector-benchmarkBM_RleDecoding(C++) with params=32768/16, source=cpp-micro, suite=parquet-column-io-benchmarkBM_PlainEncodingFloat(C++) with params=32768, source=cpp-micro, suite=parquet-encoding-benchmarkBitBlockCounterFourWordsSumWithOffset(C++) with params=64, source=cpp-micro, suite=arrow-bit-block-counter-benchmarkMoveEmptyVector(C++) with params=<SMALL_VECTOR(std::string)>, source=cpp-micro, suite=arrow-small-vector-benchmarkBuildChunkedBinaryArray(C++) with source=cpp-micro, suite=arrow-builder-benchmarkBM_RleDecoding(C++) with params=4096/16, source=cpp-micro, suite=parquet-column-io-benchmarkGreaterArrayArrayString(C++) with params=65536/1, source=cpp-micro, suite=arrow-compute-scalar-compare-benchmarkArrayArrayKernel(C++) with params=<DivideChecked, Int16Type>/size:4194304/inverse_null_proportion:0, source=cpp-micro, suite=arrow-compute-scalar-arithmetic-benchmarkBatchToTensorSimple(C++) with params=/size:65536/num_columns:3, source=cpp-micro, suite=arrow-tensor-benchmarkReadCachedFile(C++) with params=num_cols:8/is_partial:0/real_time, source=cpp-micro, suite=arrow-ipc-read-write-benchmarkArrayScalarKernel(C++) with params=<MultiplyChecked, Int8Type>/size:4194304/inverse_null_proportion:100, source=cpp-micro, suite=arrow-compute-scalar-arithmetic-benchmarkGreaterArrayScalarString(C++) with params=65536/0, source=cpp-micro, suite=arrow-compute-scalar-compare-benchmarkArrayScalarKernel(C++) with params=<Add, UInt64Type>/size:4194304/inverse_null_proportion:100, source=cpp-micro, suite=arrow-compute-scalar-arithmetic-benchmarkArrayArrayKernel(C++) with params=<DivideChecked, Int32Type>/size:4194304/inverse_null_proportion:0, source=cpp-micro, suite=arrow-compute-scalar-arithmetic-benchmarkAllocateTouchDeallocate(C++) with params=/size:65536/real_time/threads:4, source=cpp-micro, suite=arrow-memory-pool-benchmarkArrayScalarKernel(C++) with params=<MultiplyChecked, Int8Type>/size:4194304/inverse_null_proportion:0, source=cpp-micro, suite=arrow-compute-scalar-arithmetic-benchmarkGreaterArrayScalarString(C++) with params=65536/1, source=cpp-micro, suite=arrow-compute-scalar-compare-benchmarkMoveShortVector(C++) with params=<SMALL_VECTOR(std::string)>, source=cpp-micro, suite=arrow-small-vector-benchmarkGreaterArrayScalarString(C++) with params=65536/100, source=cpp-micro, suite=arrow-compute-scalar-compare-benchmarkBM_RleDecoding(C++) with params=1024/16, source=cpp-micro, suite=parquet-column-io-benchmarkHexParsing(C++) with params=, source=cpp-micro, suite=arrow-value-parsing-benchmarkGreaterArrayScalarString(C++) with params=65536/10000, source=cpp-micro, suite=arrow-compute-scalar-compare-benchmarkConstructFromEmptyStdVector(C++) with params=<SMALL_VECTOR(std::string)>, source=cpp-micro, suite=arrow-small-vector-benchmarkReadCachedFile(C++) with params=num_cols:1/is_partial:1/real_time, source=cpp-micro, suite=arrow-ipc-read-write-benchmarkTouchArea(C++) with params=size:65536/real_time/threads:4, source=cpp-micro, suite=arrow-memory-pool-benchmarkArrayScalarKernel(C++) with params=<Add, UInt64Type>/size:4194304/inverse_null_proportion:0, source=cpp-micro, suite=arrow-compute-scalar-arithmetic-benchmarkCopyEmptyVector(C++) with params=<STATIC_VECTOR(std::string)>, source=cpp-micro, suite=arrow-small-vector-benchmark
-
Pull Request Run on
arm64-t4g-linux-computeat 2024-05-10 18:30:34ZValidateSmallAscii(C++) with source=cpp-micro, suite=arrow-utf8-util-benchmarkBM_ReadInt64Column(C++) with params=<Repetition::OPTIONAL, Compression::ZSTD>/65536/1024, source=cpp-micro, suite=parquet-column-io-benchmarkValidateTinyAscii(C++) with source=cpp-micro, suite=arrow-utf8-util-benchmarkValidateLargeAscii(C++) with source=cpp-micro, suite=arrow-utf8-util-benchmarkBM_DefinitionLevelsToBitmapRepeatedMostPresent(C++) with source=cpp-micro, suite=parquet-level-conversion-benchmark
Details
This report was generated using the lookback z-score method with a z-score threshold of 30.0.
Loading