GH-41771: [C++] Iterator releases its resource immediately when it re…
conbench-apache-arrow / Conbench performance report
failed
Jun 3, 2024
Found 6 regressions
After merging your PR, Conbench analyzed the 6 benchmarking runs that have been run so far on merge-commit e6e00e7.
Benchmarks with performance regressions
There were 6 possible performance regressions, according to the lookback z-score method.
Benchmarks with regressions:
- Commit Run on
ec2-m5-4xlarge-us-east-2at 2024-05-28 15:42:38Zdataset-serialize(Python) with dataset=nyctaxi_multi_parquet_s3, format=csv, selectivity=1pcdataset-serialize(Python) with dataset=nyctaxi_multi_ipc_s3, format=csv, selectivity=1pcdataset-serialize(Python) with dataset=nyctaxi_multi_parquet_s3, format=csv, selectivity=10pcdataset-serialize(Python) with dataset=nyctaxi_multi_parquet_s3, format=csv, selectivity=100pcdataset-serialize(Python) with dataset=nyctaxi_multi_ipc_s3, format=csv, selectivity=10pcdataset-serialize(Python) with dataset=nyctaxi_multi_ipc_s3, format=csv, selectivity=100pc
All benchmark runs analyzed:
- Commit Run on
ursa-thinkcentre-m75qat 2024-05-28 16:34:54Z - Commit Run on
arm64-m6g-linux-computeat 2024-05-28 18:57:48Z - Commit Run on
arm64-t4g-linux-computeat 2024-05-28 18:13:32Z - Commit Run on
ec2-t3-xlarge-us-east-2at 2024-05-28 19:00:38Z - Commit Run on
ec2-m5-4xlarge-us-east-2at 2024-05-28 15:42:38Z - Commit Run on
test-mac-armat 2024-05-28 18:55:36Z
Unstable benchmarks with performance regressions
The following benchmark results indicate a possible performance regression, but are known to sometimes produce false positives when applying the lookback z-score analysis.
-
Commit Run on
ursa-thinkcentre-m75qat 2024-05-28 16:34:54ZFilterFixedSizeBinaryFilterNoNulls(C++) with params=524288/11/8, source=cpp-micro, suite=arrow-compute-vector-selection-benchmarkAsOfJoinOverhead(C++) with params=left_freq:1000/left_cols:20/left_ids:500/batch_size:4000/num_right_tables:1/right_freq:1000/right_cols:20/right_ids:500/real_time, source=cpp-micro, suite=arrow-acero-asof-join-benchmarkBatchToTensorSimple(C++) with params=/size:32768/num_columns:3, source=cpp-micro, suite=arrow-tensor-benchmarkParseJSONFields(C++) with params=ordered:0/schema:1/sparsity:0/num_fields:10, source=cpp-micro, suite=arrow-json-parser-benchmarkBM_FindExistingHash(C++) with source=cpp-micro, suite=parquet-bloom-filter-benchmarkAsOfJoinOverhead(C++) with params=left_freq:400/left_cols:20/left_ids:1000/batch_size:4000/num_right_tables:1/right_freq:400/right_cols:20/right_ids:1000/real_time, source=cpp-micro, suite=arrow-acero-asof-join-benchmarkIsInStringSmallSet(C++) with params=64, source=cpp-micro, suite=arrow-compute-scalar-set-lookup-benchmarkBM_RleDecoding(C++) with params=1024/16, source=cpp-micro, suite=parquet-column-io-benchmarkAsOfJoinOverhead(C++) with params=left_freq:200/left_cols:20/left_ids:500/batch_size:4000/num_right_tables:1/right_freq:200/right_cols:20/right_ids:500/real_time, source=cpp-micro, suite=arrow-acero-asof-join-benchmarkBM_BatchComputeHash(C++) with params=, source=cpp-micro, suite=parquet-bloom-filter-benchmarkRoundArrayBenchmark(C++) with params=<Round, UInt32Type, RoundMode::HALF_TO_ODD>/size:524288/inverse_null_proportion:100, source=cpp-micro, suite=arrow-compute-scalar-round-benchmarkRoundArrayBenchmark(C++) with params=<Round, FloatType, RoundMode::HALF_TO_ODD>/size:524288/inverse_null_proportion:100, source=cpp-micro, suite=arrow-compute-scalar-round-benchmarkArrayScalarKernel(C++) with params=<Add, DoubleType>/size:524288/inverse_null_proportion:100, source=cpp-micro, suite=arrow-compute-scalar-arithmetic-benchmarkArrayScalarKernel(C++) with params=<Add, FloatType>/size:524288/inverse_null_proportion:100, source=cpp-micro, suite=arrow-compute-scalar-arithmetic-benchmarkFilterFSLInt64FilterNoNulls(C++) with params=524288/5, source=cpp-micro, suite=arrow-compute-vector-selection-benchmarkInt8StridedTensorConversionFixture(C++) with params=/ConvertToSparseCOOTensorInt32, source=cpp-micro, suite=arrow-tensor-conversion-benchmarkAsOfJoinOverhead(C++) with params=left_freq:400/left_cols:10/left_ids:500/batch_size:4000/num_right_tables:1/right_freq:400/right_cols:10/right_ids:500/real_time, source=cpp-micro, suite=arrow-acero-asof-join-benchmarkRoundArrayBenchmark(C++) with params=<Round, UInt64Type, RoundMode::DOWN>/size:524288/inverse_null_proportion:0, source=cpp-micro, suite=arrow-compute-scalar-round-benchmarkRoundArrayBenchmark(C++) with params=<Round, FloatType, RoundMode::HALF_TOWARDS_ZERO>/size:524288/inverse_null_proportion:100, source=cpp-micro, suite=arrow-compute-scalar-round-benchmarkBM_RleEncoding(C++) with params=65536/1, source=cpp-micro, suite=parquet-column-io-benchmarkFilterStringFilterNoNulls(C++) with params=524288/0, source=cpp-micro, suite=arrow-compute-vector-selection-benchmarkArrayScalarKernel(C++) with params=<Add, UInt16Type>/size:524288/inverse_null_proportion:0, source=cpp-micro, suite=arrow-compute-scalar-arithmetic-benchmarkIsInStringSmallSet(C++) with params=2, source=cpp-micro, suite=arrow-compute-scalar-set-lookup-benchmarkFilterInt64FilterNoNulls(C++) with params=524288/14, source=cpp-micro, suite=arrow-compute-vector-selection-benchmarkBM_RleDecoding(C++) with params=4096/16, source=cpp-micro, suite=parquet-column-io-benchmarkFilterFixedSizeBinaryFilterWithNulls(C++) with params=524288/1/9, source=cpp-micro, suite=arrow-compute-vector-selection-benchmarkRoundArrayBenchmark(C++) with params=<Round, UInt32Type, RoundMode::HALF_TO_ODD>/size:524288/inverse_null_proportion:0, source=cpp-micro, suite=arrow-compute-scalar-round-benchmarkBatchToTensorSimple(C++) with params=/size:524288/num_columns:3, source=cpp-micro, suite=arrow-tensor-benchmarkArrayScalarKernel(C++) with params=<Add, UInt16Type>/size:524288/inverse_null_proportion:100, source=cpp-micro, suite=arrow-compute-scalar-arithmetic-benchmarkMaxElementWiseArrayScalarString(C++) with params=32768/0, source=cpp-micro, suite=arrow-compute-scalar-compare-benchmarkRoundDerivativesArrayBenchmark(C++) with params=<Trunc, Int64Type>/size:524288/inverse_null_proportion:0, source=cpp-micro, suite=arrow-compute-scalar-round-benchmarkBatchToTensorSimple(C++) with params=/size:32768/num_columns:3, source=cpp-micro, suite=arrow-tensor-benchmarkAsOfJoinOverhead(C++) with params=left_freq:400/left_cols:20/left_ids:100/batch_size:4000/num_right_tables:1/right_freq:400/right_cols:20/right_ids:100/real_time, source=cpp-micro, suite=arrow-acero-asof-join-benchmarkBM_ArrowBinaryDict(C++) with params=DecodeArrowNonNull_Dense/4096, source=cpp-micro, suite=parquet-encoding-benchmarkBM_ArrowBinaryDict(C++) with params=DecodeArrowNonNull_Dense/32768, source=cpp-micro, suite=parquet-encoding-benchmarkRoundDerivativesArrayBenchmark(C++) with params=<Trunc, UInt64Type>/size:524288/inverse_null_proportion:100, source=cpp-micro, suite=arrow-compute-scalar-round-benchmarkArrayScalarKernel(C++) with params=<Multiply, Int8Type>/size:524288/inverse_null_proportion:0, source=cpp-micro, suite=arrow-compute-scalar-arithmetic-benchmarkFilterFixedSizeBinaryFilterWithNulls(C++) with params=524288/0/9, source=cpp-micro, suite=arrow-compute-vector-selection-benchmarkChunkCSVEscapedBlock(C++) with source=cpp-micro, suite=arrow-csv-parser-benchmarkBM_DeltaLengthDecodingSpacedByteArray(C++) with params=max-string-length:1024/batch-size:2048, source=cpp-micro, suite=parquet-encoding-benchmarkCheckIndexBoundsUInt32(C++) with params=32768/1, source=cpp-micro, suite=arrow-int-util-benchmarkFilterFixedSizeBinaryFilterWithNulls(C++) with params=524288/3/9, source=cpp-micro, suite=arrow-compute-vector-selection-benchmarkParseJSONFields(C++) with params=ordered:0/schema:1/sparsity:10/num_fields:10, source=cpp-micro, suite=arrow-json-parser-benchmarkArrayScalarKernel(C++) with params=<MultiplyChecked, Int32Type>/size:524288/inverse_null_proportion:100, source=cpp-micro, suite=arrow-compute-scalar-arithmetic-benchmarkBM_RleEncoding(C++) with params=4096/1, source=cpp-micro, suite=parquet-column-io-benchmarkIsInStringSmallSet(C++) with params=4, source=cpp-micro, suite=arrow-compute-scalar-set-lookup-benchmarkBM_ExecuteScalarKernelOnScalar(C++) with source=cpp-micro, suite=arrow-compute-function-benchmarkAsOfJoinOverhead(C++) with params=left_freq:400/left_cols:20/left_ids:500/batch_size:4000/num_right_tables:50/right_freq:400/right_cols:20/right_ids:500/real_time, source=cpp-micro, suite=arrow-acero-asof-join-benchmarkBatchToTensorSimple(C++) with params=/size:524288/num_columns:3, source=cpp-micro, suite=arrow-tensor-benchmarkArrayScalarKernel(C++) with params=<Multiply, Int8Type>/size:524288/inverse_null_proportion:100, source=cpp-micro, suite=arrow-compute-scalar-arithmetic-benchmarkBM_ArrowBinaryDict(C++) with params=DecodeArrow_Dense/4096, source=cpp-micro, suite=parquet-encoding-benchmarkDoubleRowMajorTensorConversionFixture(C++) with params=/ConvertToSparseCOOTensorInt64, source=cpp-micro, suite=arrow-tensor-conversion-benchmarkBM_DeltaLengthDecodingByteArray(C++) with params=max-string-length:1024/batch-size:2048, source=cpp-micro, suite=parquet-encoding-benchmarkRoundDerivativesArrayBenchmark(C++) with params=<Floor, Int64Type>/size:524288/inverse_null_proportion:0, source=cpp-micro, suite=arrow-compute-scalar-round-benchmarkBenchmarkTemporalRounding(C++) with params=<RoundTemporal, non_zoned, round_1_month>/524288/0, source=cpp-micro, suite=arrow-compute-scalar-temporal-benchmarkRoundDerivativesArrayBenchmark(C++) with params=<Ceil, FloatType>/size:524288/inverse_null_proportion:100, source=cpp-micro, suite=arrow-compute-scalar-round-benchmarkAsOfJoinOverhead(C++) with params=left_freq:400/left_cols:20/left_ids:500/batch_size:4000/num_right_tables:1/right_freq:400/right_cols:20/right_ids:500/real_time, source=cpp-micro, suite=arrow-acero-asof-join-benchmarkRoundArrayBenchmark(C++) with params=<Round, Int64Type, RoundMode::HALF_TOWARDS_ZERO>/size:524288/inverse_null_proportion:0, source=cpp-micro, suite=arrow-compute-scalar-round-benchmarkTakeChunkedChunkedStringMonotonicIndices(C++) with params=524288/0, source=cpp-micro, suite=arrow-compute-vector-selection-benchmarkArrayScalarKernel(C++) with params=<Subtract, UInt8Type>/size:524288/inverse_null_proportion:0, source=cpp-micro, suite=arrow-compute-scalar-arithmetic-benchmarkRoundDerivativesArrayBenchmark(C++) with params=<Trunc, Int64Type>/size:524288/inverse_null_proportion:100, source=cpp-micro, suite=arrow-compute-scalar-round-benchmarkBM_DeltaBitPackingEncode_Int64_Narrow(C++) with params=65536, source=cpp-micro, suite=parquet-encoding-benchmarkFilterFixedSizeBinaryFilterWithNulls(C++) with params=524288/4/9, source=cpp-micro, suite=arrow-compute-vector-selection-benchmarkRoundArrayBenchmark(C++) with params=<Round, UInt32Type, RoundMode::DOWN>/size:524288/inverse_null_proportion:100, source=cpp-micro, suite=arrow-compute-scalar-round-benchmarkArrayScalarKernel(C++) with params=<Add, DoubleType>/size:524288/inverse_null_proportion:0, source=cpp-micro, suite=arrow-compute-scalar-arithmetic-benchmarkFilterFixedSizeBinaryFilterNoNulls(C++) with params=524288/14/8, source=cpp-micro, suite=arrow-compute-vector-selection-benchmarkTrimSingleUtf8(C++) with source=cpp-micro, suite=arrow-compute-scalar-string-benchmarkRoundArrayBenchmark(C++) with params=<Round, UInt32Type, RoundMode::DOWN>/size:524288/inverse_null_proportion:0, source=cpp-micro, suite=arrow-compute-scalar-round-benchmarkArrayScalarKernel(C++) with params=<Subtract, UInt8Type>/size:524288/inverse_null_proportion:100, source=cpp-micro, suite=arrow-compute-scalar-arithmetic-benchmarkRoundDerivativesArrayBenchmark(C++) with params=<Ceil, FloatType>/size:524288/inverse_null_proportion:0, source=cpp-micro, suite=arrow-compute-scalar-round-benchmarkBM_RleEncoding(C++) with params=32768/1, source=cpp-micro, suite=parquet-column-io-benchmarkBenchmarkTemporalBinary(C++) with params=<QuartersBetween, date64_type>/524288/100, source=cpp-micro, suite=arrow-compute-scalar-temporal-benchmarkSetBitsTo(C++) with params=16, source=cpp-micro, suite=arrow-bit-util-benchmarkBenchmarkTemporalRounding(C++) with params=<RoundTemporal, non_zoned, round_1_month>/524288/100, source=cpp-micro, suite=arrow-compute-scalar-temporal-benchmarkArrayScalarKernel(C++) with params=<Add, FloatType>/size:524288/inverse_null_proportion:0, source=cpp-micro, suite=arrow-compute-scalar-arithmetic-benchmarkBM_ArrowBinaryDict(C++) with params=DecodeArrow_Dense/65536, source=cpp-micro, suite=parquet-encoding-benchmarkAsOfJoinOverhead(C++) with params=left_freq:400/left_cols:100/left_ids:500/batch_size:4000/num_right_tables:1/right_freq:400/right_cols:100/right_ids:500/real_time, source=cpp-micro, suite=arrow-acero-asof-join-benchmarkArrayScalarKernel(C++) with params=<AddChecked, UInt8Type>/size:524288/inverse_null_proportion:100, source=cpp-micro, suite=arrow-compute-scalar-arithmetic-benchmarkRoundArrayBenchmark(C++) with params=<Round, Int32Type, RoundMode::HALF_TOWARDS_ZERO>/size:524288/inverse_null_proportion:100, source=cpp-micro, suite=arrow-compute-scalar-round-benchmarkRoundDerivativesArrayBenchmark(C++) with params=<Trunc, DoubleType>/size:524288/inverse_null_proportion:100, source=cpp-micro, suite=arrow-compute-scalar-round-benchmarkRoundDerivativesArrayBenchmark(C++) with params=<Trunc, DoubleType>/size:524288/inverse_null_proportion:0, source=cpp-micro, suite=arrow-compute-scalar-round-benchmarkBM_ReadColumn(C++) with params=<false,Int64Type>/-1/50, source=cpp-micro, suite=parquet-arrow-reader-writer-benchmark
-
Commit Run on
arm64-t4g-linux-computeat 2024-05-28 18:13:32ZGreaterArrayScalarString(C++) with params=65536/10000, source=cpp-micro, suite=arrow-compute-scalar-compare-benchmarkGreaterArrayScalarString(C++) with params=65536/2, source=cpp-micro, suite=arrow-compute-scalar-compare-benchmarkMaxElementWiseArrayScalarInt64(C++) with params=65536/1, source=cpp-micro, suite=arrow-compute-scalar-compare-benchmarkBatchToTensorSimple(C++) with params=/size:65536/num_columns:30, source=cpp-micro, suite=arrow-tensor-benchmarkGreaterArrayScalarString(C++) with params=65536/1, source=cpp-micro, suite=arrow-compute-scalar-compare-benchmarkBatchToTensorSimple(C++) with params=/size:65536/num_columns:3, source=cpp-micro, suite=arrow-tensor-benchmarkGreaterArrayScalarString(C++) with params=65536/0, source=cpp-micro, suite=arrow-compute-scalar-compare-benchmarkBatchToTensorSimple(C++) with params=/size:1048576/num_columns:3, source=cpp-micro, suite=arrow-tensor-benchmarkMaxElementWiseArrayScalarInt64(C++) with params=65536/10000, source=cpp-micro, suite=arrow-compute-scalar-compare-benchmarkGreaterArrayScalarString(C++) with params=65536/100, source=cpp-micro, suite=arrow-compute-scalar-compare-benchmarkReadLevels_BitPack(C++) with params=MaxLevel:3/NumLevels:8096/BatchSize:1024/LevelRepeatCount:1, source=cpp-micro, suite=parquet-column-reader-benchmarkGreaterArrayScalarString(C++) with params=65536/10, source=cpp-micro, suite=arrow-compute-scalar-compare-benchmark
-
Commit Run on
ec2-t3-xlarge-us-east-2at 2024-05-28 19:00:38ZArrayScalarKernel(C++) with params=<SubtractChecked, Int16Type>/size:1048576/inverse_null_proportion:0, source=cpp-micro, suite=arrow-compute-scalar-arithmetic-benchmarkMaxElementWiseArrayScalarInt64(C++) with params=32768/2, source=cpp-micro, suite=arrow-compute-scalar-compare-benchmarkArrayRankStringNarrow(C++) with params=32768/10000/tiebreaker:0, source=cpp-micro, suite=arrow-compute-vector-sort-benchmark
-
Commit Run on
test-mac-armat 2024-05-28 18:55:36ZBenchmarkTemporalRounding(C++) with params=<RoundTemporal, non_zoned, round_1_week>/4194304/0, source=cpp-micro, suite=arrow-compute-scalar-temporal-benchmarkRoundDerivativesArrayBenchmark(C++) with params=<Ceil, DoubleType>/size:4194304/inverse_null_proportion:100, source=cpp-micro, suite=arrow-compute-scalar-round-benchmarkBM_BatchComputeHash(C++) with params=, source=cpp-micro, suite=parquet-bloom-filter-benchmarkBM_PlainDecodingByteArray(C++) with params=max-string-length:64/batch-size:512, source=cpp-micro, suite=parquet-encoding-benchmarkBenchmarkTemporalRounding(C++) with params=<RoundTemporal, non_zoned, round_1_week>/4194304/100, source=cpp-micro, suite=arrow-compute-scalar-temporal-benchmark
Details
This report was generated using the lookback z-score method with a z-score threshold of 30.0.
Loading