enhance: test block transpose

[lyang24]

Which issue does this PR close?

• Closes #NNN.

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

[Dandandan]

run benchmark arrow_reader

[alamb-ghbot]

🤖 ./gh_compare_arrow.sh gh_compare_arrow.sh Running
Linux aal-dev 6.14.0-1018-gcp #19~24.04.1-Ubuntu SMP Wed Sep 24 23:23:09 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Comparing batch8 (6a7d540) to 3c6ca57 diff
BENCH_NAME=arrow_reader
BENCH_COMMAND=cargo bench --features=arrow,async,test_common,experimental,object_store --bench arrow_reader
BENCH_FILTER=
BENCH_BRANCH_NAME=batch8
Results will be posted here when complete

[alamb-ghbot]

🤖: Benchmark completed

Details

group                                                                                                      batch8                                 main
-----                                                                                                      ------                                 ----
arrow_array_reader/BYTE_ARRAY/Decimal128Array/plain encoded, mandatory, no NULLs                           1.06   1247.7±5.50µs        ? ?/sec    1.00   1178.1±7.85µs        ? ?/sec
arrow_array_reader/BYTE_ARRAY/Decimal128Array/plain encoded, optional, half NULLs                          1.02   1270.9±7.77µs        ? ?/sec    1.00   1252.1±8.78µs        ? ?/sec
arrow_array_reader/BYTE_ARRAY/Decimal128Array/plain encoded, optional, no NULLs                            1.06   1255.0±5.50µs        ? ?/sec    1.00  1187.0±14.15µs        ? ?/sec
arrow_array_reader/BinaryArray/dictionary encoded, mandatory, no NULLs                                     1.06    509.9±8.21µs        ? ?/sec    1.00    479.4±2.93µs        ? ?/sec
arrow_array_reader/BinaryArray/dictionary encoded, optional, half NULLs                                    1.00    659.0±4.57µs        ? ?/sec    1.01    665.9±7.25µs        ? ?/sec
arrow_array_reader/BinaryArray/dictionary encoded, optional, no NULLs                                      1.03    511.6±5.11µs        ? ?/sec    1.00    494.9±9.43µs        ? ?/sec
arrow_array_reader/BinaryArray/plain encoded, mandatory, no NULLs                                          1.03   559.4±11.03µs        ? ?/sec    1.00    540.9±3.94µs        ? ?/sec
arrow_array_reader/BinaryArray/plain encoded, optional, half NULLs                                         1.00    722.1±6.07µs        ? ?/sec    1.00    718.8±4.81µs        ? ?/sec
arrow_array_reader/BinaryArray/plain encoded, optional, no NULLs                                           1.03    572.3±7.35µs        ? ?/sec    1.00   557.1±22.01µs        ? ?/sec
arrow_array_reader/BinaryViewArray/dictionary encoded, mandatory, no NULLs                                 1.01    156.0±2.98µs        ? ?/sec    1.00    153.8±1.41µs        ? ?/sec
arrow_array_reader/BinaryViewArray/dictionary encoded, optional, half NULLs                                1.03    225.0±1.50µs        ? ?/sec    1.00    217.8±5.92µs        ? ?/sec
arrow_array_reader/BinaryViewArray/dictionary encoded, optional, no NULLs                                  1.07   165.6±10.72µs        ? ?/sec    1.00    154.1±2.80µs        ? ?/sec
arrow_array_reader/BinaryViewArray/plain encoded, mandatory, no NULLs                                      1.01    221.2±1.91µs        ? ?/sec    1.00    218.5±4.34µs        ? ?/sec
arrow_array_reader/BinaryViewArray/plain encoded, mandatory, no NULLs, short string                        1.00    192.8±3.31µs        ? ?/sec    1.21    232.9±1.40µs        ? ?/sec
arrow_array_reader/BinaryViewArray/plain encoded, optional, half NULLs                                     1.01    258.9±3.47µs        ? ?/sec    1.00    256.9±2.92µs        ? ?/sec
arrow_array_reader/BinaryViewArray/plain encoded, optional, no NULLs                                       1.00    230.9±4.38µs        ? ?/sec    1.01    232.5±6.79µs        ? ?/sec
arrow_array_reader/FIXED_LEN_BYTE_ARRAY/Decimal128Array/byte_stream_split encoded, mandatory, no NULLs     1.06  1139.5±18.74µs        ? ?/sec    1.00  1077.1±17.34µs        ? ?/sec
arrow_array_reader/FIXED_LEN_BYTE_ARRAY/Decimal128Array/byte_stream_split encoded, optional, half NULLs    1.00   929.5±11.67µs        ? ?/sec    1.01    940.5±8.70µs        ? ?/sec
arrow_array_reader/FIXED_LEN_BYTE_ARRAY/Decimal128Array/byte_stream_split encoded, optional, no NULLs      1.05   1143.6±8.49µs        ? ?/sec    1.00  1084.6±21.37µs        ? ?/sec
arrow_array_reader/FIXED_LEN_BYTE_ARRAY/Decimal128Array/plain encoded, mandatory, no NULLs                 1.03    464.5±3.51µs        ? ?/sec    1.00    453.0±8.26µs        ? ?/sec
arrow_array_reader/FIXED_LEN_BYTE_ARRAY/Decimal128Array/plain encoded, optional, half NULLs                1.00    590.1±6.31µs        ? ?/sec    1.06    626.9±8.55µs        ? ?/sec
arrow_array_reader/FIXED_LEN_BYTE_ARRAY/Decimal128Array/plain encoded, optional, no NULLs                  1.03    473.3±5.83µs        ? ?/sec    1.00    460.4±7.68µs        ? ?/sec
arrow_array_reader/FIXED_LEN_BYTE_ARRAY/Float16Array/byte_stream_split encoded, mandatory, no NULLs        1.04    202.3±1.20µs        ? ?/sec    1.00    195.3±4.14µs        ? ?/sec
arrow_array_reader/FIXED_LEN_BYTE_ARRAY/Float16Array/byte_stream_split encoded, optional, half NULLs       1.00    343.8±2.22µs        ? ?/sec    1.05    360.2±2.38µs        ? ?/sec
arrow_array_reader/FIXED_LEN_BYTE_ARRAY/Float16Array/byte_stream_split encoded, optional, no NULLs         1.04    207.9±3.59µs        ? ?/sec    1.00    199.5±2.56µs        ? ?/sec
arrow_array_reader/FIXED_LEN_BYTE_ARRAY/Float16Array/plain encoded, mandatory, no NULLs                    1.00    118.5±0.58µs        ? ?/sec    1.00    118.8±2.15µs        ? ?/sec
arrow_array_reader/FIXED_LEN_BYTE_ARRAY/Float16Array/plain encoded, optional, half NULLs                   1.00    301.1±3.08µs        ? ?/sec    1.07    321.1±3.47µs        ? ?/sec
arrow_array_reader/FIXED_LEN_BYTE_ARRAY/Float16Array/plain encoded, optional, no NULLs                     1.00    124.4±0.63µs        ? ?/sec    1.00    124.7±0.71µs        ? ?/sec
arrow_array_reader/FixedLenByteArray(16)/byte_stream_split encoded, mandatory, no NULLs                    1.09    750.7±7.88µs        ? ?/sec    1.00   688.3±13.43µs        ? ?/sec
arrow_array_reader/FixedLenByteArray(16)/byte_stream_split encoded, optional, half NULLs                   1.00    546.3±3.71µs        ? ?/sec    1.00    543.9±8.34µs        ? ?/sec
arrow_array_reader/FixedLenByteArray(16)/byte_stream_split encoded, optional, no NULLs                     1.09   757.1±15.49µs        ? ?/sec    1.00    693.0±6.78µs        ? ?/sec
arrow_array_reader/FixedLenByteArray(16)/plain encoded, mandatory, no NULLs                                1.02     68.9±6.54µs        ? ?/sec    1.00     67.2±2.79µs        ? ?/sec
arrow_array_reader/FixedLenByteArray(16)/plain encoded, optional, half NULLs                               1.00    211.3±9.09µs        ? ?/sec    1.12    236.2±3.97µs        ? ?/sec
arrow_array_reader/FixedLenByteArray(16)/plain encoded, optional, no NULLs                                 1.06     74.3±4.41µs        ? ?/sec    1.00     70.1±2.54µs        ? ?/sec
arrow_array_reader/FixedLenByteArray(2)/byte_stream_split encoded, mandatory, no NULLs                     1.10     94.6±2.90µs        ? ?/sec    1.00     85.8±0.89µs        ? ?/sec
arrow_array_reader/FixedLenByteArray(2)/byte_stream_split encoded, optional, half NULLs                    1.00    232.7±1.10µs        ? ?/sec    1.07    249.1±2.90µs        ? ?/sec
arrow_array_reader/FixedLenByteArray(2)/byte_stream_split encoded, optional, no NULLs                      1.08     99.1±0.59µs        ? ?/sec    1.00     91.9±1.81µs        ? ?/sec
arrow_array_reader/FixedLenByteArray(2)/plain encoded, mandatory, no NULLs                                 1.00      9.1±0.15µs        ? ?/sec    1.01      9.2±0.15µs        ? ?/sec
arrow_array_reader/FixedLenByteArray(2)/plain encoded, optional, half NULLs                                1.00    189.4±1.61µs        ? ?/sec    1.12    211.5±9.09µs        ? ?/sec
arrow_array_reader/FixedLenByteArray(2)/plain encoded, optional, no NULLs                                  1.00     14.3±0.17µs        ? ?/sec    1.00     14.3±0.25µs        ? ?/sec
arrow_array_reader/FixedLenByteArray(4)/byte_stream_split encoded, mandatory, no NULLs                     1.08    185.1±1.08µs        ? ?/sec    1.00    170.9±1.33µs        ? ?/sec
arrow_array_reader/FixedLenByteArray(4)/byte_stream_split encoded, optional, half NULLs                    1.00    368.2±3.23µs        ? ?/sec    1.06    391.0±1.63µs        ? ?/sec
arrow_array_reader/FixedLenByteArray(4)/byte_stream_split encoded, optional, no NULLs                      1.09    190.5±1.52µs        ? ?/sec    1.00    175.0±0.81µs        ? ?/sec
arrow_array_reader/FixedLenByteArray(4)/plain encoded, mandatory, no NULLs                                 1.00     14.1±0.11µs        ? ?/sec    1.00     14.1±0.18µs        ? ?/sec
arrow_array_reader/FixedLenByteArray(4)/plain encoded, optional, half NULLs                                1.00    284.6±1.96µs        ? ?/sec    1.11    315.7±6.39µs        ? ?/sec
arrow_array_reader/FixedLenByteArray(4)/plain encoded, optional, no NULLs                                  1.01     20.4±0.46µs        ? ?/sec    1.00     20.1±0.32µs        ? ?/sec
arrow_array_reader/FixedLenByteArray(8)/byte_stream_split encoded, mandatory, no NULLs                     1.08    368.5±4.86µs        ? ?/sec    1.00    341.0±1.82µs        ? ?/sec
arrow_array_reader/FixedLenByteArray(8)/byte_stream_split encoded, optional, half NULLs                    1.00    343.2±2.95µs        ? ?/sec    1.03    354.9±2.94µs        ? ?/sec
arrow_array_reader/FixedLenByteArray(8)/byte_stream_split encoded, optional, no NULLs                      1.08    374.9±3.77µs        ? ?/sec    1.00    346.8±2.25µs        ? ?/sec
arrow_array_reader/FixedLenByteArray(8)/plain encoded, mandatory, no NULLs                                 1.00     27.0±0.55µs        ? ?/sec    1.00     26.9±0.48µs        ? ?/sec
arrow_array_reader/FixedLenByteArray(8)/plain encoded, optional, half NULLs                                1.00    173.6±5.94µs        ? ?/sec    1.16    200.8±1.15µs        ? ?/sec
arrow_array_reader/FixedLenByteArray(8)/plain encoded, optional, no NULLs                                  1.00     33.0±0.34µs        ? ?/sec    1.02     33.6±0.49µs        ? ?/sec
arrow_array_reader/INT32/Decimal128Array/binary packed skip, mandatory, no NULLs                           1.01    110.1±1.94µs        ? ?/sec    1.00    108.6±0.52µs        ? ?/sec
arrow_array_reader/INT32/Decimal128Array/binary packed skip, optional, half NULLs                          1.01    131.3±0.91µs        ? ?/sec    1.00    129.7±0.76µs        ? ?/sec
arrow_array_reader/INT32/Decimal128Array/binary packed skip, optional, no NULLs                            1.01    113.7±1.10µs        ? ?/sec    1.00    112.4±2.97µs        ? ?/sec
arrow_array_reader/INT32/Decimal128Array/binary packed, mandatory, no NULLs                                1.01    160.9±0.75µs        ? ?/sec    1.00    158.5±1.90µs        ? ?/sec
arrow_array_reader/INT32/Decimal128Array/binary packed, optional, half NULLs                               1.01    224.5±1.97µs        ? ?/sec    1.00    222.4±5.78µs        ? ?/sec
arrow_array_reader/INT32/Decimal128Array/binary packed, optional, no NULLs                                 1.01    165.4±2.45µs        ? ?/sec    1.00    164.0±2.02µs        ? ?/sec
arrow_array_reader/INT32/Decimal128Array/byte_stream_split encoded, mandatory, no NULLs                    3.10    234.5±0.93µs        ? ?/sec    1.00     75.7±0.43µs        ? ?/sec
arrow_array_reader/INT32/Decimal128Array/byte_stream_split encoded, optional, half NULLs                   1.47    261.8±7.82µs        ? ?/sec    1.00    177.6±3.40µs        ? ?/sec
arrow_array_reader/INT32/Decimal128Array/byte_stream_split encoded, optional, no NULLs                     2.91    239.9±5.55µs        ? ?/sec    1.00     82.6±0.38µs        ? ?/sec
arrow_array_reader/INT32/Decimal128Array/dictionary encoded, mandatory, no NULLs                           1.07    144.1±2.90µs        ? ?/sec    1.00    134.9±1.00µs        ? ?/sec
arrow_array_reader/INT32/Decimal128Array/dictionary encoded, optional, half NULLs                          1.03    217.8±1.84µs        ? ?/sec    1.00    211.2±2.81µs        ? ?/sec
arrow_array_reader/INT32/Decimal128Array/dictionary encoded, optional, no NULLs                            1.06    148.5±1.62µs        ? ?/sec    1.00    140.5±0.79µs        ? ?/sec
arrow_array_reader/INT32/Decimal128Array/plain encoded, mandatory, no NULLs                                1.02     72.5±0.90µs        ? ?/sec    1.00     70.8±0.31µs        ? ?/sec
arrow_array_reader/INT32/Decimal128Array/plain encoded, optional, half NULLs                               1.02    179.4±1.32µs        ? ?/sec    1.00    175.5±1.75µs        ? ?/sec
arrow_array_reader/INT32/Decimal128Array/plain encoded, optional, no NULLs                                 1.00     77.5±0.56µs        ? ?/sec    1.01     78.3±1.12µs        ? ?/sec
arrow_array_reader/INT64/Decimal128Array/binary packed skip, mandatory, no NULLs                           1.00    108.5±0.39µs        ? ?/sec    1.00    108.7±0.84µs        ? ?/sec
arrow_array_reader/INT64/Decimal128Array/binary packed skip, optional, half NULLs                          1.02    123.6±0.82µs        ? ?/sec    1.00    121.4±0.99µs        ? ?/sec
arrow_array_reader/INT64/Decimal128Array/binary packed skip, optional, no NULLs                            1.00    111.9±1.84µs        ? ?/sec    1.00    111.7±3.88µs        ? ?/sec
arrow_array_reader/INT64/Decimal128Array/binary packed, mandatory, no NULLs                                1.00    160.8±1.14µs        ? ?/sec    1.01    162.0±1.77µs        ? ?/sec
arrow_array_reader/INT64/Decimal128Array/binary packed, optional, half NULLs                               1.00    209.1±1.28µs        ? ?/sec    1.00    208.1±2.76µs        ? ?/sec
arrow_array_reader/INT64/Decimal128Array/binary packed, optional, no NULLs                                 1.00    166.4±2.87µs        ? ?/sec    1.00    167.2±1.12µs        ? ?/sec
arrow_array_reader/INT64/Decimal128Array/byte_stream_split encoded, mandatory, no NULLs                    2.75    551.8±5.94µs        ? ?/sec    1.00    200.5±0.66µs        ? ?/sec
arrow_array_reader/INT64/Decimal128Array/byte_stream_split encoded, optional, half NULLs                   1.78    400.9±2.34µs        ? ?/sec    1.00    225.1±1.34µs        ? ?/sec
arrow_array_reader/INT64/Decimal128Array/byte_stream_split encoded, optional, no NULLs                     2.70    558.5±5.38µs        ? ?/sec    1.00    207.2±1.56µs        ? ?/sec
arrow_array_reader/INT64/Decimal128Array/dictionary encoded, mandatory, no NULLs                           1.00    143.8±1.05µs        ? ?/sec    1.04    150.0±2.11µs        ? ?/sec
arrow_array_reader/INT64/Decimal128Array/dictionary encoded, optional, half NULLs                          1.00    193.9±2.13µs        ? ?/sec    1.02    197.8±1.67µs        ? ?/sec
arrow_array_reader/INT64/Decimal128Array/dictionary encoded, optional, no NULLs                            1.00    146.8±0.95µs        ? ?/sec    1.06    155.6±0.88µs        ? ?/sec
arrow_array_reader/INT64/Decimal128Array/plain encoded, mandatory, no NULLs                                1.05    107.3±1.87µs        ? ?/sec    1.00    102.0±1.09µs        ? ?/sec
arrow_array_reader/INT64/Decimal128Array/plain encoded, optional, half NULLs                               1.00    171.9±0.83µs        ? ?/sec    1.00    172.6±3.68µs        ? ?/sec
arrow_array_reader/INT64/Decimal128Array/plain encoded, optional, no NULLs                                 1.05    115.9±1.19µs        ? ?/sec    1.00    110.7±3.89µs        ? ?/sec
arrow_array_reader/Int16Array/binary packed skip, mandatory, no NULLs                                      1.02     77.8±0.62µs        ? ?/sec    1.00     76.6±0.31µs        ? ?/sec
arrow_array_reader/Int16Array/binary packed skip, optional, half NULLs                                     1.01    104.6±1.66µs        ? ?/sec    1.00    103.9±0.61µs        ? ?/sec
arrow_array_reader/Int16Array/binary packed skip, optional, no NULLs                                       1.02     80.8±0.66µs        ? ?/sec    1.00     79.3±0.85µs        ? ?/sec
arrow_array_reader/Int16Array/binary packed, mandatory, no NULLs                                           1.02    108.0±0.88µs        ? ?/sec    1.00    106.2±1.83µs        ? ?/sec
arrow_array_reader/Int16Array/binary packed, optional, half NULLs                                          1.00    176.7±1.93µs        ? ?/sec    1.00    176.8±1.25µs        ? ?/sec
arrow_array_reader/Int16Array/binary packed, optional, no NULLs                                            1.02    113.2±1.65µs        ? ?/sec    1.00    110.7±0.74µs        ? ?/sec
arrow_array_reader/Int16Array/byte_stream_split encoded, mandatory, no NULLs                               4.73    202.0±6.69µs        ? ?/sec    1.00     42.7±0.49µs        ? ?/sec
arrow_array_reader/Int16Array/byte_stream_split encoded, optional, half NULLs                              1.58    223.3±9.99µs        ? ?/sec    1.00    141.2±1.22µs        ? ?/sec
arrow_array_reader/Int16Array/byte_stream_split encoded, optional, no NULLs                                4.37    205.6±1.24µs        ? ?/sec    1.00     47.1±0.92µs        ? ?/sec
arrow_array_reader/Int16Array/dictionary encoded, mandatory, no NULLs                                      1.10    109.9±2.50µs        ? ?/sec    1.00    100.4±0.35µs        ? ?/sec
arrow_array_reader/Int16Array/dictionary encoded, optional, half NULLs                                     1.03    180.1±3.68µs        ? ?/sec    1.00    175.0±3.12µs        ? ?/sec
arrow_array_reader/Int16Array/dictionary encoded, optional, no NULLs                                       1.09    114.4±0.76µs        ? ?/sec    1.00    105.3±1.51µs        ? ?/sec
arrow_array_reader/Int16Array/plain encoded, mandatory, no NULLs                                           1.02     37.1±0.27µs        ? ?/sec    1.00     36.5±0.19µs        ? ?/sec
arrow_array_reader/Int16Array/plain encoded, optional, half NULLs                                          1.00    138.9±1.13µs        ? ?/sec    1.00    138.8±2.02µs        ? ?/sec
arrow_array_reader/Int16Array/plain encoded, optional, no NULLs                                            1.01     41.8±0.19µs        ? ?/sec    1.00     41.5±1.27µs        ? ?/sec
arrow_array_reader/Int32Array/binary packed skip, mandatory, no NULLs                                      1.01     83.4±0.49µs        ? ?/sec    1.00     82.3±0.38µs        ? ?/sec
arrow_array_reader/Int32Array/binary packed skip, optional, half NULLs                                     1.00    102.2±0.58µs        ? ?/sec    1.00    101.9±0.64µs        ? ?/sec
arrow_array_reader/Int32Array/binary packed skip, optional, no NULLs                                       1.02     86.1±1.21µs        ? ?/sec    1.00     84.7±0.86µs        ? ?/sec
arrow_array_reader/Int32Array/binary packed, mandatory, no NULLs                                           1.01    108.1±2.19µs        ? ?/sec    1.00    106.8±0.78µs        ? ?/sec
arrow_array_reader/Int32Array/binary packed, optional, half NULLs                                          1.00    166.7±0.68µs        ? ?/sec    1.00    166.1±2.82µs        ? ?/sec
arrow_array_reader/Int32Array/binary packed, optional, no NULLs                                            1.01    112.3±0.44µs        ? ?/sec    1.00    110.9±0.73µs        ? ?/sec
arrow_array_reader/Int32Array/byte_stream_split encoded, mandatory, no NULLs                               7.79    182.3±1.02µs        ? ?/sec    1.00     23.4±0.83µs        ? ?/sec
arrow_array_reader/Int32Array/byte_stream_split encoded, optional, half NULLs                              1.67    202.9±2.52µs        ? ?/sec    1.00    121.3±1.47µs        ? ?/sec
arrow_array_reader/Int32Array/byte_stream_split encoded, optional, no NULLs                                6.83    187.7±3.43µs        ? ?/sec    1.00     27.5±0.43µs        ? ?/sec
arrow_array_reader/Int32Array/dictionary encoded, mandatory, no NULLs                                      1.11     90.9±0.81µs        ? ?/sec    1.00     82.0±0.71µs        ? ?/sec
arrow_array_reader/Int32Array/dictionary encoded, optional, half NULLs                                     1.04    160.7±2.14µs        ? ?/sec    1.00    154.9±1.49µs        ? ?/sec
arrow_array_reader/Int32Array/dictionary encoded, optional, no NULLs                                       1.10     94.5±0.72µs        ? ?/sec    1.00     85.8±0.45µs        ? ?/sec
arrow_array_reader/Int32Array/plain encoded, mandatory, no NULLs                                           1.00     15.5±0.82µs        ? ?/sec    1.00     15.5±0.22µs        ? ?/sec
arrow_array_reader/Int32Array/plain encoded, optional, half NULLs                                          1.00    120.4±0.66µs        ? ?/sec    1.01    121.8±9.56µs        ? ?/sec
arrow_array_reader/Int32Array/plain encoded, optional, no NULLs                                            1.00     21.4±0.56µs        ? ?/sec    1.00     21.5±0.32µs        ? ?/sec
arrow_array_reader/Int64Array/binary packed skip, mandatory, no NULLs                                      1.00     80.0±0.76µs        ? ?/sec    1.00     79.6±0.77µs        ? ?/sec
arrow_array_reader/Int64Array/binary packed skip, optional, half NULLs                                     1.00     89.9±0.65µs        ? ?/sec    1.00     90.3±0.98µs        ? ?/sec
arrow_array_reader/Int64Array/binary packed skip, optional, no NULLs                                       1.00     82.3±0.51µs        ? ?/sec    1.00     82.2±0.68µs        ? ?/sec
arrow_array_reader/Int64Array/binary packed, mandatory, no NULLs                                           1.00    106.0±1.31µs        ? ?/sec    1.01    107.2±1.04µs        ? ?/sec
arrow_array_reader/Int64Array/binary packed, optional, half NULLs                                          1.04    149.3±4.45µs        ? ?/sec    1.00    144.0±2.88µs        ? ?/sec
arrow_array_reader/Int64Array/binary packed, optional, no NULLs                                            1.01    112.1±1.70µs        ? ?/sec    1.00    111.2±1.22µs        ? ?/sec
arrow_array_reader/Int64Array/byte_stream_split encoded, mandatory, no NULLs                               3.39    496.9±9.98µs        ? ?/sec    1.00    146.6±0.68µs        ? ?/sec
arrow_array_reader/Int64Array/byte_stream_split encoded, optional, half NULLs                              2.03    343.2±7.25µs        ? ?/sec    1.00    169.0±1.08µs        ? ?/sec
arrow_array_reader/Int64Array/byte_stream_split encoded, optional, no NULLs                                3.31   500.9±11.37µs        ? ?/sec    1.00    151.4±0.64µs        ? ?/sec
arrow_array_reader/Int64Array/dictionary encoded, mandatory, no NULLs                                      1.00     87.5±0.72µs        ? ?/sec    1.10     96.0±1.00µs        ? ?/sec
arrow_array_reader/Int64Array/dictionary encoded, optional, half NULLs                                     1.00    135.6±0.91µs        ? ?/sec    1.03    139.6±0.92µs        ? ?/sec
arrow_array_reader/Int64Array/dictionary encoded, optional, no NULLs                                       1.00     92.5±1.01µs        ? ?/sec    1.09    100.8±0.66µs        ? ?/sec
arrow_array_reader/Int64Array/plain encoded, mandatory, no NULLs                                           1.14     42.9±3.56µs        ? ?/sec    1.00     37.5±0.63µs        ? ?/sec
arrow_array_reader/Int64Array/plain encoded, optional, half NULLs                                          1.00    111.0±2.92µs        ? ?/sec    1.00    111.3±4.53µs        ? ?/sec
arrow_array_reader/Int64Array/plain encoded, optional, no NULLs                                            1.11     48.2±1.91µs        ? ?/sec    1.00     43.6±0.61µs        ? ?/sec
arrow_array_reader/Int8Array/binary packed skip, mandatory, no NULLs                                       1.02     82.0±0.62µs        ? ?/sec    1.00     80.3±0.57µs        ? ?/sec
arrow_array_reader/Int8Array/binary packed skip, optional, half NULLs                                      1.00    103.9±0.96µs        ? ?/sec    1.00    103.5±3.75µs        ? ?/sec
arrow_array_reader/Int8Array/binary packed skip, optional, no NULLs                                        1.02     84.7±0.68µs        ? ?/sec    1.00     82.9±1.46µs        ? ?/sec
arrow_array_reader/Int8Array/binary packed, mandatory, no NULLs                                            1.02    110.2±1.49µs        ? ?/sec    1.00    107.6±0.50µs        ? ?/sec
arrow_array_reader/Int8Array/binary packed, optional, half NULLs                                           1.00    172.4±0.87µs        ? ?/sec    1.00    172.1±1.02µs        ? ?/sec
arrow_array_reader/Int8Array/binary packed, optional, no NULLs                                             1.02    114.3±0.49µs        ? ?/sec    1.00    112.3±2.53µs        ? ?/sec
arrow_array_reader/Int8Array/byte_stream_split encoded, mandatory, no NULLs                                5.51    192.9±0.74µs        ? ?/sec    1.00     35.0±0.76µs        ? ?/sec
arrow_array_reader/Int8Array/byte_stream_split encoded, optional, half NULLs                               1.61    213.9±1.23µs        ? ?/sec    1.00    133.2±0.94µs        ? ?/sec
arrow_array_reader/Int8Array/byte_stream_split encoded, optional, no NULLs                                 5.01    197.6±0.86µs        ? ?/sec    1.00     39.5±1.12µs        ? ?/sec
arrow_array_reader/Int8Array/dictionary encoded, mandatory, no NULLs                                       1.10    101.7±1.08µs        ? ?/sec    1.00     92.9±0.32µs        ? ?/sec
arrow_array_reader/Int8Array/dictionary encoded, optional, half NULLs                                      1.03    171.7±1.50µs        ? ?/sec    1.00    167.4±4.14µs        ? ?/sec
arrow_array_reader/Int8Array/dictionary encoded, optional, no NULLs                                        1.09    106.3±1.03µs        ? ?/sec    1.00     97.8±0.58µs        ? ?/sec
arrow_array_reader/Int8Array/plain encoded, mandatory, no NULLs                                            1.00     28.8±0.18µs        ? ?/sec    1.00     29.0±0.41µs        ? ?/sec
arrow_array_reader/Int8Array/plain encoded, optional, half NULLs                                           1.01    131.6±2.97µs        ? ?/sec    1.00    130.4±1.02µs        ? ?/sec
arrow_array_reader/Int8Array/plain encoded, optional, no NULLs                                             1.00     33.6±0.19µs        ? ?/sec    1.00     33.5±0.25µs        ? ?/sec
arrow_array_reader/ListArray/plain encoded optional strings half NULLs                                     1.04      6.5±0.07ms        ? ?/sec    1.00      6.2±0.13ms        ? ?/sec
arrow_array_reader/ListArray/plain encoded optional strings no NULLs                                       1.13     13.8±0.21ms        ? ?/sec    1.00     12.3±0.24ms        ? ?/sec
arrow_array_reader/StringArray/dictionary encoded, mandatory, no NULLs                                     1.03   509.3±13.27µs        ? ?/sec    1.00   493.2±12.55µs        ? ?/sec
arrow_array_reader/StringArray/dictionary encoded, optional, half NULLs                                    1.02    662.7±6.09µs        ? ?/sec    1.00    647.3±8.00µs        ? ?/sec
arrow_array_reader/StringArray/dictionary encoded, optional, no NULLs                                      1.07    518.2±5.64µs        ? ?/sec    1.00   484.9±11.30µs        ? ?/sec
arrow_array_reader/StringArray/plain encoded, mandatory, no NULLs                                          1.03   662.3±17.31µs        ? ?/sec    1.00   644.6±15.84µs        ? ?/sec
arrow_array_reader/StringArray/plain encoded, optional, half NULLs                                         1.01   769.7±10.72µs        ? ?/sec    1.00    763.8±9.88µs        ? ?/sec
arrow_array_reader/StringArray/plain encoded, optional, no NULLs                                           1.02    663.2±7.12µs        ? ?/sec    1.00    653.0±8.38µs        ? ?/sec
arrow_array_reader/StringDictionary/dictionary encoded, mandatory, no NULLs                                1.00    329.8±4.51µs        ? ?/sec    1.00    329.0±4.77µs        ? ?/sec
arrow_array_reader/StringDictionary/dictionary encoded, optional, half NULLs                               1.00    375.5±5.30µs        ? ?/sec    1.00    377.1±4.23µs        ? ?/sec
arrow_array_reader/StringDictionary/dictionary encoded, optional, no NULLs                                 1.04   347.2±52.94µs        ? ?/sec    1.00    333.8±2.36µs        ? ?/sec
arrow_array_reader/StringViewArray/dictionary encoded, mandatory, no NULLs                                 1.07    151.5±2.57µs        ? ?/sec    1.00    141.6±2.02µs        ? ?/sec
arrow_array_reader/StringViewArray/dictionary encoded, optional, half NULLs                                1.04    210.4±1.77µs        ? ?/sec    1.00    202.0±4.16µs        ? ?/sec
arrow_array_reader/StringViewArray/dictionary encoded, optional, no NULLs                                  1.07    147.4±2.42µs        ? ?/sec    1.00    137.4±2.40µs        ? ?/sec
arrow_array_reader/StringViewArray/plain encoded, mandatory, no NULLs                                      1.05    382.1±5.61µs        ? ?/sec    1.00    363.0±8.12µs        ? ?/sec
arrow_array_reader/StringViewArray/plain encoded, optional, half NULLs                                     1.05    336.9±4.44µs        ? ?/sec    1.00    320.3±2.89µs        ? ?/sec
arrow_array_reader/StringViewArray/plain encoded, optional, no NULLs                                       1.06    392.9±6.80µs        ? ?/sec    1.00    372.4±4.19µs        ? ?/sec
arrow_array_reader/UInt16Array/binary packed skip, mandatory, no NULLs                                     1.02     93.5±0.68µs        ? ?/sec    1.00     91.6±0.62µs        ? ?/sec
arrow_array_reader/UInt16Array/binary packed skip, optional, half NULLs                                    1.01    113.4±2.63µs        ? ?/sec    1.00    111.8±1.24µs        ? ?/sec
arrow_array_reader/UInt16Array/binary packed skip, optional, no NULLs                                      1.02     95.8±2.10µs        ? ?/sec    1.00     94.1±0.62µs        ? ?/sec
arrow_array_reader/UInt16Array/binary packed, mandatory, no NULLs                                          1.02    127.5±0.59µs        ? ?/sec    1.00    125.4±2.32µs        ? ?/sec
arrow_array_reader/UInt16Array/binary packed, optional, half NULLs                                         1.00    187.0±0.94µs        ? ?/sec    1.00    187.6±5.75µs        ? ?/sec
arrow_array_reader/UInt16Array/binary packed, optional, no NULLs                                           1.02    132.7±2.40µs        ? ?/sec    1.00    129.7±0.62µs        ? ?/sec
arrow_array_reader/UInt16Array/byte_stream_split encoded, mandatory, no NULLs                              4.93    201.0±1.17µs        ? ?/sec    1.00     40.8±0.18µs        ? ?/sec
arrow_array_reader/UInt16Array/byte_stream_split encoded, optional, half NULLs                             1.58    222.2±1.41µs        ? ?/sec    1.00    140.7±0.89µs        ? ?/sec
arrow_array_reader/UInt16Array/byte_stream_split encoded, optional, no NULLs                               4.54    205.4±1.25µs        ? ?/sec    1.00     45.2±0.31µs        ? ?/sec
arrow_array_reader/UInt16Array/dictionary encoded, mandatory, no NULLs                                     1.09    109.3±0.75µs        ? ?/sec    1.00    100.4±0.39µs        ? ?/sec
arrow_array_reader/UInt16Array/dictionary encoded, optional, half NULLs                                    1.03    179.7±2.50µs        ? ?/sec    1.00    174.0±2.26µs        ? ?/sec
arrow_array_reader/UInt16Array/dictionary encoded, optional, no NULLs                                      1.08    114.0±1.11µs        ? ?/sec    1.00    105.2±0.56µs        ? ?/sec
arrow_array_reader/UInt16Array/plain encoded, mandatory, no NULLs                                          1.01     36.8±0.18µs        ? ?/sec    1.00     36.6±0.14µs        ? ?/sec
arrow_array_reader/UInt16Array/plain encoded, optional, half NULLs                                         1.00    139.3±1.39µs        ? ?/sec    1.00    139.3±0.91µs        ? ?/sec
arrow_array_reader/UInt16Array/plain encoded, optional, no NULLs                                           1.01     41.8±0.21µs        ? ?/sec    1.00     41.3±0.48µs        ? ?/sec
arrow_array_reader/UInt32Array/binary packed skip, mandatory, no NULLs                                     1.02     83.7±0.67µs        ? ?/sec    1.00     82.4±0.74µs        ? ?/sec
arrow_array_reader/UInt32Array/binary packed skip, optional, half NULLs                                    1.01    103.1±1.63µs        ? ?/sec    1.00    101.9±0.79µs        ? ?/sec
arrow_array_reader/UInt32Array/binary packed skip, optional, no NULLs                                      1.02     86.5±0.88µs        ? ?/sec    1.00     84.7±0.31µs        ? ?/sec
arrow_array_reader/UInt32Array/binary packed, mandatory, no NULLs                                          1.01    108.2±1.43µs        ? ?/sec    1.00    106.8±0.99µs        ? ?/sec
arrow_array_reader/UInt32Array/binary packed, optional, half NULLs                                         1.01    167.6±2.22µs        ? ?/sec    1.00    166.7±1.47µs        ? ?/sec
arrow_array_reader/UInt32Array/binary packed, optional, no NULLs                                           1.01    112.9±1.56µs        ? ?/sec    1.00    111.6±1.56µs        ? ?/sec
arrow_array_reader/UInt32Array/byte_stream_split encoded, mandatory, no NULLs                              7.52    183.0±2.92µs        ? ?/sec    1.00     24.3±0.28µs        ? ?/sec
arrow_array_reader/UInt32Array/byte_stream_split encoded, optional, half NULLs                             1.66    202.3±1.21µs        ? ?/sec    1.00    121.6±0.91µs        ? ?/sec
arrow_array_reader/UInt32Array/byte_stream_split encoded, optional, no NULLs                               6.50    187.6±1.98µs        ? ?/sec    1.00     28.9±0.50µs        ? ?/sec
arrow_array_reader/UInt32Array/dictionary encoded, mandatory, no NULLs                                     1.10     91.2±0.80µs        ? ?/sec    1.00     83.0±2.19µs        ? ?/sec
arrow_array_reader/UInt32Array/dictionary encoded, optional, half NULLs                                    1.03    160.6±2.32µs        ? ?/sec    1.00    155.9±1.12µs        ? ?/sec
arrow_array_reader/UInt32Array/dictionary encoded, optional, no NULLs                                      1.10     96.0±1.59µs        ? ?/sec    1.00     87.2±1.40µs        ? ?/sec
arrow_array_reader/UInt32Array/plain encoded, mandatory, no NULLs                                          1.07     19.8±1.63µs        ? ?/sec    1.00     18.5±0.46µs        ? ?/sec
arrow_array_reader/UInt32Array/plain encoded, optional, half NULLs                                         1.00    119.5±1.64µs        ? ?/sec    1.00    119.8±1.10µs        ? ?/sec
arrow_array_reader/UInt32Array/plain encoded, optional, no NULLs                                           1.07     25.1±1.68µs        ? ?/sec    1.00     23.4±0.59µs        ? ?/sec
arrow_array_reader/UInt64Array/binary packed skip, mandatory, no NULLs                                     1.00     80.1±0.54µs        ? ?/sec    1.00     80.1±1.70µs        ? ?/sec
arrow_array_reader/UInt64Array/binary packed skip, optional, half NULLs                                    1.00     89.8±0.68µs        ? ?/sec    1.01     91.1±0.98µs        ? ?/sec
arrow_array_reader/UInt64Array/binary packed skip, optional, no NULLs                                      1.00     82.9±1.43µs        ? ?/sec    1.00     82.6±0.69µs        ? ?/sec
arrow_array_reader/UInt64Array/binary packed, mandatory, no NULLs                                          1.00    106.0±0.97µs        ? ?/sec    1.01    107.5±0.79µs        ? ?/sec
arrow_array_reader/UInt64Array/binary packed, optional, half NULLs                                         1.03    149.1±0.70µs        ? ?/sec    1.00    145.4±2.02µs        ? ?/sec
arrow_array_reader/UInt64Array/binary packed, optional, no NULLs                                           1.01    111.2±0.71µs        ? ?/sec    1.00    110.1±0.63µs        ? ?/sec
arrow_array_reader/UInt64Array/byte_stream_split encoded, mandatory, no NULLs                              3.38    495.3±2.00µs        ? ?/sec    1.00    146.6±0.74µs        ? ?/sec
arrow_array_reader/UInt64Array/byte_stream_split encoded, optional, half NULLs                             2.03    343.2±5.25µs        ? ?/sec    1.00   169.1±10.41µs        ? ?/sec
arrow_array_reader/UInt64Array/byte_stream_split encoded, optional, no NULLs                               3.32    500.5±2.54µs        ? ?/sec    1.00    150.8±0.53µs        ? ?/sec
arrow_array_reader/UInt64Array/dictionary encoded, mandatory, no NULLs                                     1.00     88.1±0.70µs        ? ?/sec    1.10     96.6±1.52µs        ? ?/sec
arrow_array_reader/UInt64Array/dictionary encoded, optional, half NULLs                                    1.00    135.8±1.35µs        ? ?/sec    1.03    140.2±1.24µs        ? ?/sec
arrow_array_reader/UInt64Array/dictionary encoded, optional, no NULLs                                      1.00     92.5±0.85µs        ? ?/sec    1.10    101.4±2.89µs        ? ?/sec
arrow_array_reader/UInt64Array/plain encoded, mandatory, no NULLs                                          1.05     45.7±3.37µs        ? ?/sec    1.00     43.5±1.61µs        ? ?/sec
arrow_array_reader/UInt64Array/plain encoded, optional, half NULLs                                         1.00    110.7±1.09µs        ? ?/sec    1.02    112.6±0.99µs        ? ?/sec
arrow_array_reader/UInt64Array/plain encoded, optional, no NULLs                                           1.05     51.8±3.23µs        ? ?/sec    1.00     49.3±1.44µs        ? ?/sec
arrow_array_reader/UInt8Array/binary packed skip, mandatory, no NULLs                                      1.01     88.7±0.25µs        ? ?/sec    1.00     87.8±0.49µs        ? ?/sec
arrow_array_reader/UInt8Array/binary packed skip, optional, half NULLs                                     1.00    107.2±0.64µs        ? ?/sec    1.00    107.0±0.50µs        ? ?/sec
arrow_array_reader/UInt8Array/binary packed skip, optional, no NULLs                                       1.01     91.5±0.39µs        ? ?/sec    1.00     90.8±2.47µs        ? ?/sec
arrow_array_reader/UInt8Array/binary packed, mandatory, no NULLs                                           1.01    119.2±0.47µs        ? ?/sec    1.00    117.6±0.70µs        ? ?/sec
arrow_array_reader/UInt8Array/binary packed, optional, half NULLs                                          1.00    177.9±3.14µs        ? ?/sec    1.00    177.4±3.05µs        ? ?/sec
arrow_array_reader/UInt8Array/binary packed, optional, no NULLs                                            1.01    123.6±1.26µs        ? ?/sec    1.00    122.5±2.91µs        ? ?/sec
arrow_array_reader/UInt8Array/byte_stream_split encoded, mandatory, no NULLs                               5.54    193.6±2.79µs        ? ?/sec    1.00     34.9±0.26µs        ? ?/sec
arrow_array_reader/UInt8Array/byte_stream_split encoded, optional, half NULLs                              1.61    214.6±5.28µs        ? ?/sec    1.00    133.3±1.71µs        ? ?/sec
arrow_array_reader/UInt8Array/byte_stream_split encoded, optional, no NULLs                                5.07    197.7±1.00µs        ? ?/sec    1.00     39.0±0.27µs        ? ?/sec
arrow_array_reader/UInt8Array/dictionary encoded, mandatory, no NULLs                                      1.09    101.9±0.87µs        ? ?/sec    1.00     93.3±0.64µs        ? ?/sec
arrow_array_reader/UInt8Array/dictionary encoded, optional, half NULLs                                     1.03    172.4±5.62µs        ? ?/sec    1.00    167.6±1.83µs        ? ?/sec
arrow_array_reader/UInt8Array/dictionary encoded, optional, no NULLs                                       1.09    106.3±0.59µs        ? ?/sec    1.00     98.0±0.85µs        ? ?/sec
arrow_array_reader/UInt8Array/plain encoded, mandatory, no NULLs                                           1.00     28.6±0.34µs        ? ?/sec    1.01     28.8±0.73µs        ? ?/sec
arrow_array_reader/UInt8Array/plain encoded, optional, half NULLs                                          1.00    131.6±2.03µs        ? ?/sec    1.00    131.0±0.88µs        ? ?/sec
arrow_array_reader/UInt8Array/plain encoded, optional, no NULLs                                            1.00     33.5±0.71µs        ? ?/sec    1.00     33.7±0.17µs        ? ?/sec
arrow_array_reader/struct/Int32Array/plain encoded, mandatory struct, optional data, half NULLs            1.00    120.7±1.98µs        ? ?/sec    1.00    120.7±0.76µs        ? ?/sec
arrow_array_reader/struct/Int32Array/plain encoded, mandatory struct, optional data, no NULLs              1.01     23.2±0.61µs        ? ?/sec    1.00     22.9±0.93µs        ? ?/sec
arrow_array_reader/struct/Int32Array/plain encoded, optional struct, optional data, half NULLs             1.00    241.6±2.99µs        ? ?/sec    1.02    246.3±3.72µs        ? ?/sec
arrow_array_reader/struct/Int32Array/plain encoded, optional struct, optional data, no NULLs               1.00    120.6±2.68µs        ? ?/sec    1.03    124.5±2.15µs        ? ?/sec

[lyang24]

not helpful across the board

[jhorstmann]

Last time I looked, the compiler was surprisingly good at auto-vectorizing that code. So the manual loop-unrolling should not be needed, and the additional code maybe throws off some optimization heuristic.