Apache Arrow Go 18.8.0

[github-actions[bot]]

What's Changed

• feat(parquet/schema): Wire up logical type functionality for writing geo types by @C-Loftus in feat(parquet/schema): Wire up logical type functionality for writing geo types #960
• chore: Bump actions/setup-go from 6.5.0 to 7.0.0 by @dependabot[bot] in chore: Bump actions/setup-go from 6.5.0 to 7.0.0 #961
• chore: Bump actions/checkout from 7.0.0 to 7.0.1 by @dependabot[bot] in chore: Bump actions/checkout from 7.0.0 to 7.0.1 #963
• chore(dev/release): filter out non numeric go releases (rc's and beta's) when finding latest go by @raulcd in chore(dev/release): filter out non numeric go releases (rc's and beta's) when finding latest go #968
• ci: fix release workflow discussions by @zeroshade in ci: fix release workflow discussions #980
• chore: Bump modernc.org/sqlite from 1.53.0 to 1.54.0 by @dependabot[bot] in chore: Bump modernc.org/sqlite from 1.53.0 to 1.54.0 #964
• chore: Bump google.golang.org/grpc from 1.82.0 to 1.82.1 by @dependabot[bot] in chore: Bump google.golang.org/grpc from 1.82.0 to 1.82.1 #965
• chore: Bump github.com/klauspost/compress from 1.19.0 to 1.19.1 by @dependabot[bot] in chore: Bump github.com/klauspost/compress from 1.19.0 to 1.19.1 #966
• chore: Bump actions/setup-python from 6.3.0 to 7.0.0 by @dependabot[bot] in chore: Bump actions/setup-python from 6.3.0 to 7.0.0 #962
• ci: fix lint on release workflow by @zeroshade in ci: fix lint on release workflow #982
• fix: remove superfluous struct field index compare by @Willem-J-an in fix: remove superfluous struct field index compare #970
• fix(ipc): reject truncated message frames by @fallintoplace in fix(ipc): reject truncated message frames #991
• fix(internal/utils): handle reader reset and short peeks by @fallintoplace in fix(internal/utils): handle reader reset and short peeks #987
• fix(arrow/ipc): propagate CLI stream errors by @fallintoplace in fix(arrow/ipc): propagate CLI stream errors #986
• fix(arrow/flight): avoid blocking after stream cancellation by @fallintoplace in fix(arrow/flight): avoid blocking after stream cancellation #989
• fix(parquet/variant): handle empty metadata keys by @fallintoplace in fix(parquet/variant): handle empty metadata keys #985
• fix(flight/flightsql/example): propagate row iteration errors by @fallintoplace in fix(flight/flightsql/example): propagate row iteration errors #990
• fix(parquet): reject truncated scalar bit reads by @fallintoplace in fix(parquet): reject truncated scalar bit reads #944
• fix: Fix arm64 assembly stack frame manipulation by @nsrip-dd in fix: Fix arm64 assembly stack frame manipulation #984
• fix(arrow/array): support zero-column tables by @fallintoplace in fix(arrow/array): support zero-column tables #988
• docs(parquet): Add thrift generation contribution info by @C-Loftus in docs(parquet): Add thrift generation contribution info #971
• fix(array): reject mismatched known null counts during full validation by @fallintoplace in fix(array): reject mismatched known null counts during full validation #993
• fix(arrow/array): Resize(0) on non-empty builder causes memory leak by @serramatutu in fix(arrow/array): Resize(0) on non-empty builder causes memory leak #995
• feat(parquet/pqarrow): Read support for extension types / geo by @C-Loftus in feat(parquet/pqarrow): Read support for extension types / geo #969
• fix(array): validate run-end encoded invariants by @fallintoplace in fix(array): validate run-end encoded invariants #992
• fix(array): validate binary_view and string_view payloads by @fallintoplace in fix(array): validate binary_view and string_view payloads #994
• fix(parquet): wrap DELTA_BINARY_PACKED deltas at the physical type width by @tom-eon in fix(parquet): wrap DELTA_BINARY_PACKED deltas at the physical type width #1027
• fix(arrow/compute): use is_null/is_not_null in expressions by @fallintoplace in fix(arrow/compute): use is_null/is_not_null in expressions #997
• fix(arrow/ipc): reject NaN minSpaceSavings values by @fallintoplace in fix(arrow/ipc): reject NaN minSpaceSavings values #998
• fix(arrow/tensor): default nil dimension names to empty strings by @fallintoplace in fix(arrow/tensor): default nil dimension names to empty strings #999
• fix(arrow/scalar): allow null extension scalars to keep null storage by @fallintoplace in fix(arrow/scalar): allow null extension scalars to keep null storage #1000
• fix(arrow/scalar): handle nil child values in run-end scalars by @fallintoplace in fix(arrow/scalar): handle nil child values in run-end scalars #1001
• fix(arrow/array): ignore zero-count union appends by @fallintoplace in fix(arrow/array): ignore zero-count union appends #1026
• fix(arrow/csv): preserve decimal precision by @fallintoplace in fix(arrow/csv): preserve decimal precision #1022
• fix(parquet/metadata): compare bloom filter caps in bits by @fallintoplace in fix(parquet/metadata): compare bloom filter caps in bits #996
• fix(parquet/pqarrow): reject timestamp coercion overflow by @fallintoplace in fix(parquet/pqarrow): reject timestamp coercion overflow #1008
• fix(parquet/file): reject records starting with nonzero repetition levels by @fallintoplace in fix(parquet/file): reject records starting with nonzero repetition levels #1009
• fix(parquet/variant): reject truncated scalar values by @fallintoplace in fix(parquet/variant): reject truncated scalar values #1017
• fix(parquet/encoding): reject invalid delta byte prefixes by @fallintoplace in fix(parquet/encoding): reject invalid delta byte prefixes #1014
• fix(parquet/metadata): preserve infinite floating-point statistics by @fallintoplace in fix(parquet/metadata): preserve infinite floating-point statistics #1007
• fix(parquet/encoding): bound lazy dictionary growth by @fallintoplace in fix(parquet/encoding): bound lazy dictionary growth #1011
• fix(parquet/encoding): respect declared RLE payload lengths by @fallintoplace in fix(parquet/encoding): respect declared RLE payload lengths #1015
• fix(parquet/encoding): validate delta binary packed headers by @fallintoplace in fix(parquet/encoding): validate delta binary packed headers #1012
• fix(parquet/encoding): reject invalid delta length payloads by @fallintoplace in fix(parquet/encoding): reject invalid delta length payloads #1013
• fix(parquet/pqarrow): preserve INT96 timestamps across units by @fallintoplace in fix(parquet/pqarrow): preserve INT96 timestamps across units #1006
• fix(parquet/pqarrow): Change interface name for arrow type on read for extension types by @C-Loftus in fix(parquet/pqarrow): Change interface name for arrow type on read for extension types #1028
• chore: Bump docker/login-action from 4.4.0 to 4.5.1 by @dependabot[bot] in chore: Bump docker/login-action from 4.4.0 to 4.5.1 #1030
• fix(arrow/csv): respect timestamp timezones by @fallintoplace in fix(arrow/csv): respect timestamp timezones #1025
• fix(parquet/compress): avoid closing caller streams for uncompressed wrappers by @fallintoplace in fix(parquet/compress): avoid closing caller streams for uncompressed wrappers #1003
• fix(parquet/compress): allocate uncompressed output for short destinations by @fallintoplace in fix(parquet/compress): allocate uncompressed output for short destinations #1004
• fix(internal/utils): keep byteReader position after failed seeks by @fallintoplace in fix(internal/utils): keep byteReader position after failed seeks #1005
• fix(parquet/file): count repeated skips in logical rows by @fallintoplace in fix(parquet/file): count repeated skips in logical rows #1002
• fix(parquet/metadata): reject malformed fixed-width statistics by @fallintoplace in fix(parquet/metadata): reject malformed fixed-width statistics #1010
• fix(arrow/array): skip empty table chunks by @fallintoplace in fix(arrow/array): skip empty table chunks #1023
• fix(arrow/scalar): reject negative array lengths by @fallintoplace in fix(arrow/scalar): reject negative array lengths #1024
• fix(arrow/array): validate fixed-size list child lengths by @fallintoplace in fix(arrow/array): validate fixed-size list child lengths #1019
• fix(parquet/pqarrow): calculate large decimal widths by @fallintoplace in fix(parquet/pqarrow): calculate large decimal widths #1034
• fix(parquet/schema): reject invalid fixed lengths by @fallintoplace in fix(parquet/schema): reject invalid fixed lengths #1035
• fix(parquet/encoding): reject truncated plain boolean data by @fallintoplace in fix(parquet/encoding): reject truncated plain boolean data #1031
• fix(parquet/metadata): return page index decode errors by @fallintoplace in fix(parquet/metadata): return page index decode errors #1032
• fix(parquet/metadata): validate adaptive bloom options by @fallintoplace in fix(parquet/metadata): validate adaptive bloom options #1033
• fix(tmpl): report generated output write errors by @fallintoplace in fix(tmpl): report generated output write errors #1037
• fix(arrow/array): propagate embedded pointer nullability by @fallintoplace in fix(arrow/array): propagate embedded pointer nullability #1038
• fix(flight/flightsql/driver): convert Date32 values by @fallintoplace in fix(flight/flightsql/driver): convert Date32 values #1039
• fix(arrow/scalar): support same-width time casts by @fallintoplace in fix(arrow/scalar): support same-width time casts #1040
• fix(arrow/scalar): use duration type for units by @fallintoplace in fix(arrow/scalar): use duration type for units #1041
• fix(arrow/array): validate union child upper bound by @fallintoplace in fix(arrow/array): validate union child upper bound #1043
• chore: Bump google.golang.org/grpc from 1.82.1 to 1.83.0 by @dependabot[bot] in chore: Bump google.golang.org/grpc from 1.82.1 to 1.83.0 #1092
• chore: Bump docker/login-action from 4.5.1 to 4.6.0 by @dependabot[bot] in chore: Bump docker/login-action from 4.5.1 to 4.6.0 #1093
• refactor(arrow/avro): migrate from hamba/avro to twmb/avro by @prochac in refactor(arrow/avro): migrate from hamba/avro to twmb/avro #830
• fix(arrow/array): validate struct child lengths by @fallintoplace in fix(arrow/array): validate struct child lengths #1021
• fix(arrow/cdata): validate imported schema topology by @fallintoplace in fix(arrow/cdata): validate imported schema topology #1046
• fix(parquet/metadata): validate row-group ordinals in PageIndexReader.WillNeed by @fallintoplace in fix(parquet/metadata): validate row-group ordinals in PageIndexReader.WillNeed #1057
• fix(arrow/flight): limit descriptors to next payload by @fallintoplace in fix(arrow/flight): limit descriptors to next payload #1042
• fix(parquet): propagate nonce generation failures by @fallintoplace in fix(parquet): propagate nonce generation failures #1045
• fix(parquet/metadata): reject negative row-group indices in Subset by @fallintoplace in fix(parquet/metadata): reject negative row-group indices in Subset #1055
• fix(parquet/metadata): use default writer properties when props is nil by @fallintoplace in fix(parquet/metadata): use default writer properties when props is nil #1058
• fix(parquet/metadata): return schema initialization errors by @fallintoplace in fix(parquet/metadata): return schema initialization errors #1056
• fix(parquet/pqarrow): propagate level builder errors by @fallintoplace in fix(parquet/pqarrow): propagate level builder errors #1049
• fix(arrow/flight/session): clone option values by @fallintoplace in fix(arrow/flight/session): clone option values #1044
• fix(parquet/variant): validate metadata offset tables by @fallintoplace in fix(parquet/variant): validate metadata offset tables #1063
• fix(arrow/csv): release Reader record builders by @fallintoplace in fix(arrow/csv): release Reader record builders #1059
• fix(parquet/pqarrow): release partially constructed field readers by @fallintoplace in fix(parquet/pqarrow): release partially constructed field readers #1061
• fix(arrow/array): make dictionary suffix null indexes relative by @fallintoplace in fix(arrow/array): make dictionary suffix null indexes relative #1064
• fix(parquet/pqarrow): prefer stored schema extension type on read by @twuebi in fix(parquet/pqarrow): prefer stored schema extension type on read #1051
• fix(arrow/flight/flightsql): avoid double-retaining parameter readers by @fallintoplace in fix(arrow/flight/flightsql): avoid double-retaining parameter readers #1066
• fix(flightsql): isolate example SQLite databases by @fallintoplace in fix(flightsql): isolate example SQLite databases #1067
• fix(arrow): return an owned slice from Schema.FieldsByName by @fallintoplace in fix(arrow): return an owned slice from Schema.FieldsByName #1069
• fix(arrow/ipc): return errors for malformed file record batches by @fallintoplace in fix(arrow/ipc): return errors for malformed file record batches #1062
• fix(flight): return Unauthenticated for invalid unary auth tokens by @fallintoplace in fix(flight): return Unauthenticated for invalid unary auth tokens #1070
• fix(arrow/array): always release JSON reader builder by @fallintoplace in fix(arrow/array): always release JSON reader builder #1065
• fix(flight): accept case-insensitive authorization schemes by @fallintoplace in fix(flight): accept case-insensitive authorization schemes #1071
• fix(parquet): reject short page writes by @fallintoplace in fix(parquet): reject short page writes #1050
• fix(parquet/file): close file on OpenParquetFile errors by @fallintoplace in fix(parquet/file): close file on OpenParquetFile errors #1053
• fix(parquet): reject truncated footer signatures by @fallintoplace in fix(parquet): reject truncated footer signatures #1074
• fix(arrow/ipc): reject non-dictionary initial messages by @fallintoplace in fix(arrow/ipc): reject non-dictionary initial messages #1054
• feat(flightsql): allow configuring the driver memory allocator by @fallintoplace in feat(flightsql): allow configuring the driver memory allocator #1076
• fix(arrow): avoid retains on invalid chunk construction by @fallintoplace in fix(arrow): avoid retains on invalid chunk construction #1078
• fix(parquet): row-align V1 pages with offset indexes by @fallintoplace in fix(parquet): row-align V1 pages with offset indexes #1075
• fix(arrow/array): clean up partial table construction by @fallintoplace in fix(arrow/array): clean up partial table construction #1077
• fix(parquet_reader): propagate batch read errors by @fallintoplace in fix(parquet_reader): propagate batch read errors #1081
• fix(arrow): isolate schemas created by AddField by @fallintoplace in fix(arrow): isolate schemas created by AddField #1079
• fix(arrow/util): avoid protobuf map schema goroutines by @fallintoplace in fix(arrow/util): avoid protobuf map schema goroutines #1082
• fix(array): format date arrays in String by @fallintoplace in fix(array): format date arrays in String #1068
• fix(parquet): report initialized row-group columns correctly by @fallintoplace in fix(parquet): report initialized row-group columns correctly #1072
• fix(parquet): serialize column chunk file offsets by @fallintoplace in fix(parquet): serialize column chunk file offsets #1073
• fix(arrow/util): release protobuf record builders by @fallintoplace in fix(arrow/util): release protobuf record builders #1084
• fix(arrow/util): release protobuf enum dictionaries by @fallintoplace in fix(arrow/util): release protobuf enum dictionaries #1083
• fix(arrow/avro): stop reader workers before reuse by @fallintoplace in fix(arrow/avro): stop reader workers before reuse #1048
• fix(parquet): restore Arrow duration units from schema metadata by @fallintoplace in fix(parquet): restore Arrow duration units from schema metadata #1095
• fix(compute): round negative HalfToOdd ties correctly by @fallintoplace in fix(compute): round negative HalfToOdd ties correctly #1094
• perf(arrow/bitutil): add SetBitSwap/ClearBitSwap, use in builder.SetNull by @tom-eon in perf(arrow/bitutil): add SetBitSwap/ClearBitSwap, use in builder.SetNull #1090
• chore: Bump modernc.org/sqlite from 1.54.0 to 1.55.0 by @dependabot[bot] in chore: Bump modernc.org/sqlite from 1.54.0 to 1.55.0 #1091
• fix(array): compare sparse union slices at relative offsets by @fallintoplace in fix(array): compare sparse union slices at relative offsets #1096
• fix(arrow/ipc): make writer failures terminal by @fallintoplace in fix(arrow/ipc): make writer failures terminal #1047
• feat(parquet): add WithDictionaryCostFallback writer property by @tom-eon in feat(parquet): add WithDictionaryCostFallback writer property #1036
• fix(parquet/pqarrow): release extension arrays after chunking by @fallintoplace in fix(parquet/pqarrow): release extension arrays after chunking #1060
• fix(parquet/encoding): read ahead in the streaming value buffer by @joechenrh in fix(parquet/encoding): read ahead in the streaming value buffer #937
• fix(parquet/variant): preserve metadata keys when cloning by @fallintoplace in fix(parquet/variant): preserve metadata keys when cloning #1080
• test(arrow/util): cover zero-field protobuf records by @fallintoplace in test(arrow/util): cover zero-field protobuf records #1085
• fix(arrow/scalar): preserve metadata in FromScalar by @fallintoplace in fix(arrow/scalar): preserve metadata in FromScalar #1100
• fix(arrow/scalar): preserve binary cast target types by @fallintoplace in fix(arrow/scalar): preserve binary cast target types #1101
• fix(arrow/scalar): support boolean identity casts by @fallintoplace in fix(arrow/scalar): support boolean identity casts #1102
• fix(arrow/util): handle invalid protobuf Any values by @fallintoplace in fix(arrow/util): handle invalid protobuf Any values #1086
• fix(arrow/array): avoid copying initialized extension refcounts by @fallintoplace in fix(arrow/array): avoid copying initialized extension refcounts #1089
• fix(parquet): preserve dictionary index types by @fallintoplace in fix(parquet): preserve dictionary index types #1098
• fix(parquet_reader): propagate statistics decoding errors by @fallintoplace in fix(parquet_reader): propagate statistics decoding errors #1087
• fix(arrow/array): avoid panics cleaning partial tables by @fallintoplace in fix(arrow/array): avoid panics cleaning partial tables #1103
• fix(arrow/ipc): release reader state on schema errors by @fallintoplace in fix(arrow/ipc): release reader state on schema errors #1112
• fix(arrow/scalar): format extension values from storage scalars by @fallintoplace in fix(arrow/scalar): format extension values from storage scalars #1111
• fix(arrow/array): isolate table columns from caller slices by @fallintoplace in fix(arrow/array): isolate table columns from caller slices #1105
• fix(arrow/scalar): preserve exact decimal string casts by @fallintoplace in fix(arrow/scalar): preserve exact decimal string casts #1107
• fix(arrow/tensor): validate typed constructor data by @fallintoplace in fix(arrow/tensor): validate typed constructor data #1104
• fix(arrow/scalar): format interval scalars in String by @fallintoplace in fix(arrow/scalar): format interval scalars in String #1115
• fix(arrow/array): preserve rows for empty-schema JSON input by @fallintoplace in fix(arrow/array): preserve rows for empty-schema JSON input #1117
• fix(arrow/util): return protobuf record conversion errors by @fallintoplace in fix(arrow/util): return protobuf record conversion errors #1088
• fix(arrow/scalar): validate FromScalar destinations by @fallintoplace in fix(arrow/scalar): validate FromScalar destinations #1106
• fix(arrow/scalar): use wide offsets for large binary scalars by @fallintoplace in fix(arrow/scalar): use wide offsets for large binary scalars #1108
• fix(arrow/scalar): apply timestamp zones when casting to dates by @fallintoplace in fix(arrow/scalar): apply timestamp zones when casting to dates #1110
• fix(arrow/ipc): validate file block framing and lengths by @fallintoplace in fix(arrow/ipc): validate file block framing and lengths #1097
• fix(arrow/scalar): release partial children after failed struct extraction by @fallintoplace in fix(arrow/scalar): release partial children after failed struct extraction #1118
• fix(arrow/ipc): return errors for invalid record indexes by @fallintoplace in fix(arrow/ipc): return errors for invalid record indexes #1120
• fix(parquet): read null fixed-size-list parents by @fallintoplace in fix(parquet): read null fixed-size-list parents #1099
• fix(arrow/scalar): release partial children after failed struct conversion by @fallintoplace in fix(arrow/scalar): release partial children after failed struct conversion #1114
• chore: Bump github.com/klauspost/compress from 1.19.1 to 1.19.2 by @dependabot[bot] in chore: Bump github.com/klauspost/compress from 1.19.1 to 1.19.2 #1166
• chore: Bump github.com/pierrec/lz4/v4 from 4.1.27 to 4.1.28 by @dependabot[bot] in chore: Bump github.com/pierrec/lz4/v4 from 4.1.27 to 4.1.28 #1168
• chore: Bump modernc.org/sqlite from 1.55.0 to 1.56.0 by @dependabot[bot] in chore: Bump modernc.org/sqlite from 1.55.0 to 1.56.0 #1165
• chore: Bump github.com/twmb/avro from 1.5.0 to 1.7.2 by @dependabot[bot] in chore: Bump github.com/twmb/avro from 1.5.0 to 1.7.2 #1167
• fix(parquet/metadata): return error for missing column metadata by @fallintoplace in fix(parquet/metadata): return error for missing column metadata #1137
• fix(arrow/array): widen dictionary index bounds checks by @fallintoplace in fix(arrow/array): widen dictionary index bounds checks #1129
• fix(compute/exprs): avoid empty field reference panic by @fallintoplace in fix(compute/exprs): avoid empty field reference panic #1130
• fix(compute): handle odd temporal multiples by @fallintoplace in fix(compute): handle odd temporal multiples #1119
• fix(arrow/csv): preserve first row during headerless inference by @fallintoplace in fix(arrow/csv): preserve first row during headerless inference #1145
• fix(arrow/array): bound table reader batches by @fallintoplace in fix(arrow/array): bound table reader batches #1146
• fix(arrow/extensions): compare JSON storage types semantically by @fallintoplace in fix(arrow/extensions): compare JSON storage types semantically #1150
• fix(arrow/ipc): preserve minSpaceSavings in stream writer by @fallintoplace in fix(arrow/ipc): preserve minSpaceSavings in stream writer #1143
• fix(parquet/compress): honor LZ4 raw destination contract by @fallintoplace in fix(parquet/compress): honor LZ4 raw destination contract #1147
• fix(arrow): reject duplicate union type codes by @fallintoplace in fix(arrow): reject duplicate union type codes #1131
• fix(parquet/metadata): handle invalid row-group column indexes by @fallintoplace in fix(parquet/metadata): handle invalid row-group column indexes #1125
• fix(compute/exec): handle missing validity buffers by @fallintoplace in fix(compute/exec): handle missing validity buffers #1136
• fix(compute/exec): report ListView buffer counts by @fallintoplace in fix(compute/exec): report ListView buffer counts #1133
• fix(arrow): include KeysSorted in map equality by @fallintoplace in fix(arrow): include KeysSorted in map equality #1134
• fix(compute): reject non-array-like filters by @fallintoplace in fix(compute): reject non-array-like filters #1135
• fix(memory): handle zero-length checked reallocations by @fallintoplace in fix(memory): handle zero-length checked reallocations #1132
• fix(parquet/pqarrow): validate reader indexes by @fallintoplace in fix(parquet/pqarrow): validate reader indexes #1124
• fix(arrow/csv): validate custom converter row counts by @fallintoplace in fix(arrow/csv): validate custom converter row counts #1148
• fix(parquet/encoding): honor BufferWriter offsets by @fallintoplace in fix(parquet/encoding): honor BufferWriter offsets #1144
• fix(flightsql): stop row streaming after close by @fallintoplace in fix(flightsql): stop row streaming after close #1151
• fix(flightsql): protect configured authorization metadata by @fallintoplace in fix(flightsql): protect configured authorization metadata #1153
• perf(parquet/pqarrow): avoid reflection in integer transfers by @fallintoplace in perf(parquet/pqarrow): avoid reflection in integer transfers #1163
• fix(flightsql): reject unsupported transaction options by @fallintoplace in fix(flightsql): reject unsupported transaction options #1152
• fix(arrow/scalar): handle zero-length scalar arrays by @fallintoplace in fix(arrow/scalar): handle zero-length scalar arrays #1116
• perf(arrow/array): pack validity values in batches by @fallintoplace in perf(arrow/array): pack validity values in batches #1185
• feat(arrow/array): roll back JSON builder state after failed rows by @fallintoplace in feat(arrow/array): roll back JSON builder state after failed rows #1113
• fix(arrow/ipc): avoid deadlock on compression errors by @fallintoplace in fix(arrow/ipc): avoid deadlock on compression errors #1142
• fix(arrow/csv): append null for later parse failures by @fallintoplace in fix(arrow/csv): append null for later parse failures #1149
• feat(arrow/array): add ValueAsAny for native Go values by @dgvj-work in feat(arrow/array): add ValueAsAny for native Go values #1170
• perf(arrow/array): bulk append fixed-size list nulls and empty values by @fallintoplace in perf(arrow/array): bulk append fixed-size list nulls and empty values #1183
• perf(parquet): encode spaced booleans without compaction by @fallintoplace in perf(parquet): encode spaced booleans without compaction #1175
• fix(parquet/pqarrow): reject decimal overflow in integer columns by @fallintoplace in fix(parquet/pqarrow): reject decimal overflow in integer columns #1122
• fix(arrow/array/arreflect): reject unrepresentable timestamps by @fallintoplace in fix(arrow/array/arreflect): reject unrepresentable timestamps #1123
• feat(compute): add cumulative_sum and cumulative_sum_checked by @fallintoplace in feat(compute): add cumulative_sum and cumulative_sum_checked #1139
• fix(arrow/ipc): return compression errors from record encoder by @fallintoplace in fix(arrow/ipc): return compression errors from record encoder #1141
• perf(arrow/array): bulk append fixed-width empty values by @fallintoplace in perf(arrow/array): bulk append fixed-width empty values #1180
• perf(arrow/csv): reserve record builders for chunked reads by @fallintoplace in perf(arrow/csv): reserve record builders for chunked reads #1198
• perf(arrow/array): compare list values by valid runs by @fallintoplace in perf(arrow/array): compare list values by valid runs #1184
• perf(parquet): optimize numeric dictionary insertion by @fallintoplace in perf(parquet): optimize numeric dictionary insertion #1178
• feat(arrow/flight/sql): Add is_update field to ActionCreatePreparedStatementResult by @ennuite in feat(arrow/flight/sql): Add is_update field to ActionCreatePreparedStatementResult #732
• chore: Bump google.golang.org/protobuf from 1.36.11 to 1.36.12 by @dependabot[bot] in chore: Bump google.golang.org/protobuf from 1.36.11 to 1.36.12 #1207
• chore: Bump github.com/stretchr/testify from 1.11.1 to 1.12.0 by @dependabot[bot] in chore: Bump github.com/stretchr/testify from 1.11.1 to 1.12.0 #1208
• feat(compute): add dictionary_encode by @fallintoplace in feat(compute): add dictionary_encode #1140
• perf(parquet): batch plain BYTE_ARRAY encoding by @fallintoplace in perf(parquet): batch plain BYTE_ARRAY encoding #1193
• fix(arrow/csv): append null for invalid fixed-size binary values by @fallintoplace in fix(arrow/csv): append null for invalid fixed-size binary values #1121
• fix(parquet/pqarrow): reject out-of-range INT96 timestamps by @fallintoplace in fix(parquet/pqarrow): reject out-of-range INT96 timestamps #1128
• perf(parquet): batch delta binary packed decoding by @fallintoplace in perf(parquet): batch delta binary packed decoding #1169
• perf(arrow/array): bulk append list nulls and empty values by @fallintoplace in perf(arrow/array): bulk append list nulls and empty values #1187
• perf(parquet/pqarrow): decode booleans directly to bitmaps by @fallintoplace in perf(parquet/pqarrow): decode booleans directly to bitmaps #1182
• perf(parquet/pqarrow): build nullable fixed-size list children directly by @fallintoplace in perf(parquet/pqarrow): build nullable fixed-size list children directly #1195
• perf(internal/hashing): build binary memo offsets in one pass by @fallintoplace in perf(internal/hashing): build binary memo offsets in one pass #1196
• perf(arrow/compute): take chunked binary values without concatenation by @fallintoplace in perf(arrow/compute): take chunked binary values without concatenation #1199
• chore: Bump github.com/twmb/avro from 1.7.2 to 1.8.0 by @dependabot[bot] in chore: Bump github.com/twmb/avro from 1.7.2 to 1.8.0 #1209
• perf(arrow/array): reuse TableReader scratch slices by @fallintoplace in perf(arrow/array): reuse TableReader scratch slices #1200
• fix(parquet/pqarrow): reject decimal overflow when writing integers by @fallintoplace in fix(parquet/pqarrow): reject decimal overflow when writing integers #1161
• fix(arrow/array): preserve run-end encoded empty values by @fallintoplace in fix(arrow/array): preserve run-end encoded empty values #1156
• fix(compute): preserve caller context during execution by @fallintoplace in fix(compute): preserve caller context during execution #1154
• fix(compute): propagate nested expression serialization errors by @fallintoplace in fix(compute): propagate nested expression serialization errors #1160
• fix(compute): report FuncMeta for meta functions by @fallintoplace in fix(compute): report FuncMeta for meta functions #1155
• chore: Bump google.golang.org/grpc from 1.83.0 to 1.83.1 by @dependabot[bot] in chore: Bump google.golang.org/grpc from 1.83.0 to 1.83.1 #1212
• chore: Bump github.com/pierrec/lz4/v4 from 4.1.28 to 4.1.29 by @dependabot[bot] in chore: Bump github.com/pierrec/lz4/v4 from 4.1.28 to 4.1.29 #1213
• chore: Bump modernc.org/sqlite from 1.56.0 to 1.57.0 by @dependabot[bot] in chore: Bump modernc.org/sqlite from 1.56.0 to 1.57.0 #1210
• perf(arrow/array): optimize bulk appends for binary builders by @fallintoplace in perf(arrow/array): optimize bulk appends for binary builders #1188
• chore: Bump github.com/stretchr/testify from 1.12.0 to 1.12.1 by @dependabot[bot] in chore: Bump github.com/stretchr/testify from 1.12.0 to 1.12.1 #1211
• perf(arrow/array): avoid temporary buffers when concatenating by @fallintoplace in perf(arrow/array): avoid temporary buffers when concatenating #1194
• perf(compute): batch contiguous filter take indices by @fallintoplace in perf(compute): batch contiguous filter take indices #1197
• perf(arrow/ipc): slice already-zero-based offset buffers by @fallintoplace in perf(arrow/ipc): slice already-zero-based offset buffers #1191
• perf(parquet): encode RLE levels in batches by @fallintoplace in perf(parquet): encode RLE levels in batches #1177
• fix(arrow): include REE value nullability in type equality by @fallintoplace in fix(arrow): include REE value nullability in type equality #1158
• fix(parquet): build bloom filters from dictionary entries by @fallintoplace in fix(parquet): build bloom filters from dictionary entries #1164
• fix(arrow): validate precision in NewDecimalType by @fallintoplace in fix(arrow): validate precision in NewDecimalType #1162
• perf(arrow/csv): avoid materializing rows before writing by @fallintoplace in perf(arrow/csv): avoid materializing rows before writing #1190
• perf(arrow/array): batch run-end encoded appends by @fallintoplace in perf(arrow/array): batch run-end encoded appends #1214
• fix(arrow/array): validate union JSON type codes by @fallintoplace in fix(arrow/array): validate union JSON type codes #1157
• fix(parquet/variant): validate compound value bounds by @fallintoplace in fix(parquet/variant): validate compound value bounds #1126
• perf(arrow/array): validate dictionary indices by valid runs by @fallintoplace in perf(arrow/array): validate dictionary indices by valid runs #1181
• perf(parquet): bound Bloom filter hash batches by @fallintoplace in perf(parquet): bound Bloom filter hash batches #1218
• perf(arrow/array): optimize bulk appends for struct builders by @fallintoplace in perf(arrow/array): optimize bulk appends for struct builders #1189
• fix(arrow): return owned field index slices by @fallintoplace in fix(arrow): return owned field index slices #1159
• perf(arrow/array): avoid temporary slices in Diff comparisons by @fallintoplace in perf(arrow/array): avoid temporary slices in Diff comparisons #1239
• perf(parquet/compress): reuse gzip writers by @fallintoplace in perf(parquet/compress): reuse gzip writers #1238
• perf(parquet): reuse fixed-length zero scratch by @fallintoplace in perf(parquet): reuse fixed-length zero scratch #1236
• perf(compute): share logical row maps across aligned sorts by @fallintoplace in perf(compute): share logical row maps across aligned sorts #1234
• perf(compute): fast-path single-span execution by @fallintoplace in perf(compute): fast-path single-span execution #1233
• perf(parquet): reuse boolean spaced scratch by @fallintoplace in perf(parquet): reuse boolean spaced scratch #1231
• perf(parquet): reuse delta PutSpaced scratch by @fallintoplace in perf(parquet): reuse delta PutSpaced scratch #1230
• perf(arrow/array): remove map validity slice by @fallintoplace in perf(arrow/array): remove map validity slice #1228
• perf(arrow/array): copy same-dictionary indices directly by @fallintoplace in perf(arrow/array): copy same-dictionary indices directly #1225
• perf(parquet): reuse dictionary decoder scratch by @fallintoplace in perf(parquet): reuse dictionary decoder scratch #1222
• perf(arrow/array): compare fixed-width values in bulk by @fallintoplace in perf(arrow/array): compare fixed-width values in bulk #1171
• perf(arrow/array): compare boolean values by valid runs by @fallintoplace in perf(arrow/array): compare boolean values by valid runs #1186
• perf(arrow/array): avoid temporary buffers for binary concatenation by @fallintoplace in perf(arrow/array): avoid temporary buffers for binary concatenation #1215
• perf(arrow/array): batch union null and empty appends by @fallintoplace in perf(arrow/array): batch union null and empty appends #1216
• perf(parquet): decode levels directly into int16 batches by @fallintoplace in perf(parquet): decode levels directly into int16 batches #1176
• perf(parquet): batch DELTA_BYTE_ARRAY encoding by @fallintoplace in perf(parquet): batch DELTA_BYTE_ARRAY encoding #1179
• perf(parquet/variant): skip metadata construction in Encode by @fallintoplace in perf(parquet/variant): skip metadata construction in Encode #1217
• perf(parquet): decode BSS FLBA into contiguous storage by @fallintoplace in perf(parquet): decode BSS FLBA into contiguous storage #1172
• perf(parquet): stop spaced expansion once aligned by @fallintoplace in perf(parquet): stop spaced expansion once aligned #1220
• perf(arrow/ipc): reuse record encoder scratch by @fallintoplace in perf(arrow/ipc): reuse record encoder scratch #1221
• perf(arrow/array): batch dictionary empty appends by @fallintoplace in perf(arrow/array): batch dictionary empty appends #1223
• perf(compute): reserve binary view cast payload by @fallintoplace in perf(compute): reserve binary view cast payload #1227
• perf(compute): use stack scratch for null propagation by @fallintoplace in perf(compute): use stack scratch for null propagation #1229
• perf(arrow/array): bulk append BinaryView nulls by @fallintoplace in perf(arrow/array): bulk append BinaryView nulls #1232
• perf(arrow/array): use unsafe writes for binary bulk appends by @fallintoplace in perf(arrow/array): use unsafe writes for binary bulk appends #1240
• perf(arrow/array): compare union values in runs by @fallintoplace in perf(arrow/array): compare union values in runs #1244
• perf(arrow/array): batch-pack boolean values by @fallintoplace in perf(arrow/array): batch-pack boolean values #1237
• fix(parquet/pqarrow): require Variant value field when writing by @dgvj-work in fix(parquet/pqarrow): require Variant value field when writing #1242
• perf(arrow/compute): avoid goroutines for serial record filtering by @fallintoplace in perf(arrow/compute): avoid goroutines for serial record filtering #1245
• perf(parquet): reuse dictionary RLE decoder across pages by @fallintoplace in perf(parquet): reuse dictionary RLE decoder across pages #1247
• perf(parquet/pqarrow): write record ranges directly by @fallintoplace in perf(parquet/pqarrow): write record ranges directly #1248
• perf(parquet/pqarrow): avoid goroutines for serial struct reads by @fallintoplace in perf(parquet/pqarrow): avoid goroutines for serial struct reads #1250
• perf(parquet): decode RLE booleans into bitmaps by @fallintoplace in perf(parquet): decode RLE booleans into bitmaps #1246
• feat(compute): add list_element by @fallintoplace in feat(compute): add list_element #1138
• perf(parquet): reuse DataPageV2 assembly buffer for eager writes by @fallintoplace in perf(parquet): reuse DataPageV2 assembly buffer for eager writes #1192
• perf(arrow/array): append fixed-width nulls in bulk by @fallintoplace in perf(arrow/array): append fixed-width nulls in bulk #1173
• perf(arrow/array): concatenate bitmaps directly by @fallintoplace in perf(arrow/array): concatenate bitmaps directly #1224
• perf(arrow/compute): avoid goroutines for serial take by @fallintoplace in perf(arrow/compute): avoid goroutines for serial take #1226
• perf(compute): use typed memo insertion for numeric hashes by @fallintoplace in perf(compute): use typed memo insertion for numeric hashes #1251
• perf(parquet): decode DELTA_BYTE_ARRAY into contiguous storage by @fallintoplace in perf(parquet): decode DELTA_BYTE_ARRAY into contiguous storage #1252
• perf(parquet/pqarrow): use synchronous path for serial reads by @fallintoplace in perf(parquet/pqarrow): use synchronous path for serial reads #1235
• perf(arrow/array): reuse full chunks in NewChunkedSlice by @fallintoplace in perf(arrow/array): reuse full chunks in NewChunkedSlice #1257
• perf(parquet): reuse boolean Bloom filter hashes by @fallintoplace in perf(parquet): reuse boolean Bloom filter hashes #1258
• perf(arrow/ipc): release dictionary payloads eagerly by @fallintoplace in perf(arrow/ipc): release dictionary payloads eagerly #1259
• perf(parquet): SIMD BYTE_STREAM_SPLIT encoding by @fallintoplace in perf(parquet): SIMD BYTE_STREAM_SPLIT encoding #1260
• perf(parquet): reuse DELTA byte-array scratch by @fallintoplace in perf(parquet): reuse DELTA byte-array scratch #1261
• perf(parquet/pqarrow): write FixedSizeBinary values directly by @fallintoplace in perf(parquet/pqarrow): write FixedSizeBinary values directly #1263
• chore: Bump github.com/andybalholm/brotli from 1.2.2 to 1.2.3 by @dependabot[bot] in chore: Bump github.com/andybalholm/brotli from 1.2.2 to 1.2.3 #1270
• chore: Bump google.golang.org/grpc from 1.83.1 to 1.83.2 by @dependabot[bot] in chore: Bump google.golang.org/grpc from 1.83.1 to 1.83.2 #1271
• feat(extensions): add VariantGet for path extraction from variant arrays by @nssalian in feat(extensions): add VariantGet for path extraction from variant arrays #1206
• perf(parquet): use typed memo insertion for byte-array dictionaries by @derekperkins in perf(parquet): use typed memo insertion for byte-array dictionaries #1272
• fix(arrow/array): clamp run ends when concatenating a sliced RunEndEncoded array by @winklemad in fix(arrow/array): clamp run ends when concatenating a sliced RunEndEncoded array #1219
• fix(parquet): guard byte-stream-split encoder release against never-flushed buffer by @tom-eon in fix(parquet): guard byte-stream-split encoder release against never-flushed buffer #1273
• perf(bitutil): add ARM64 NEON bitmap operations by @fallintoplace in perf(bitutil): add ARM64 NEON bitmap operations #1264
• perf(arrow/array): compare binary values by valid runs by @fallintoplace in perf(arrow/array): compare binary values by valid runs #1174
• perf(compute): add ARM64 NEON arithmetic kernels by @fallintoplace in perf(compute): add ARM64 NEON arithmetic kernels #1265
• perf(parquet/compress): reuse Brotli writers across encodes by @fallintoplace in perf(parquet/compress): reuse Brotli writers across encodes #1254
• perf(arrow/array): skip full-range SliceEqual slices by @fallintoplace in perf(arrow/array): skip full-range SliceEqual slices #1249
• perf(parquet): vectorize fixed-width dictionary materialization by @fallintoplace in perf(parquet): vectorize fixed-width dictionary materialization #1269
• perf(compute): add ARM64 NEON comparison kernels by @fallintoplace in perf(compute): add ARM64 NEON comparison kernels #1266
• perf(parquet/pqarrow): write binary values from Arrow offsets by @fallintoplace in perf(parquet/pqarrow): write binary values from Arrow offsets #1262
• fix(arrow/extensions): reject Null Variant typed_value by @dgvj-work in fix(arrow/extensions): reject Null Variant typed_value #1243
• fix(arrow/extensions): use canonical Variant extension name by @dgvj-work in fix(arrow/extensions): use canonical Variant extension name #1241
• chore: bump version number by @zeroshade in chore: bump version number #1276
• test(parquet): reduce macOS race test memory pressure by @zeroshade in test(parquet): reduce macOS race test memory pressure #1275
• fix(parquet): explicitly release bloom builders by @zeroshade in fix(parquet): explicitly release bloom builders #1280
• ci: retry and cache dependency downloads by @zeroshade in ci: retry and cache dependency downloads #1279
• fix(compute): discard results after cancellation by @zeroshade in fix(compute): discard results after cancellation #1278
• ci: use bundled GTest for conda integration by @zeroshade in ci: use bundled GTest for conda integration #1277
• fix(parquet): stop BYTE_STREAM_SPLIT FLBA spaced decode aliasing output by @zeroshade in fix(parquet): stop BYTE_STREAM_SPLIT FLBA spaced decode aliasing output #1256
• perf(compute): vectorize mixed int32 filters with AVX2 by @fallintoplace in perf(compute): vectorize mixed int32 filters with AVX2 #1284
• perf(compute): add ARM64 NEON constant multiplication by @fallintoplace in perf(compute): add ARM64 NEON constant multiplication #1268
• perf(compute): vectorize numeric to boolean casts by @fallintoplace in perf(compute): vectorize numeric to boolean casts #1282
• fix(arrow/scalar): align timestamp timezone parsing by @fallintoplace in fix(arrow/scalar): align timestamp timezone parsing #1109
• perf(internal/utils): add ARM64 NEON min/max for 8- and 16-bit integers by @fallintoplace in perf(internal/utils): add ARM64 NEON min/max for 8- and 16-bit integers #1281
• perf(arrow/array): SIMD-pack boolean slices into bitmaps by @fallintoplace in perf(arrow/array): SIMD-pack boolean slices into bitmaps #1283
• chore: fix failing build tags by @zeroshade in chore: fix failing build tags #1285
• perf(compute): SIMD-compact fragmented filters into take indices by @fallintoplace in perf(compute): SIMD-compact fragmented filters into take indices #1287
• perf(compute): add ARM64 NEON comparisons for 8- and 16-bit integers by @fallintoplace in perf(compute): add ARM64 NEON comparisons for 8- and 16-bit integers #1288
• perf(compute): add ARM64 NEON compression for null-free 32-bit fixed-width filters by @fallintoplace in perf(compute): add ARM64 NEON compression for null-free 32-bit fixed-width filters #1286

New Contributors

• @C-Loftus made their first contribution in feat(parquet/schema): Wire up logical type functionality for writing geo types #960
• @tom-eon made their first contribution in fix(parquet): wrap DELTA_BINARY_PACKED deltas at the physical type width #1027
• @prochac made their first contribution in refactor(arrow/avro): migrate from hamba/avro to twmb/avro #830
• @dgvj-work made their first contribution in feat(arrow/array): add ValueAsAny for native Go values #1170
• @ennuite made their first contribution in feat(arrow/flight/sql): Add is_update field to ActionCreatePreparedStatementResult #732
• @derekperkins made their first contribution in perf(parquet): use typed memo insertion for byte-array dictionaries #1272
• @winklemad made their first contribution in fix(arrow/array): clamp run ends when concatenating a sliced RunEndEncoded array #1219

Full Changelog: v18.7.0...v18.8.0

---

This discussion was created from the release Apache Arrow Go 18.8.0.

[fallintoplace]

Great job!