20.0.0 (2022-08-05)
Breaking changes:
- Add more const evaluation for
GenericBinaryArray
andGenericListArray
: addPREFIX
and data type constructor #2327 [parquet] [arrow] (HaoYang670) - Make FFI support optional, change APIs to be
safe
(#2302) #2303 [arrow] (tustvold) - Remove
test_utils
from default features (#2298) #2299 [arrow] (tustvold) - Rename
DataType::Decimal
toDataType::Decimal128
#2229 [parquet] [arrow] (viirya) - Add
Decimal128Iter
andDecimal256Iter
and do maximum precision/scale check #2140 [arrow] (viirya)
Implemented enhancements:
- Add the constant data type constructors for
ListArray
#2311 [arrow] - Update
FlightSqlService
trait to pass session info along #2308 [arrow-flight] - Optimize
take_bits
for non-null indices #2306 [arrow] - Make FFI support optional via Feature Flag
ffi
#2302 [arrow] - Mark
ffi::ArrowArray::try_new
is safe #2301 [arrow] - Remove test_utils from default arrow-rs features #2298 [arrow]
- Remove
JsonEqual
trait #2296 [arrow] - Move
with_precision_and_scale
toDecimal
array traits #2291 [arrow] - Improve readability and maybe performance of string --> numeric/time/date/timetamp cast kernels #2285 [arrow]
- Add vectorized unpacking for 8, 16, and 64 bit integers #2276 [parquet]
- Use initial capacity for interner hashmap #2273 [arrow]
- Impl FromIterator for Decimal256Array #2248 [arrow]
- Separate
ArrayReader::next_batch
withArrayReader::read_records
andArrayReader::consume_batch
#2236 [parquet] - Rename
DataType::Decimal
toDataType::Decimal128
#2228 [arrow] - Automatically Grow Parquet BitWriter Buffer #2226 [parquet]
- Add
append_option
support toDecimal128Builder
andDecimal256Builder
#2224 [arrow] - Split the
FixedSizeBinaryArray
andFixedSizeListArray
fromarray_binary.rs
andarray_list.rs
#2217 [arrow] - Don't
Box
Values inPrimitiveDictionaryBuilder
#2215 [arrow] - Use BitChunks in equal_bits #2186 [arrow]
- Implement
Hash
forSchema
#2182 [arrow] - read decimal data type from parquet file with binary physical type #2159 [parquet]
- The
GenericStringBuilder
should useGenericBinaryBuilder
#2156 [arrow] - Update Rust version to 1.62 #2143 [parquet] [arrow] [arrow-flight]
- Check precision and scale against maximum value when constructing
Decimal128
andDecimal256
#2139 [arrow] - Use
ArrayAccessor
inDecimal128Iter
andDecimal256Iter
#2138 [arrow] - Use
ArrayAccessor
andFromIterator
in Cast Kernels #2137 [arrow] - Add
TypedDictionaryArray
for more ergonomic interaction withDictionaryArray
#2136 [arrow] - Use
ArrayAccessor
in Comparison Kernels #2135 [arrow] - Support
peek_next_page()
and skip_next_page
inInMemoryColumnChunkReader
#2129 [parquet] - Lazily materialize the null buffer builder for all array builders. #2125 [arrow]
- Do value validation for
Decimal256
#2112 [arrow] - Support
skip_def_levels
forColumnLevelDecoder
#2107 [parquet] - Add integration test for scan rows with selection #2106 [parquet]
- Support for casting from Utf8/String to
Time32
/Time64
#2053 [arrow] - Update prost and tonic related crates #2268 [arrow-flight] (carols10cents)
Fixed bugs:
- temporal conversion functions cannot work on negative input properly #2325 [arrow]
- IPC writer should truncate string array with all empty string #2312 [arrow]
- Error order for comparing
Decimal128
orDecimal256
#2256 [arrow] - Fix maximum and minimum for decimal values for precision greater than 38 #2246 [arrow]
IntervalMonthDayNanoType::make_value()
does not match C implementation #2234 [arrow]FlightSqlService
trait does not allowimpl
s to do handshake #2210 [arrow-flight]EnabledStatistics::None
not working #2185 [parquet]- Boolean ArrayData Equality Incorrect Slice Handling #2184 [arrow]
- Publicly export MapFieldNames #2118 [arrow]
Documentation updates:
- Update instructions on How to join the slack #arrow-rust channel -- or maybe try to switch to discord?? #2192
- [Minor] Improve arrow and parquet READMEs, document parquet feature flags #2324 [parquet] [arrow] (alamb)
Performance improvements:
- Improve speed of writing string dictionaries to parquet by skipping a copy(#1764) #2322 [parquet] [arrow] (tustvold)
Closed issues:
- Fix wrong logic in calculate_row_count when skipping values #2328 [parquet]
- Support filter for parquet data type #2126 [parquet]
- Make skip value in ByteArrayDecoderDictionary avoid decoding #2088 [parquet]
Merged pull requests:
- fix: Fix skip error in calculate_row_count. #2329 [parquet] (Ted-Jiang)
- temporal conversion functions should work on negative input properly #2326 [arrow] (viirya)
- Increase DeltaBitPackEncoder miniblock size to 64 for 64-bit integers (#2282) #2319 [parquet] (tustvold)
- Remove JsonEqual #2317 [parquet] [arrow] (viirya)
- fix: IPC writer should truncate string array with all empty string #2314 [arrow] (JasonLi-cn)
- Pass pull
Request<FlightDescriptor>
toFlightSqlService
impl
s #2309 [parquet] [arrow-flight] (avantgardnerio) - Speedup take_boolean / take_bits for non-null indices (~4 - 5x speedup) #2307 [arrow] (Dandandan)
- Add typed dictionary (#2136) #2297 [arrow] (tustvold)
- [Minor] Improve types shown in cast error messages #2295 [arrow] (alamb)
- Move
with_precision_and_scale
toBasicDecimalArray
trait #2292 [parquet] [arrow] (viirya) - Replace the
fn get_data_type
byconst DATA_TYPE
in BinaryArray and StringArray #2289 [arrow] (HaoYang670) - Clean up string casts and improve performance #2284 [arrow] (alamb)
- [Minor] Add tests for temporal cast error paths #2283 [arrow] (alamb)
- Add unpack8, unpack16, unpack64 (#2276) ~10-50% faster #2278 [parquet] (tustvold)
- Fix bugs in the
from_list
function. #2277 [arrow] (HaoYang670) - fix: use signed comparator to compare decimal128 and decimal256 #2275 [arrow] (liukun4515)
- Use initial capacity for interner hashmap #2272 [parquet] (Dandandan)
- Remove fallibility from paruqet RleEncoder (#2226) #2259 [parquet] (tustvold)
- Fix escaped like wildcards in
like_utf8
/nlike_utf8
kernels #2258 [arrow] (daniel-martinez-maqueda-sap) - Add tests for reading nested decimal arrays from parquet #2254 [parquet] (tustvold)
- feat: Implement string cast operations for Time32 and Time64 #2251 [arrow] (stuartcarnie)
- move
FixedSizeList
toarray_fixed_size_list.rs
#2250 [arrow] (HaoYang670) - Impl FromIterator for Decimal256Array #2247 [arrow] (viirya)
- Fix max and min value for decimal precision greater than 38 #2245 [arrow] (viirya)
- Make
Schema::fields
andSchema::metadata
pub
(public) #2239 [arrow] (alamb) - [Minor] Improve Schema metadata mismatch error #2238 [arrow] (alamb)
- Separate ArrayReader::next_batch with read_records and consume_batch #2237 [parquet] (Ted-Jiang)
- Update
IntervalMonthDayNanoType::make_value()
to conform to specifications #2235 [arrow] (avantgardnerio) - Disable value validation for Decimal256 case #2232 [arrow] (viirya)
- Automatically grow parquet BitWriter (#2226) (~10% faster) #2231 [parquet] (tustvold)
- Only trigger
arrow
CI on changes to arrow #2227 (alamb) - Add append_option support to decimal builders #2225 [arrow] (bphillips-exos)
- Optimized writing of byte array to parquet (#1764) (2x faster) #2221 [parquet] (tustvold)
- Increase test coverage of ArrowWriter #2220 [parquet] (tustvold)
- Update instructions on how to join the Slack channel #2219 (HaoYang670)
- Move
FixedSizeBinaryArray
toarray_fixed_size_binary.rs
#2218 [arrow] (HaoYang670) - Avoid boxing in PrimitiveDictionaryBuilder #2216 [arrow] (tustvold)
- remove redundant CI benchmark check, cleanups #2212 [parquet] (alamb)
- Update
FlightSqlService
trait to proxy handshake #2211 [arrow-flight] (avantgardnerio) - parquet: export json api with
serde_json
feature name #2209 [parquet] (flisky) - Cleanup record skipping logic and tests (#2158) #2199 [parquet] (tustvold)
- Use BitChunks in equal_bits #2194 [arrow] (tustvold)
- Fix disabling parquet statistics (#2185) #2191 [parquet] (tustvold)
- Change CI names to match crate names #2189 (alamb)
- Fix offset handling in boolean_equal (#2184) #2187 [arrow] (tustvold)
- Implement
Hash
forSchema
#2183 [arrow] (crepererum) - Let the
StringBuilder
useBinaryBuilder
#2181 [arrow] (HaoYang670) - Use ArrayAccessor and FromIterator in Cast Kernels #2169 [arrow] (viirya)
- Split most arrow specific CI checks into their own workflows (reduce common CI time to 21 minutes) #2168 (alamb)
- Remove another attempt to cache target directory in action.yaml #2167 (alamb)
- Run actions on push to master, pull requests #2166 (alamb)
- Break parquet_derive and arrow_flight tests into their own workflows #2165 (alamb)
- [minor] use type aliases refine code. #2161 [parquet] (Ted-Jiang)
- parquet reader: Support reading decimals from parquet
BYTE_ARRAY
type #2160 [parquet] (liukun4515) - Add integration test for scan rows with selection #2158 [parquet] (Ted-Jiang)
- Use ArrayAccessor in Comparison Kernels #2157 [arrow] (viirya)
- Implement
peek\_next\_page
andskip\_next\_page
for `InMemoryColumnCh… #2155 [parquet] (thinkharderdev) - Avoid decoding unneeded values in ByteArrayDecoderDictionary #2154 [parquet] (thinkharderdev)
- Only run integration tests when
arrow
changes #2152 (alamb) - Break out docs CI job to its own github action #2151 (alamb)
- Do not pretend to cache rust build artifacts, speed up CI by ~20% #2150 (alamb)
- Update rust version to 1.62 #2144 [parquet] [arrow] [arrow-flight] (Ted-Jiang)
- Make MapFieldNames public (#2118) #2134 [arrow] (tustvold)
- Add ArrayAccessor trait, remove duplication in array iterators (#1948) #2133 [arrow] (tustvold)
- Lazily materialize the null buffer builder for all array builders. #2127 [arrow] (HaoYang670)
- Faster parquet DictEncoder (~20%) #2123 [parquet] (tustvold)
- Add validation for Decimal256 #2113 [arrow] (viirya)
- Support skip_def_levels for ColumnLevelDecoder #2111 [parquet] (Ted-Jiang)
- Donate
object_store
code from object_store_rs to arrow-rs #2081 (alamb) - Improve
validate_utf8
performance #2048 [arrow] (tfeda)
* This Changelog was automatically generated by github_changelog_generator