28.0.0 (2022-11-25)
Breaking changes:
- StructArray::columns return slice #3186 [parquet] [arrow] (tustvold)
- Return slice from GenericByteArray::value_data #3171 [arrow] (tustvold)
- Support decimal negative scale #3152 [arrow] (viirya)
- refactor: convert
Field::metadata
toHashMap
#3148 [parquet] [arrow] (crepererum) - Don't Skip Serializing Empty Metadata (#3082) #3126 [arrow] (askoa)
- Add Decimal128, Decimal256, Float16 to DataType::is_numeric #3121 [arrow] (tustvold)
- Upgrade to thrift 0.17 and fix issues #3104 [parquet] [arrow] (Jimexist)
- Fix prettyprint for Interval second fractions #3093 [arrow] (Jefffrey)
- Remove Option from
Field::metadata
#3091 [parquet] [arrow] (askoa)
Implemented enhancements:
- Add iterator to RowSelection #3172 [parquet]
- create an integration test set for parquet crate against pyspark for working with bloom filters #3167 [parquet]
- Row Format Size Tracking #3160 [arrow]
- Add ArrayBuilder::finish_cloned() #3154 [arrow]
- Optimize memory usage of json reader #3150
- Add
Field::size
andDataType::size
#3147 [parquet] [arrow] - Add like_utf8_scalar_dyn kernel #3145 [arrow]
- support comparison for decimal128 array with scalar in kernel #3140 [arrow]
- audit and create a document for bloom filter configurations #3138 [parquet]
- Should be the rounding vs truncation when cast decimal to smaller scale #3137 [arrow]
- Upgrade chrono to 0.4.23 #3120
- Implements more temporal kernels using time_fraction_dyn #3108 [arrow]
- Upgrade to thrift 0.17 #3105 [parquet] [arrow]
- Be able to parse time formatted strings #3100 [arrow]
- Improve "Fail to merge schema" error messages #3095 [arrow]
- Expose
SortingColumn
when reading and writing parquet metadata #3090 [parquet] - Change Field::metadata to HashMap #3086 [parquet] [arrow]
- Support bloom filter reading and writing for parquet #3023 [parquet]
- API to take back ownership of an ArrayRef #2901 [arrow]
- Specialized Interleave Kernel #2864 [arrow]
Fixed bugs:
- arithmatic overflow leads to segfault in
concat_batches
#3123 [arrow] - Clippy failing on master : error: use of deprecated associated function chrono::NaiveDate::from_ymd: use from_ymd_opt() instead #3097 [parquet] [arrow]
- Pretty print for interval types has wrong formatting #3092 [arrow]
- Field is not serializable with binary formats #3082 [arrow]
- Decimal Casts are Unchecked #2986 [arrow]
Closed issues:
- Release Arrow
27.0.0
(next release after26.0.0
) #3045 [parquet] [arrow] [arrow-flight] - Perf about ParquetRecordBatchStream vs ParquetRecordBatchReader #2916
Merged pull requests:
- Improve regex related kernels by upto 85% #3192 [arrow] (psvri)
- Derive clone for arrays #3184 [arrow] (tustvold)
- Row decode cleanups #3180 [arrow] (tustvold)
- Update zstd requirement from 0.11.1 to 0.12.0 #3178 [parquet] [arrow] (dependabot[bot])
- Move decimal constants from
arrow-data
toarrow-schema
crate #3177 [arrow] (mbrobbel) - bloom filter part V: add an integration with pytest against pyspark #3176 [parquet] (Jimexist)
- Bloom filter config tweaks (#3023) #3175 [parquet] (tustvold)
- Add RowParser #3174 [arrow] (tustvold)
- Add
RowSelection::iter()
,Into<Vec<RowSelector>>
and example #3173 [parquet] (alamb) - Add read parquet examples #3170 [parquet] (xudong963)
- Faster BinaryArray to StringArray conversion (~67%) #3168 [arrow] (tustvold)
- Remove unnecessary downcasts in builders #3166 [arrow] (tustvold)
- bloom filter part IV: adjust writer properties, bloom filter properties, and incorporate into column encoder #3165 [parquet] (Jimexist)
- Fix parquet decimal precision #3164 [parquet] (psvri)
- Add Row size methods (#3160) #3163 [arrow] (tustvold)
- Prevent precision=0 for decimal type #3162 [arrow] (psvri)
- Remove unnecessary Buffer::from_slice_ref reference #3161 [arrow] (tustvold)
- Add finish_cloned to ArrayBuilder #3158 [arrow] (askoa)
- Check overflow in MutableArrayData extend offsets (#3123) #3157 [arrow] (tustvold)
- Extend Decimal256 as Primitive #3156 [arrow] (tustvold)
- Doc improvements #3155 [arrow] (psvri)
- Add collect.rs example #3153 [arrow] (viirya)
- Implement Neg for i256 #3151 [arrow] (tustvold)
- feat:
{Field,DataType}::size
#3149 [arrow] (crepererum) - Add like_utf8_scalar_dyn kernel #3146 [arrow] (viirya)
- comparison op: decimal128 array with scalar #3141 [arrow] (liukun4515)
- Cast: should get the round result for decimal to a decimal with smaller scale #3139 [arrow] (liukun4515)
- Fix Panic on Reading Corrupt Parquet Schema (#2855) #3130 [parquet] (psvri)
- Clippy parquet fixes #3124 [parquet] [arrow] (psvri)
- Add GenericByteBuilder (#2969) #3122 [arrow] (tustvold)
- parquet bloom filter part III: add sbbf writer, remove
bloom
default feature, add reader properties #3119 [parquet] (Jimexist) - Add downcast_array (#2901) #3117 [arrow] (tustvold)
- Add COW conversion for Buffer and PrimitiveArray and unary_mut #3115 [arrow] (viirya)
- Include field name in merge error message #3113 [arrow] (andygrove)
- Add PrimitiveArray::unary_opt #3110 [arrow] (tustvold)
- Implements more temporal kernels using time_fraction_dyn #3107 [arrow] (viirya)
- cast: support unsigned numeric type to decimal128 #3106 [arrow] (liukun4515)
- Expose
SortingColumn
in parquet files #3103 [parquet] (askoa) - parquet bloom filter part II: read sbbf bitset from row group reader, update API, and add cli demo #3102 [parquet] (Jimexist)
- Parse Time32/Time64 from formatted string #3101 [arrow] (Jefffrey)
- Cleanup temporal _internal functions #3099 [arrow] (viirya)
- Improve schema mismatch error message #3098 [arrow] (askoa)
- Fix clippy by avoiding deprecated functions in chrono #3096 [parquet] [arrow] (viirya)
- Minor: Add diagrams and documentation to row format #3094 [arrow] (alamb)
- Minor: Use ArrowNativeTypeOp instead of total_cmp directly #3087 [arrow] (viirya)
- Check overflow while casting between decimal types #3076 [arrow] (viirya)
- add bloom filter implementation based on split block (sbbf) spec #3057 [parquet] (Jimexist)
- Add FixedSizeBinaryArray::try_from_sparse_iter_with_size #3054 [arrow] (maxburke)
* This Changelog was automatically generated by github_changelog_generator