Skip to content

Commit

Permalink
Allow DELTA for V1 pages
Browse files Browse the repository at this point in the history
  • Loading branch information
martindurant committed Sep 21, 2023
1 parent df12193 commit ff36eba
Showing 1 changed file with 4 additions and 0 deletions.
4 changes: 4 additions & 0 deletions fastparquet/core.py
Original file line number Diff line number Diff line change
Expand Up @@ -159,6 +159,10 @@ def read_data_page(f, helper, header, metadata, skip_nulls=False,
values = values.data[:nval]
else:
values = np.zeros(nval, dtype=np.int8)
elif daph.encoding == parquet_thrift.Encoding.DELTA_BINARY_PACKED:
values = np.empty(daph.num_values - num_nulls, dtype=np.int32)
o = encoding.NumpyIO(values.view('uint8'))
encoding.delta_binary_unpack(io_obj, o)
else:
raise NotImplementedError('Encoding %s' % daph.encoding)
return definition_levels, repetition_levels, values[:nval]
Expand Down

0 comments on commit ff36eba

Please sign in to comment.