-
Notifications
You must be signed in to change notification settings - Fork 4.1k
[C++] Support delta dictionaries in the IPC file format #29130
Description
The spec states:
In the file format, there is no requirement that dictionary keys should be defined in a
DictionaryBatchbefore they are used in aRecordBatch, as long as the keys are
defined somewhere in the file. Further more, it is invalid to have more than one non-delta
dictionary batch per dictionary ID (i.e. dictionary replacement is not supported). Delta
dictionaries are applied in the order they appear in the file footer.
This seems to suggest that dictionary deltas (but not replacements) are allowed. However, both reading and writing an IPC file with a dictionary delta fails.
Reporter: Weston Pace / @westonpace
Assignee: Weston Pace / @westonpace
Watchers: Rok Mihevc / @rok
Related issues:
- [Integration] Add delta dictionaries in file format to integration tests (is depended upon by)
PRs and other links:
Note: This issue was originally created as ARROW-13467. Please see the migration documentation for further details.