Skip to content

[C++] Support delta dictionaries in the IPC file format #29130

@asfimport

Description

@asfimport

The spec states:

In the file format, there is no requirement that dictionary keys should be defined in a
DictionaryBatch before they are used in a RecordBatch, as long as the keys are
defined somewhere in the file. Further more, it is invalid to have more than one non-delta
dictionary batch per dictionary ID (i.e. dictionary replacement is not supported). Delta
dictionaries are applied in the order they appear in the file footer.

This seems to suggest that dictionary deltas (but not replacements) are allowed. However, both reading and writing an IPC file with a dictionary delta fails.

Reporter: Weston Pace / @westonpace
Assignee: Weston Pace / @westonpace
Watchers: Rok Mihevc / @rok

Related issues:

PRs and other links:

Note: This issue was originally created as ARROW-13467. Please see the migration documentation for further details.

Metadata

Metadata

Assignees

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions