You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We permit dictionaries to grow using the isDelta property in the IPC protocol. I think it should be allowed for the same dictionary ID to appear in an IPC protocol stream but with isDelta=false. This would indicate that the dictionary in that message is to replace any prior-observed ones in subsequent record batches.
For example, we might have dictionary batches in a stream:
We permit dictionaries to grow using the
isDelta
property in the IPC protocol. I think it should be allowed for the same dictionary ID to appear in an IPC protocol stream but withisDelta=false
. This would indicate that the dictionary in that message is to replace any prior-observed ones in subsequent record batches.For example, we might have dictionary batches in a stream:
Such data could easily be produced by a stream producer that is creating dictionaries in different execution threads.
Reporter: Wes McKinney / @wesm
Related issues:
Note: This issue was originally created as ARROW-5767. Please see the migration documentation for further details.
The text was updated successfully, but these errors were encountered: