You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When reading in a dataset where the schema specifies that column X is of type int64 but the partition actually contains the data stored in that columns as int32, an upcast should be done.
Francois Saint-Jacques / @fsaintjacques:
Once we have instanciated Fragment, we can create a CastFragment composing an existing Fragment. Some format do support casting, e.g. CSV, and some don't e.g. Parquet or IPC.
Joris Van den Bossche / @jorisvandenbossche:
Do we need a separate Fragment type? We could also do the cast when scanning (eg we already do some edits at that point, like projection, adding null columns, etc)
When reading in a dataset where the schema specifies that column X is of type
int64
but the partition actually contains the data stored in that columns asint32
, an upcast should be done.Reporter: Uwe Korn / @xhochy
Related issues:
Note: This issue was originally created as ARROW-8282. Please see the migration documentation for further details.
The text was updated successfully, but these errors were encountered: