You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently, the default FileSystemFactoryOptions::exclude_invalid_files will silently ignore unsupported files (either IO error, not of the valid format, corruption, missing compression codecs, etc...) when creating a FileSystemSource.
We should change this behavior to propagate an error in the Inspect/Finish calls by default and allow the user to toggle exclude_invalid_files. The error should contain at least the file path and a decipherable error (if possible).
Currently, the default
FileSystemFactoryOptions::exclude_invalid_files
will silently ignore unsupported files (either IO error, not of the valid format, corruption, missing compression codecs, etc...) when creating aFileSystemSource
.We should change this behavior to propagate an error in the Inspect/Finish calls by default and allow the user to toggle
exclude_invalid_files
. The error should contain at least the file path and a decipherable error (if possible).Reporter: Francois Saint-Jacques / @fsaintjacques
Assignee: Francois Saint-Jacques / @fsaintjacques
Related issues:
Note: This issue was originally created as ARROW-7673. Please see the migration documentation for further details.
The text was updated successfully, but these errors were encountered: