Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Don't track column count when parsing CSV files
When parsing a CSV file we used to check the column count for each row and track the highest number of columns that we found. This information then could be used to create an INSERT statement large enough for all the data. This column number tracking code is removed by this commit. Instead it analyses the first 20 rows only. It does that while generating the field list. Performance-wise this should take a (very) little longer but makes it easier to improve the performance in other ways later which should more than compensate this commit. Feature-wise this should fix some (technically invalid) corner-case CSV files with fewer fields in the title row than in the other rows. It should also break some other (technically invalid) corner-case CSV files if they are imported into an existing table and have less columns than the existing table in their first 20 rows but later on the exact same number. Both cases, I think, don't matter too much.
- Loading branch information
1 parent
67adb99
commit b7a00d3
Showing
5 changed files
with
52 additions
and
44 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters