Skip to content

zinob/csvloader

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

98 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

csvloader

Built to parse a specific set of CSV-files corresponding to a specific database. The format of these files is a bit impractical why this script attempts to do some rudimentary type-inference and data-massaging. For general purpose you are probably better of using the Pandas CSV-reader. But if you need to tweak your own, this reader might be an option.

Usage example

This would read the complete file to guess the column-types and would just skip lines which contains more or fewer columns than the header. It would load the ; -sepparated file "foo" to the table "bar_table", several csv-files and --map commands can be supplied.

filetodb.py -v --database "mysql://user:pass@localhost/test?use_unicode=1charset=utf8" --sep ";" --ignore-invalid --full-file --map "foo.csv:bar_table" foo.csv

If -- is ommitted it will just attempt to read a few thousand lines from the file in order to speed the analysis up. This might lead to poor estimates on line-lenght and some times even wrong types. This option should probbably be inverted and called --sparase-read or something...

About

Attempts to type inferr a rather peculiar CSV format and uppload it to an SQL-database

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages