CLI tool to convert CSV/Excel/HTML/JSON/LTSV/Markdown/TSV/Google-Sheets to a SQLite database file.
Python Batchfile
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
docs
requirements
sqlitebiter
test
.gitignore Update .gitignore Jun 19, 2016
.travis.yml
LICENSE
MANIFEST.in Update MANIFEST.in Oct 22, 2016
README.rst Update README Jan 8, 2017
appveyor.yml
cli.py
packaging.bat
setup.cfg
setup.py
tox.ini

README.rst

sqlitebiter

Linux CI test status Windows CI test status

Summary

A CLI tool to convert CSV/Excel/HTML/JSON/LTSV/Markdown/TSV/Google-Sheets to a SQLite database file.

Features

Usage

Create SQLite database from files

docs/gif/usage_example.gif

Create SQLite database from URL

Following is an example that convert HTML table tags within a web page to SQLite tables.

$ sqlitebiter -v url "https://en.wikipedia.org/wiki/Comparison_of_firewalls"
[INFO] convert 'https://en.wikipedia.org/wiki/Comparison_of_firewalls' to 'Comparison_of_firewalls_Wikipedia_html1 (Firewall TEXT, License TEXT, CostUsageLimits TEXT, OS TEXT)' table
[INFO] convert 'https://en.wikipedia.org/wiki/Comparison_of_firewalls' to 'Comparison_of_firewalls_Wikipedia_html2 (Firewall TEXT, License TEXT, Cost TEXT, OS TEXT)' table
[INFO] convert 'https://en.wikipedia.org/wiki/Comparison_of_firewalls' to 'Comparison_of_firewalls_Wikipedia_html3 (CanTarget TEXT, Changingdefaultpolicytoacceptrejectbyissuingasinglerule TEXT, IPdestinationaddresses TEXT, IPsourceaddresses TEXT, TCPUDPdestinationports TEXT, TCPUDPsourceports TEXT, EthernetMACdestinationaddress TEXT, EthernetMACsourceaddress TEXT, Inboundfirewallingress TEXT, Outboundfirewallegress TEXT)' table
[INFO] convert 'https://en.wikipedia.org/wiki/Comparison_of_firewalls' to 'Comparison_of_firewalls_Wikipedia_html4 (Can TEXT, [workatOSILayer4statefulfirewall] TEXT, [workatOSILayer7applicationinspection] TEXT, ChangeTTLTransparenttotraceroute TEXT, ConfigureREJECTwithanswer TEXT, DMZdemilitarizedzoneallowsforsingleseveralhostsnottobefirewalled TEXT, Filteraccordingtotimeofday TEXT, RedirectTCPUDPportsportforwarding TEXT, RedirectIPaddressesforwarding TEXT, FilteraccordingtoUserAuthorization TEXT, TrafficratelimitQoS TEXT, Tarpit TEXT, Log TEXT)' table
[INFO] convert 'https://en.wikipedia.org/wiki/Comparison_of_firewalls' to 'Comparison_of_firewalls_Wikipedia_html5 (Features TEXT, ConfigurationGUItextorbothmodes TEXT, [RemoteAccessWebHTTPTelnetSSHRDPSerialCOMRS232] TEXT, Changeruleswithoutrequiringrestart TEXT, Abilitytocentrallymanageallfirewallstogether TEXT)' table
[INFO] convert 'https://en.wikipedia.org/wiki/Comparison_of_firewalls' to 'Comparison_of_firewalls_Wikipedia_html6 (Features TEXT, Modularitysupportsthirdpartymodulestoextendfunctionality TEXT, [IPS : Intrusion prevention system] TEXT, OpenSourceLicense TEXT, [supports IPv6 ?] TEXT, ClassHomeProfessional TEXT, OperatingSystemsonwhichitruns TEXT)' table
[INFO] convert 'https://en.wikipedia.org/wiki/Comparison_of_firewalls' to 'Comparison_of_firewalls_Wikipedia_html7 (Can TEXT, [NAT44staticdynamicwoportsPAT] TEXT, [NAT64NPTv6] TEXT, IDSIntrusionDetectionSystem TEXT, VPNVirtualPrivateNetwork TEXT, AVAntiVirus TEXT, Sniffer TEXT, Profileselection TEXT)' table

For more information

More examples are available at http://sqlitebiter.rtfd.io/en/latest/pages/usage/index.html

Installation

Install via pip (recommended)

sqlitebiter can be installed via pip (Python package manager).

sudo pip install sqlitebiter

Python package dependencies are as follows.

Mandatory Python package dependencies

Mandatory Python packages are automatically installed during sqlitebiter installation via pip.

Google Sheets dependencies

Manual installation required to use Google Sheets feature.

Test dependencies

Misc

  • lxml (Faster HTML convert if installed)

Installing executable file in Windows

  1. Navigate to https://github.com/thombashi/sqlitebiter/releases
  2. Download the latest version of the sqlitebiter_win_x64.zip
  3. Unzip the file

Documentation

http://sqlitebiter.rtfd.io/