Couchbase SQL Importer
This tool allows you to copy the content of a your tables into Couchbase.
- Import all rows as JSON document. All table columns are JSON attributes.
- Optionnaly it is possible to create views, the import tool create 2 types of views:
- A view all/by_type that allows to query document by type (table)
- Views for each type (table) to query on Primary Key (type/by_pk)
- All the views are using the _count reduce function.
If you do not want to build it from source, you can download the JAR here
Configure an import.properties file with all the parameters
## SQL Information ## sql.connection=jdbc:mysql://192.168.99.19:3306/world?zeroDateTimeBehavior=convertToNull sql.username=root sql.password=password ## Couchbase Information ## cb.uris=http://localhost:8091/pools cb.bucket=default cb.password= ## Import information import.tables=ALL import.createViews=true import.typefield=type import.fieldcase=lower
Download the JDBC driver for your database
Run the following command
java -cp "./CouchbaseSqlImporter.jar:./mysql-connector-java-5.1.25-bin.jar" com.couchbase.util.SqlImporter import.properties
This tool is a small utility that I have developed in few hours as a demonstrator.
I have tested the tool mainly on MySQL with some sample schemas: World (few hundreds of records), Employees (4 millions records); but I have not tried to optimize the import "speed" (for example multithreading, divide select is chunks, ...).
I addition to the tests that I need to do with multiple databases types, I also need to create a test suite.
- Implement a test suite
- Provide "Report/Logging"
- Create Document keys using PK columns
- Error/Log for long keys (>250char)
- Test multiple databases, at least Oracle