Couchbase SQL Importer
Java
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.

README.md

Couchbase SQL Importer

This tool allows you to copy the content of a your tables into Couchbase.

The import

  • Import all rows as JSON document. All table columns are JSON attributes.
  • Optionnaly it is possible to create views, the import tool create 2 types of views:
    • A view all/by_type that allows to query document by type (table)
    • Views for each type (table) to query on Primary Key (type/by_pk)
    • All the views are using the _count reduce function.

If you do not want to build it from source, you can download the JAR here

Usage

  1. Configure an import.properties file with all the parameters

     ## SQL Information ##
     sql.connection=jdbc:mysql://192.168.99.19:3306/world?zeroDateTimeBehavior=convertToNull
     sql.username=root
     sql.password=password
    
     ## Couchbase Information ##
     cb.uris=http://localhost:8091/pools
     cb.bucket=default
     cb.password=
    
     ## Import information
     import.tables=ALL
     import.createViews=true
     import.typefield=type
     import.fieldcase=lower
    
  2. Download the JDBC driver for your database

  3. Run the following command

     java -cp "./CouchbaseSqlImporter.jar:./mysql-connector-java-5.1.25-bin.jar" com.couchbase.util.SqlImporter import.properties
    

Notes

This tool is a small utility that I have developed in few hours as a demonstrator.

I have tested the tool mainly on MySQL with some sample schemas: World (few hundreds of records), Employees (4 millions records); but I have not tried to optimize the import "speed" (for example multithreading, divide select is chunks, ...).

I addition to the tests that I need to do with multiple databases types, I also need to create a test suite.

To do

  • Implement a test suite
  • Provide "Report/Logging"
  • Create Document keys using PK columns
    • Error/Log for long keys (>250char)
  • Test multiple databases, at least Oracle