Skip to content
This repository

JDBC river for Elasticsearch

Merge pull request #225 from salyh/bulkfixes

RiverMouth.flush() suggest alternative,  RiverMouth.startup() refactored, and minor code cleanups
latest commit 771490b7f9
Jörg Prante authored April 23, 2014
Octocat-spinner-32 bin Fixing bugs, closing #208, #203, #197 March 31, 2014
Octocat-spinner-32 src again 'fixed' problems with random failing testcases - now the hard w… April 23, 2014
Octocat-spinner-32 .gitignore Fixing bugs, closing #208, #203, #197 March 31, 2014
Octocat-spinner-32 .travis.yml travis, maven plugins update October 28, 2013
Octocat-spinner-32 LICENSE.txt 2.3.0 December 10, 2013
Octocat-spinner-32 README.rst 1.1.0.0 April 05, 2014
Octocat-spinner-32 pom.xml 1.1.0.0 April 05, 2014
README.rst
../../../elasticsearch-river-jdbc/raw/master/src/site/resources/database-128.png

Image by icons8 Creative Commons Attribution-NoDerivs 3.0 Unported.

Elasticsearch JDBC river

The Java Database Connection (JDBC) river allows to fetch data from JDBC sources for indexing into Elasticsearch.

It is implemented as an Elasticsearch plugin.

The relational data is internally transformed into structured JSON objects for the schema-less indexing model in Elasticsearch.

Creating a JDBC river is easy. Install the plugin. Download a JDBC driver jar from your vendor's site (here MySQL) and put the jar into the folder of the plugin $ES_HOME/plugins/river-jdbc. Then issue this simple command:

curl -XPUT 'localhost:9200/_river/my_jdbc_river/_meta' -d '{
    "type" : "jdbc",
    "jdbc" : {
        "url" : "jdbc:mysql://localhost:3306/test",
        "user" : "",
        "password" : "",
        "sql" : "select * from orders",
        "index" : "my_jdbc_index",
        "type" : "my_jdbc_type"
    }
}'

Installation

https://travis-ci.org/jprante/elasticsearch-river-jdbc.png

Prerequisites

  • a JDBC driver jar for your database (download from vendor site and put into JDBC river plugin folder)
ES version Plugin Release date Command
0.90.3 0.90.3.1 Jan 31, 2014 ./bin/plugin -install river-jdbc -url http://bit.ly/1emqDH9
0.90.10 0.90.10.2 Jan 31, 2014 ./bin/plugin -install river-jdbc -url http://bit.ly/1a8Mcve
1.0.0 1.0.0.2 Mar 31, 2014 ./bin/plugin --install river-jdbc --url http://bit.ly/1gIk4jW
1.1.0 1.1.0.0 Apr 5, 2014 ./bin/plugin --install river-jdbc --url http://bit.ly/1iadfnF

Do not forget to restart the node after installing.

Project docs

The Maven project site is available at Github

Binaries

Binaries are available at Bintray

Documentation

Attention: working on the documentation for 1.0.0.x is still in progress!

Overview

Quickstart

JDBC river parameters

How to declare a sequence of SQL statements per river run

Strategies

Moving a table

Labeled columns

Structured objects

RiverSource, RiverMouth, RiverFlow

Bulk indexing

Setting up the river with PostgreSQL

License

Elasticsearch JDBC River Plugin

Copyright (C) 2012,2013 Jörg Prante

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Something went wrong with that request. Please try again.