Skip to content

Commit

Permalink
Merge branch 'master' of github.com:jprante/elasticsearch-jdbc
Browse files Browse the repository at this point in the history
  • Loading branch information
jprante committed May 27, 2016
2 parents c8bb884 + 3658931 commit 51708cf
Show file tree
Hide file tree
Showing 3 changed files with 37 additions and 5 deletions.
4 changes: 0 additions & 4 deletions README.md
Expand Up @@ -722,10 +722,6 @@ the first time you run the script, it will generate the statefile.json file like
```
after this, you can select incremental data from table.

There is a problem here, the first time you run the script, it can't select any data from table, it have two solutions here:
* in another script, do full-import, later you can use the incremental script to select incremental data
* define a statefile.json file before the first time you run the incremental script, set the lastexecutionstart to 0, so that you can select all the data from table.

## Stored procedures or callable statements

Stored procedures can also be used for fetchng data, like this example fo MySQL illustrates.
Expand Down
36 changes: 36 additions & 0 deletions bin/mysql-blog.sh
@@ -0,0 +1,36 @@
#!/bin/sh

DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
bin=${DIR}/../bin
lib=${DIR}/../lib

echo '
{
"type" : "jdbc",
"jdbc" : {
"url" : "jdbc:mysql://localhost:3306/blog",
"statefile" : "statefile.json",
"schedule" : "0 0-59 0-23 ? * *",
"user" : "blog",
"password" : "12345678",
"sql" : [{
"statement": "select id as _id, id, post_title as title, post_content as content from wp_posts where post_status = ? and post_modified > ? ",
"parameter": ["publish", "$metrics.lastexecutionstart"]}
],
"index" : "article",
"type" : "blog",
"metrics": {
"enabled" : true
},
"elasticsearch" : {
"cluster" : "elasticsearch",
"host" : "localhost",
"port" : 9300
}
}
}
' | java \
-cp "${lib}/*" \
-Dlog4j.configurationFile=${bin}/log4j2.xml \
org.xbib.tools.Runner \
org.xbib.tools.JDBCImporter
Expand Up @@ -1296,7 +1296,7 @@ private void bind(PreparedStatement statement, int i, Object value) throws SQLEx
statement.setString(i, ExceptionFormatter.format(context.getThrowable()));
} else if ("$metrics.lastexecutionstart".equals(s)) {
DateTime dateTime = sourceMetric != null ? sourceMetric.getLastExecutionStart() : null;
statement.setTimestamp(i, dateTime != null ? new Timestamp(dateTime.getMillis()) : null);
statement.setTimestamp(i, dateTime != null ? new Timestamp(dateTime.getMillis()) : new Timestamp(new DateTime(0).getMillis()));
} else if ("$metrics.lastexecutionend".equals(s)) {
DateTime dateTime = sourceMetric != null ? sourceMetric.getLastExecutionEnd() : null;
statement.setTimestamp(i, dateTime != null ? new Timestamp(dateTime.getMillis()) : null);
Expand Down

0 comments on commit 51708cf

Please sign in to comment.