Indexes Stellar Core data into Elasticsearch
Java
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
gradle/wrapper
lib
src/main
.gitignore
LICENSE
README.md
build.gradle
config.properties.example
gradlew
gradlew.bat
settings.gradle

README.md

Stellar Core Elasticsearch Indexer

Overview

This project reads the stellar-core database txhistory table, de-serializes the XDR data structures, and then sends the data to an elasticsearch cluster. It is intended to be used with Kibana to visualize, analyze and explore the stream of transaction/operation events.

Configuration

Configuration is done within a properties file.

By default, it will try to find a config.properties file in the current directory. See the config.properties.example file for an example configuration.

To use an alternative location, provide a file location to parameter before running.

E.g. java -jar stellar-core-es-indexer.jar ../path/to/myconfig.properties

Configuration Values

REFRESH_INTERVAL

The number of seconds to wait before polling the database for new ledgers. If set to 0, indexing will only run once. Default Value is 0.

LAST_LEDGER_INDEXED

The indexing will start with a query for the txhistory rows with a legerseq greater than LAST_LEDGER_INDEXED. The log output contains the last ledgerseq that was indexed which could be used with this configuration to pause/resume indexing. To do a full indexing, this could be set to 0.

Building

Run ./gradlew shadowJar

This builds a runnable, shaded jar into build/libs/stellar-core-es-indexer.jar.

Alternatively run ./gradlew distShadowZip to get a zipped distribution in the /distributions folder which also contains bin scripts for running.

Running

java -jar build/libs/stellar-core-es-indexer.jar java -jar build/libs/stellar-core-es-indexer.jar ../path/to/myconfig.properties

License

See LICENSE file.

TODO

  • Index more operations
  • Document json objects
  • Example Queries

Elasticsearch Data Examples

Transaction:

"_id": "3389e9f0f1a65f19736cacf544c2e825313e8447f569233bb8db39aa607c8889"
"_type": "transaction"

   {
     "status": "txSUCCESS",
     "created_at": "2015-09-30T17:15:54.000Z",
     "source_account": "GAAZI4TCR3TY5OJHCTJC2A4QSY6CJWJH5IAJTGKIN2ER7LBNVKOCCWN7"
     "operation_count": 2
   }

Payment

"_id": "3389e9f0f1a65f19736cacf544c2e825313e8447f569233bb8db39aa607c8889-0" -0 is the index of the operation within the transaction.

"_type": "payment"

   {  
      "status":"txSUCCESS",
      "created_at":"2015-09-30T17:15:54.000Z",
      "amount":9.999999995999997E10,
      "source_account":"GAAZI4TCR3TY5OJHCTJC2A4QSY6CJWJH5IAJTGKIN2ER7LBNVKOCCWN7",
      "to":"GALPCCZN4YXA3YMJHKL6CVIECKPLJJCTVMSNYWBTKJW4K5HQLYLDMZTB"
   }

Create Account

"_id": "3389e9f0f1a65f19736cacf544c2e825313e8447f569233bb8db39aa607c8889-0"
"_type": "create-account"

{  
   "status":"txSUCCESS",
   "created_at":"2015-09-30T17:15:54.000Z",
   "starting_balance":20.0,
   "source_account":"GAAZI4TCR3TY5OJHCTJC2A4QSY6CJWJH5IAJTGKIN2ER7LBNVKOCCWN7",
   "account":"GALPCCZN4YXA3YMJHKL6CVIECKPLJJCTVMSNYWBTKJW4K5HQLYLDMZTB"
}