Permalink
Fetching contributors…
Cannot retrieve contributors at this time
91 lines (76 sloc) 3.14 KB

Elasticsearch Writer

Build Status

Writer expects that mapping of types and indexes in your Elasticsearch exists. If it is missing and you have enabled automatic index creation, new mapping will be created.

--

Configuration

  • Configuration has 2 parts - elastic and tables
  • The elastic section defines connection info and import config
    • host - server address
    • port - elasticsearch listening port
    • bulkSize (optional) - size of a batch to upload to Elasticsearch (default is 10.000)
    • ssh - SSH tunnel configuration
      • enabled - enable SSH tunnel for connection to Elasticsearch
      • sshHost - address of the SSH server
      • port (optional) - SSH listening port (default is 22)
      • user - SSH login
      • keys
        • private - Your private key used for authentication
  • The tables section defines database tables, their columns and their data types

Example

{
    "elastic": {
        "host": "my.hostname.com",
        "port": 9200,
        "bulkSize": 10000
    },
    "tables": [
        {
            "file": "products.csv",
            "index": "production",
            "type": "products",
            "id": "id",
            "export": true
        }
    ]
}

Example with SSH

{
    "elastic": {
        "host": "my.hostname.com",
        "port": 9200,
        "bulkSize": 10000,
        "ssh": {
            "sshHost": "10.112.1.1",
            "port": 22,
            "user": "extractor",
            "keys": {
                "private": "YOUR_PRIVATE_KEY_WITHOUT_PASSPHRASE"
            }
        }
    },
    "tables": [
        {
            "file": "products.csv",
            "index": "production",
            "type": "products",
            "id": "id",
            "export": true
        }
    ]
}

Configuring in Keboola Connection

Elasticsearch Writer is integrated in Keboola Connection.

image

Available with standard KB Docker Generic UI

image