Athena input plugins for Embulk
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
config/checkstyle
gradle/wrapper
lib/embulk/input
src
.gitignore
Dockerfile
LICENSE
README.md
build.gradle
docker-compose.yml
gradlew
gradlew.bat

README.md

Athena input plugin for Embulk

Gem Version MIT License

//TODO: https://github.com/dtan4/terraforming

Athena input plugin for Embulk loads records from Athena(AWS).

Overview

  • Plugin type: input
  • Resume supported: no
  • Cleanup supported: no
  • Guess supported: no

Configuration

  • driver_path: path to the jar file of the Athena JDBC driver. If not set, the bundled JDBC driver(AthenaJDBC41-1.1.0.jar) will be used. (string)
  • database: description (string, required)
  • athena_url: description (string, required)
  • s3_staging_dir: The S3 location to which your query output is written, for example s3://query-results-bucket/folder/. (string, required)
  • access_key: description (string, required)
  • secret_key: description (string, required)
  • query: description (string, required)
  • columns: description (string, required)
  • options: description (string, default: {})

Example

in:
  type: athena
  database: log_test
  athena_url: "jdbc:awsathena://athena.ap-northeast-1.amazonaws.com:443"
  s3_staging_dir: "s3://aws-athena-query-results-11111111111-ap-northeast-1/"
  access_key: ""
  secret_key: ""
  query: |
    select uid, created_at from log_test.sample
  columns:
    - {name: uid, type: string}
    - {name: created_at, type: timestamp}

Build

$ docker-compose up -d
$ docker-compose exec embulk bash
embulk>$ ./gradlew gem  # -t to watch change of files and rebuild continuously