Skip to content
Embulk plugin that loads records from SORACOM Harvest.
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
lib/embulk/input
test
.gitignore
.ruby-version
Gemfile
LICENSE.txt
README.md
Rakefile
embulk-input-soracom_harvest.gemspec

README.md

SORACOM Harvest input plugin for Embulk

SORACOM Harvest is the data store service to store the data collected from IoT devices. This plugin allows you to load data from SORACOM Harvest and load into other data store and RDBMS with other Embulk plugins.

Overview

  • Plugin type: input
  • Resume supported: yes
  • Cleanup supported: yes
  • Guess supported: yes

Configuration

  • auth_key_id: AUTH_KEY that you can get at SORACOM user console (string, required)
  • auth_key: AUTH_KEY_ID that you can get at SORACOM user console (string, required)
  • target: 'harvest' or 'sims'(string, default: 'harvest')
  • filter: filter to when get SIMs(string, default: null)
  • tag_value_match_mode: Tag search mode exact or prefix (string, optional, default: exact)
  • start_datetime: get data time is after this value (works only when target is 'harvest')
  • end_datetime: get data time is after this value (works only when target is 'harvest')
  • retry_limit: Try to retry this times (integer, default: 5)
  • retry_initial_wait_sec: Wait seconds for exponential backoff initial value (integer, default: 2)
  • endpoint: endpoint url of SORACOM API server. e.g. "https://api.soracom.io/v1" (string, default: null)

Example

in:
  type: soracom_harvest
  auth_key_id: keyId-ABCDEFGHIJKLMNOPQRSTUVWXYZ
  auth_key: secret-abcdefghijklmnopqrstuvwxyz
  tartet: harvest
  filter: status: active|ready
  start_datetime: '2016-11-13 06:13:12.034 +0000'
  end_datetime: '2016-12-21 09:00:14.042 +0000'

Usage

  1. Please configure minimum seed config.
  2. Run embulk guess /path/to/seed.yml -o /path/to/config.yml.
    • If you don't have registered SIMs, guess doesn't work.
    • If you don't have records at Harvest, guess doesn't work.
  3. Run embulk preview /path/to/config.yml
  4. Run embulk run /path/to/config.yml

filter

You can filter SIMs when get data by filter option.

This plugin doesn't support multiple filter condition like 'AND' or 'OR'.

IMSI

filter: imsi: 440123456789012

MSISDN

filter: msisdn: 811234567890

Status

filter: status: active
filter: status: active|ready

status value can be taken (active, inactive, ready, instock, shipped, suspended, terminated).

Also accepts multiple vaules separated with |

Speed class

filter: speed_class: s1.minimum
filter: speed_class: s1.minimum|s1.slow

Tag

filter: tag_name: tag_value
tag_value_match_mode: exact # or 'prefix'

You can set tag_value_match_mode. This option can be taken (exact, prefix).

FAQ

Build

$ rake

Development

$ git clone git@github.com:sakama/embulk-input-soracom_harvest.git
$ cd embulk-input-soracom_harvest
$ embulk bundle install --path vendor/bundle
$ embulk run -I ./lib /path/to/config.yml
You can’t perform that action at this time.