Skip to content

mikoto2000/embulk-input-salesforce_bulk

master
Switch branches/tags

Name already in use

A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch?
Code

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
src
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Salesforce Bulk input plugin for Embulk

Salesforce Bulk API の一括クエリ結果を取得します。

Overview

  • Plugin type: input
  • Resume supported: no
  • Cleanup supported: no
  • Guess supported: no

Configuration

  • userName: Salesforce user name.(string, required)
  • password: Salesforce password.(string, required)
    • Set a string that concatenated the password and security token.
  • authEndpointUrl: Salesforce login endpoint URL.(string, default is "https://login.salesforce.com/services/Soap/u/39.0")
  • objectType: object type of JobInfo.(string, required)
    • Usually same as query's object.(If querySelectFrom is (snip) FROM xxx (snip) then dataType is xxx)
  • pollingIntervalMillisecond: polling interval millisecond.(string, default is 30000)
  • querySelectFrom: part of query. SELECT and FROM.(string, required)
  • queryWhere: part of query. WHERE.(string, default is "")
  • queryOrder: part of query. ORDER BY.(string, default is "")
  • columns: schema config.(SchemaConfig, required)
  • startRowMarkerName: 開始レコードを特定するための目印とするカラム名を指定する.(String, default is null)
  • start_row_marker: 抽出条件に、『カラム「startRowMarkerName」がこの値よりも大きい』を追加する.(string, default is null)
  • queryAll: if true, uses the queryAll operation so that deleted records are returned.(boolean, default is false)

More information about objectType:

objectType is field of JobInfo. See: JobInfo | Bulk API Developer Guide | Salesforce Developers

These documents will aid in understanding.

Example

query で指定したものをすべて抽出

in:
  type: salesforce_bulk
  userName: USER_NAME
  password: PASSWORD
  authEndpointUrl: https://login.salesforce.com/services/Soap/u/39.0
  objectType: Account
  pollingIntervalMillisecond: 5000
  querySelectFrom: SELECT Id,Name,LastModifiedDate FROM Account
  queryWhere: Name like 'Test%'
  queryOrder: Name desc
  columns:
  - {type: string, name: Id}
  - {type: string, name: Name}
  - {type: timestamp, name: LastModifiedDate, format: '%FT%T.%L%Z'}

前回取得時点から変更があったオブジェクトのみ取得

startRowMarkerName に LastModifiedDate を指定したうえで、 -o オプションを指定して embulk を実行する。

config.yaml

in:
  type: salesforce_bulk
  userName: USER_NAME
  password: PASSWORD
  authEndpointUrl: https://login.salesforce.com/services/Soap/u/39.0
  objectType: Account
  pollingIntervalMillisecond: 5000
  querySelectFrom: SELECT Id,Name,LastModifiedDate FROM Account
  queryOrder: Name desc
  columns:
  - {type: string, name: Id}
  - {type: string, name: Name}
  - {type: timestamp, name: LastModifiedDate, format: '%FT%T.%L%Z'}
  startRowMarkerName: LastModifiedDate

実行コマンド

embulk run config.yaml -o config.yaml

TODO

  • エラーログ出力を真面目にやる
  • guess 対応
  • 効率化

Build

$ ./gradlew gem