Skip to content
Permalink
Branch: master
Find file Copy path
Find file Copy path
Fetching contributors…
Cannot retrieve contributors at this time
232 lines (161 sloc) 7.18 KB

QingStor

Description

This input plugin enables Logstash to receive events from the QingCloud QingStor service.

The following example shows how to configure Logstash with a minimal configuration options to fetch files from QingStor and to output into another bucket in QingStor:

input {
  qingstor {
    access_key_id => 'your_access_key_id'           #required
    secret_access_key => 'your_secret_access_key'   #required
    bucket => 'source_bucket'                       #required
  }
}

output {
  qingstor {
    access_key_id => 'your_access_key_id'           #required
    secret_access_key => 'your_secret_access_key'   #required
    bucket => 'destined_bucket'                     #required
  }
}

QingStor output Configuration Options

This plugin supports the following configuration options plus the [plugins-outputs-common-options] described later.

Setting Input type Required

access_key_id

string

Yes

secret_access_key

string

Yes

host

string

No

port

number

No

bucket

string

Yes

region

string

No

prefix

string

No

tmpdir

string

No

tags

array

No

encoding

string one of ["gzip", "none"]

No

rotation_strategy

string one of ["size_and_time", "size", "time"]

No

file_size

number

No

file_time

number

No

upload_workers_count

number

No

upload_queue_size

number

No

server_side_encryption_algorithm

string one of ["AES256", "none"]

No

customer_key

string

No

restore

boolean

No

Also see [plugins-outputs-common-options] for a list of options supported by all output plugins.

 

access_key_id
  • This is a required setting.

  • Value type is string

  • There is no default value for this setting.

The key id to access your QingStor.

secret_access_key
  • This is a required setting.

  • Value type is string

  • There is no default value for this setting.

The key to access your QingStor.

host
  • Value type is string

  • Default value is nil

The QingStor service IP address if you want to redirect.

port
  • Value type is number

  • Default value is 443.

The QingStor service IP address port which coordinate with host.

bucket
  • This is a required setting.

  • Value type is string

  • There is no default value for this setting.

The name of the qingstor bucket.

region
  • Value type is string

  • There is no default value for this setting.

The region of the qingstor bucket.

prefix
  • Value type is string

  • Default value is nil.

The prefix to filter the files.

tmpdir
  • Value type is string

  • Default value is /tmp/logstash2qingstor in Linux System.

Set the directory where logstash store the tmp files before sending it to logstash, default directory in linux is /tmp/logstash2qingstor.

tags
  • Value type is array

  • Default value is []

Define tags to append to the file on the qingstor bucket

encoding
  • Value type is string, one of ["gzip", "none"]

  • Default value is none

Specify the content encoding. Supports "gzip".

rotation_strategy
  • Value type is string, one of ["size_and_time", "size", "time"]

  • Default value is size_and_time.

Define the strategy to use to decide when we need to rotate the file and push it to QingStor,

The default strategy is to check for both size and time, the first one to match will rotate the file.

file_size
  • Value type is number

  • Default value is 1024 * 1024 * 5.

Define the size requirement for each file to upload to qingstor. In byte.

file_time
  • Value type is number

  • Default value is 15.

Define the time interval for each file to upload to qingstor. In minutes.

upload_workers_count
  • Value type is number

  • Default value is maximal processors / 2

Specify maximum number of workers to use to upload the files to QingStor

upload_queue_size
  • Value type is number

  • Default value is maximal processors / 2

Number of items we can keep in the local queue before uploading them

server_side_encryption_algorithm
  • Value type is string, one of ["AES256", "none"]

  • Default value is none.

Specifies what type of encryption to use when SSE is enabled.

customer_key
  • Value type is string

  • There is no default value for this setting.

Specifies the encryption customer key that would be used in server side

restore
  • Value type is boolean

  • Default value is false

Specifies if set to true, it would upload existing file in targeting folder at the beginning.

You can’t perform that action at this time.