Skip to content
Fluentd plugin for Treasure Data Service
Branch: master
Clone or download
Latest commit dcf2eaf Nov 27, 2017
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
lib/fluent/plugin v1.0.0 Nov 27, 2017
test
.gitignore use bundler instead of jeweler for packaging Oct 24, 2011
.travis.yml
AUTHORS
ChangeLog
Gemfile use bundler instead of jeweler for packaging Oct 24, 2011
Gemfile.msgpack.0.4 Add msgpack v0.4 test on travis Feb 5, 2015
Gemfile.v0.12
README.rdoc Add use_gzip_command parameter to README Jun 9, 2015
Rakefile
example.conf
fluent-plugin-td.gemspec

README.rdoc

Treasure Data output plugin for Fluentd

This Fluentd output plugin is used to upload logs to Treasure Data using Treasure Data's REST APIs.

This plugin relies on the Treasure Data's Ruby client library to communicate with the Treasure Data's REST APIs.

For more information, please visit the official Fluentd TD Output plugin page at docs.fluentd.org.

Configuration

Please refer to the example.conf example configuration file contained in the root of this repository for some examples. The configuration file allows the user to specify various settings for this output plugin.

The configuration options currently supported are:

apikey

Sets the configuration parameter to specify the Treasure Data API key.

Requires the API key as argument.

This parameter is required.

auto_create_table

Specifies whether to create a database and/or table if the destination database and/or table do not exist.

Requires no argument to enable; one optional boolean (true or false) argument accepted. When this option is enabled, the destination database and table will be determined based on the event tag: e.g. an even with tag 'td.mydb.mytable' will require the plugin to upload data to the 'mydb' database and 'mytable' table within it.

If this option is not used or set to false, the user is required to specify the database and table configuration parameters as explained below.

NOTE

depending on the access control permissions associated to the API key, a database can or cannot be created if not already available. See the Treasure Data Access Control documentation page for details.

database

Specifies the destination database in the Treasure Data cloud.

Requires the database name as argument.

This parameter is required unless the auto_create_table option is used.

table

Specifies the destination table in the above mentioned database within the Treasure Data cloud.

Requires the table name as argument.

This parameter is required unless the auto_create_table option is used.

endpoint

Specifies the Treasure Data's REST API endpoint.

Requires the endpoint as argument.

If not specified, a default is used; please refer to the td-client-ruby’s :endpoint option for more details.

use_ssl

Specifies whether to communicate using SSL encryption over HTTPS.

Requires no argument to enable; one optional boolean (true or false) argument accepted.

This option corresponds to td-client-ruby’s :ssl option: please refer to that page for important information concerning it.

http_proxy

If communication with the Treasure Data API endpoint has to occur through a proxy, you can use this option to configure it.

Requires the proxy URL as argument. Please refer to the td-client-ruby’s :http_proxy option for details.

connect_timeout read_timeout send_timeout

All three options specify the timeout used for every connection, when reading data from the Treasure Data API endpoints, and when sending data to it.

They require the number of seconds for the timeout as argument.

Please refer to the td-client-ruby’s :connect_timeout, :read_timeout, :send_timeout options for details.

tmpdir

Specifies the temporary folder to use for storing the temporarily converted and compressed inputs.

Requires the path to the temporary folder as argument. If not specified, the system default as specified by Ruby's Dir.tmpdir is used (the TMPDIR environment variable is honored).

use_gzip_command

Use external gzip command instead of Ruby's Gzip library. It uses another CPU core so it may improve compression and IO performance.

Copyright

Copyright

Copyright © 2011 Treasure Data, Inc.

License

Apache License, Version 2.0

You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.