No description or website provided.
Ruby
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Failed to load latest commit information.
lib/fluent/plugin
test/fluent/plugin
.gitignore
Gemfile
LICENSE.txt
README.md
Rakefile
fluent-plugin-sqlite3.gemspec

README.md

README fluent-plugin-sqlite3

Fluent::Sqlite3Output

Required gems

  • sqlite3

Getting started

Add a following directive to fluent.conf and launch fluentd.

<match sqlite3.**>
  type sqlite3
  path test.db
</match>

Then run next.

echo '{"name":"Jhon","age":24}' | fluent-cat sqlite3.persons
echo '{"name":"Mike","age":27}' | fluent-cat sqlite3.persons

Let's see the database.

sqlite3 test.db "select * from persons"
1|Jhon|24
2|Mike|27

Ad hoc mode

Getting started is in ad hoc mode.

  • tables are created when it's needed
  • table names are the 2nd later part of tag, so don't contain illegal characters for table name
  • primary key is id and auto increment
  • column names are corresponding to all keys for given json

Strict mode

If a schema you use is already fixed, you can use strict mode.

In the case, table and columns parameters should be used.

<match sqlite3.**>
  type    sqlite3
  path    test.db
  table   persons
  columns name, age, country
</match>

Optional parameters

  • includes: is in order to insert the columns you need
  • excludes: is in order to filter out some keys you don't need to insert

Configuration

<match sqlite3.**>
  type sqlite3
  path test.db

  ## strict mode
  ## -----------
  ## Expects the table already exists and it has the columns.
  #table   persons
  #columns name, age, country

  ## ad hoc mode
  ## -----------
  ## this mode is enabled in case table and columns
  ## parameters are empty.
  ##
  ## echo '{"name":"Jhon","age":24}' | sqlite3.persons
  ##
  ## This creates a new table PERSONS which
  ## has "name" and "age" columns.
  ## The primary key is "id" which is AUTOINCREMENT.

  ## includes keys of json when inserting a new record
  #includes name, eye
  ## excludes keys of json when inserting a new record
  #excludes hair, eye

  ## fluentd built-in options
  buffer_type    memory
  flush_interval 1s
  retry_limit    1
  retry_wait     1s
  num_threads    1
</match>

License

MIT License