Fluentd output plugin which can write to PostgreSQL hstore extension.
Ruby
Latest commit 564e29d Apr 4, 2017 @shirou shirou version 0.2.7

README.rdoc

fluent-plugin-pghstore

Component

PgHStoreOutput

Output to PostgreSQL hstore database.

Output table should have tag, time and record column.:

CREATE TABLE #{tablename} (
  tag TEXT[],
  time TIMESTAMP WITH TIME ZONE,
  record HSTORE
);

A tag is splited by “.” and stored in the TEXT[]. (Yes, postgres can handle an array!)

Requirement

  • PostgreSQL 9.0 or higher

    • hstore changed at PostgreSQL 9.0.

  • postgres-contrib

  • ruby-pg

How to Install hstore

hstore is in the contrib.

9.1 or higher:

psql <dbname> -c "CREATE EXTENSION hstore;"

9.0:

psql <dbname> -f ${PGHOME}/share/contrib/hstore.sql

Configuration

Example:

<match apache.*>
  type pghstore
  database test
  table test
  table_option CREATE INDEX time_index ON testb (time);
</match>

Options

  • Required

    • database

      • database name

  • Optional

    • table

      • tablename.

      • If not set, use fluentd_store.

      • If not exists, creates automatically.

    • host

    • port

    • user

    • password

    • table_option

      • Add some SQL. This SQL is called only once when table is created from this plugin.

Limitation

  • Nested output is not allowd.

  • Since using only one connection, the performance may become bad. When you meet this, use connection pooling and write patch!

Thanks

This source code is mainly borrowed from

{fluent-plugin-datacounter}{rubygems.org/gems/fluent-plugin-datacounter}. Thank you for tagomoris.

Copyright

Copyright

Copyright © 2012- Shirou WAKAYAMA

License

Apache License

ChangeLog

  • 0.2.2 / 2013-04-20

0.2.1 change caused error in 9.2. 0.2.1 change was not necessary. Thank you kaz29!

  • 0.2.1 / 2012-09-11

PostgreSQL 9.2 {removes hstore's => operator}{wiki.postgresql.org/wiki/What’s_new_in_PostgreSQL_9.2#Remove_hstore.27s_.3D.3E_operator}. This plugin checks the server's version and change the operator.

But not fully tested yet.