Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 6 additions & 8 deletions pipeline/outputs/postgresql.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

[PostgreSQL](https://www.postgresql.org) is an open source database management system that supports the SQL language and is capable of storing both structured and unstructured data, such as JSON objects.

Fluent Bit is designed to work with JSON objects, and the `pgsql` output plugin allows users to send their data to a PostgreSQL database and store it using the `JSONB` type.
The `pgsql` output plugin accepts log records and stores each record body in a PostgreSQL database using the `JSONB` type.

PostgreSQL 9.4 or higher is required.

Expand Down Expand Up @@ -56,19 +56,17 @@ This plugin supports the following parameters:

| Key | Description | Default |
|:----|:------------|:-------|
| `async` | Define if the plugin will use asynchronous or synchronous connections. | `false` |
| `cockroachdb` | Set to `true` if you will connect the plugin with a CockroachDB. | `false` |
| `connection_options` | Specifies any valid [PostgreSQL connection options](https://www.postgresql.org/docs/devel/libpq-connect.html#LIBPQ-CONNECT-OPTIONS). | `_none_` |
| `database` | Database name to connect to. | `fluentbit` |
| `host` | Hostname/IP address of the PostgreSQL instance. | `127.0.0.1` |
| `max_pool_size` | Maximum amount of connections in asynchronous mode. | `4` |
| `min_pool_size` | Minimum number of connections in asynchronous mode. | `1` |
| `password` | Password of PostgreSQL username. | `_none_` |
| `port` | PostgreSQL port. | `5432` |
| `table` | Table name where to store data. | `fluentbit` |
| `timestamp_key` | Key in the JSON object containing the record timestamp. | `date` |
| `user` | PostgreSQL username. | `_none_` |
| `workers` | The number of [workers](../../administration/multithreading.md#outputs) to perform flush operations for this output. | `0` |
| `workers` | The number of [workers](../../administration/multithreading.md#outputs) to perform flush operations for this output. | `1` |

The plugin uses one PostgreSQL connection per worker. The previous `async`, `max_pool_size`, and `min_pool_size` settings are no longer supported.

### Libpq

Expand Down Expand Up @@ -96,7 +94,6 @@ pipeline:
database: fluentbit
table: fluentbit
connection_options: '-c statement_timeout=0'
timestamp_key: ts
```

{% endtab %}
Expand All @@ -113,7 +110,6 @@ pipeline:
Database fluentbit
Table fluentbit
Connection_Options -c statement_timeout=0
Timestamp_Key ts
```

{% endtab %}
Expand All @@ -127,6 +123,8 @@ The output plugin automatically creates a table with the name specified by the `
- `time TIMESTAMP WITHOUT TIMEZONE`
- `data JSONB`

Each flushed log record becomes one row in the destination table. The `time` column is populated from the Fluent Bit event timestamp, and the `data` column stores only the record body as `JSONB`, so a separate `timestamp_key` setting is no longer needed.

The timestamp doesn't contain any information about the time zone, and it's therefore referred to the time zone used by the connection to PostgreSQL (`timezone` setting).

For more information about the `JSONB` data type in PostgreSQL, refer to the [JSON types](https://www.postgresql.org/docs/current/datatype-json.html) page in the official documentation. You can find instructions on how to index or query the objects (including `jsonpath` introduced in PostgreSQL 12).
Expand Down
Loading