Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sql_input: Add merge_results feature to SQL Input package #6922

Merged
merged 7 commits into from Jul 14, 2023
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
1 change: 1 addition & 0 deletions packages/sql_input/agent/input/input.yml.hbs
Expand Up @@ -7,5 +7,6 @@ driver: {{driver}}
sql_queries: {{sql_queries}}
raw_data.enabled: true
period: {{period}}
merge_results: {{merge_results}}
data_stream:
dataset: {{data_stream.dataset}}
5 changes: 5 additions & 0 deletions packages/sql_input/changelog.yml
@@ -1,4 +1,9 @@
# newer versions go on top
- version: "0.3.0"
changes:
- description: Add merge_results feature
type: enhancement
link: https://github.com/elastic/integrations/pull/6922
- version: "0.2.1"
changes:
- description: Add system test cases.
Expand Down
22 changes: 22 additions & 0 deletions packages/sql_input/docs/README.md
Expand Up @@ -81,3 +81,25 @@ Expects any number of columns. This mode generates a single event for each row.

For more examples of response format pelase refer [here](https://www.elastic.co/guide/en/beats/metricbeat/current/metricbeat-module-sql.html)


### Merge Results
ishleenk17 marked this conversation as resolved.
Show resolved Hide resolved
Merge multiple queries into a single event.

Multiple queries will create multiple events, one for each query. It may be preferable to create a single event by combining the metrics together in a single event.

This feature can be enabled using the `merge_results` config.

`merge_results` can merge queries having response format as "variable".
However, for queries with a response format as "table", a merge is possible only if each table query produces a single row.

For example, if we have 2 queries as below for PostgreSQL:

sql_queries:
- query: "SELECT blks_hit,blks_read FROM pg_stat_database LIMIT 1;"
response_format: table

- query: "SELECT checkpoints_timed,checkpoints_req FROM pg_stat_bgwriter;"
response_format: table

The `merge_results` feature will create a combined event, where `blks_hit`, `blks_read`, `checkpoints_timed` and `checkpoints_req` are part of the same event.

10 changes: 9 additions & 1 deletion packages/sql_input/manifest.yml
@@ -1,7 +1,7 @@
format_version: 2.0.0
name: sql
title: "SQL Input"
version: "0.2.1"
version: "0.3.0"
description: "Collects Metrics by Quering on SQL Databases"
type: input
categories:
Expand Down Expand Up @@ -48,5 +48,13 @@ policy_templates:
required: true
show_user: true
default: "- query: SHOW GLOBAL STATUS LIKE 'Innodb_system%'\n response_format: variables\n \n"
shmsr marked this conversation as resolved.
Show resolved Hide resolved
- name: merge_results
type: bool
title: Merge Results
multi: false
required: false
show_user: false
default: false
description: Merge results from multiple queries to a single event(restrictions apply)
ishleenk17 marked this conversation as resolved.
Show resolved Hide resolved
owner:
github: elastic/obs-infraobs-integrations