[Microsoft SQL Server] metrics for Always On Availability Groups by jakubgalecki0 · Pull Request #16759 · elastic/integrations

jakubgalecki0 · 2026-01-05T16:03:19Z

Proposed commit message

Add new data_stream for health metrics on Always On Availability Groups.

TSDB test

You're testing with version 8.19.0.

Testing data stream metrics-microsoft_sqlserver.availability_groups-default.

Index being used for the documents is .ds-metrics-microsoft_sqlserver.availability_groups-default-2026.01.26-000001.
Index being used for the settings and mappings is .ds-metrics-microsoft_sqlserver.availability_groups-default-2026.01.26-000002.

The time series fields for the TSDB index are:
	- dimension (13 fields):
		- agent.id
		- cloud.account.id
		- cloud.availability_zone
		- cloud.instance.id
		- cloud.provider
		- cloud.region
		- container.id
		- host.name
		- mssql.metrics.group_id
		- mssql.metrics.name
		- mssql.metrics.primary_replica
		- mssql.metrics.server_name
		- service.address
	- gauge (3 fields):
		- mssql.metrics.primary_recovery_health
		- mssql.metrics.secondary_recovery_health
		- mssql.metrics.synchronization_health
	- routing_path (13 fields):
		- agent.id
		- cloud.account.id
		- cloud.availability_zone
		- cloud.instance.id
		- cloud.provider
		- cloud.region
		- container.id
		- host.name
		- mssql.metrics.group_id
		- mssql.metrics.name
		- mssql.metrics.primary_replica
		- mssql.metrics.server_name
		- service.address

Index tsdb-index-enabled successfully created.

Copying documents from .ds-metrics-microsoft_sqlserver.availability_groups-default-2026.01.26-000001 to tsdb-index-enabled...
All 29 documents taken from index .ds-metrics-microsoft_sqlserver.availability_groups-default-2026.01.26-000001 were successfully placed to index tsdb-index-enabled.

Checklist

I have reviewed tips for building integrations and this pull request is aligned with them.
I have verified that all data streams collect metrics or logs.
I have added an entry to my package's changelog.yml file.
I have verified that Kibana version constraints are current according to guidelines.
I have verified that any added dashboard complies with Kibana's Dashboard good practices

Author's Checklist

[ ]

How to test this PR locally

Related issues

Closes #40607

Screenshots

elastic-vault-github-plugin-prod · 2026-01-05T16:27:01Z

🚀 Benchmarks report

Package `microsoft_sqlserver` 👍(1) 💚(1) 💔(1)

Expand to view

Data stream	Previous EPS	New EPS	Diff (%)	Result
`performance`	4975.12	3846.15	-1128.97 (-22.69%)	💔

To see the full report comment with /test benchmark fullreport

muthu-mps · 2026-01-12T14:48:05Z

packages/microsoft_sqlserver/data_stream/availability_groups/fields/fields.yml

+        - name: primary_recovery_health
+          type: long
+          metric_type: gauge
+          description: Primary replica recovery health (0 = ONLINE_IN_PROGRESS, 1 = ONLINE. NULL if not primary).


Is this field going to get NULL value? Should the field data type changed to accept NULL value?

same applies for seconday_recovery_health field.

Yeah values can be NULL depending on replica we are connected to (primary/secondary). Adjusted data type to keyword.

As per the Microsoft document, these fields are of type integer. I don't think we would get NULL values for these fields. Can you double check once?

As I understand it according to the documentation, primary_recovery_health will be NULL if the host is a secondary replica, and likewise, secondary_recovery_health will be NULL if the host is the primary replica.

mykola-elastic · 2026-01-20T14:50:37Z

packages/microsoft_sqlserver/changelog.yml

@@ -1,4 +1,9 @@
 # newer versions go on top
+- version: "2.15.1"


Suggested change

- version: "2.15.1"

- version: "2.16.0"

Shall we increase the minor version instead of patch for the enhancement?

updated version

mykola-elastic · 2026-01-20T14:53:17Z

packages/microsoft_sqlserver/manifest.yml

 name: microsoft_sqlserver
 title: "Microsoft SQL Server"
-version: "2.15.0"
+version: "2.15.1"


Suggested change

version: "2.15.1"

version: "2.16.0"

ishleenk17 · 2026-01-20T17:49:03Z

packages/microsoft_sqlserver/data_stream/availability_groups/fields/agent.yml

+      example: "stretch"
+      description: >
+        OS codename, if any.
+


None of the fields here are dimension fields. So we need not define them here.

ishleenk17 · 2026-01-20T17:49:58Z

packages/microsoft_sqlserver/data_stream/availability_groups/manifest.yml

+    title: Microsoft SQL Server Always On Availability Groups metrics
+    description: Collect Microsoft SQL Server Always On Availability Groups metrics. Monitors overall AG health, synchronization status, and primary/secondary recovery state.
+elasticsearch:
+  index_mode: "time_series"


Do we need to do TSDB testing for this ?

result of tsdb testing

You're testing with version 8.19.0. Testing data stream metrics-microsoft_sqlserver.availability_groups-default. Index being used for the documents is .ds-metrics-microsoft_sqlserver.availability_groups-default-2026.01.26-000001. Index being used for the settings and mappings is .ds-metrics-microsoft_sqlserver.availability_groups-default-2026.01.26-000002. The time series fields for the TSDB index are: - dimension (13 fields): - agent.id - cloud.account.id - cloud.availability_zone - cloud.instance.id - cloud.provider - cloud.region - container.id - host.name - mssql.metrics.group_id - mssql.metrics.name - mssql.metrics.primary_replica - mssql.metrics.server_name - service.address - gauge (3 fields): - mssql.metrics.primary_recovery_health - mssql.metrics.secondary_recovery_health - mssql.metrics.synchronization_health - routing_path (13 fields): - agent.id - cloud.account.id - cloud.availability_zone - cloud.instance.id - cloud.provider - cloud.region - container.id - host.name - mssql.metrics.group_id - mssql.metrics.name - mssql.metrics.primary_replica - mssql.metrics.server_name - service.address Index tsdb-index-enabled successfully created. Copying documents from .ds-metrics-microsoft_sqlserver.availability_groups-default-2026.01.26-000001 to tsdb-index-enabled... All 29 documents taken from index .ds-metrics-microsoft_sqlserver.availability_groups-default-2026.01.26-000001 were successfully placed to index tsdb-index-enabled.

muthu-mps · 2026-01-21T05:02:28Z

packages/microsoft_sqlserver/data_stream/availability_groups/fields/fields.yml

+          type: keyword
+          description: Server name of the current primary replica.
+        - name: synchronization_health
+          type: keyword


Update the field type to int type. Reference - link

gpop63 · 2026-01-21T23:29:12Z

packages/microsoft_sqlserver/data_stream/availability_groups/agent/stream/stream.yml.hbs

@@ -0,0 +1,31 @@
+metricsets: ["query"]
+# Specify hosts in the below format. TODO: hosts need to be updated to support multiple entries.


This comment can be removed, its probably from beats

Copilot

Pull request overview

This PR adds a new data stream for monitoring Always On Availability Groups in Microsoft SQL Server, providing health and synchronization metrics for high availability configurations.

Changes:

Added availability_groups data stream with metrics for AG health, synchronization status, and replica recovery states
Updated package version from 2.15.0 to 2.16.0
Added comprehensive documentation for the new data stream including prerequisites and field descriptions

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
packages/microsoft_sqlserver/manifest.yml	Bumped package version to 2.16.0
packages/microsoft_sqlserver/docs/README.md	Added documentation for availability_groups data stream with prerequisites and field reference
packages/microsoft_sqlserver/data_stream/availability_groups/sample_event.json	Provided sample JSON event showing AG metrics structure
packages/microsoft_sqlserver/data_stream/availability_groups/manifest.yml	Defined data stream configuration with TSDB index mode
packages/microsoft_sqlserver/data_stream/availability_groups/fields/fields.yml	Defined metric fields including dimensions and gauges for AG monitoring
packages/microsoft_sqlserver/data_stream/availability_groups/fields/ecs.yml	Configured ECS dimension fields for TSDB compatibility
packages/microsoft_sqlserver/data_stream/availability_groups/fields/base-fields.yml	Defined base data stream fields
packages/microsoft_sqlserver/data_stream/availability_groups/elasticsearch/ingest_pipeline/default.yml	Created ingest pipeline for processing AG metrics
packages/microsoft_sqlserver/data_stream/availability_groups/agent/stream/stream.yml.hbs	Defined SQL query template for collecting AG metrics
packages/microsoft_sqlserver/changelog.yml	Added changelog entry for version 2.16.0
packages/microsoft_sqlserver/_dev/build/docs/README.md	Updated generated documentation with AG data stream information

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-02-06T09:50:23Z

packages/microsoft_sqlserver/docs/README.md

+**Prerequisites**: To collect Availability Groups metrics, ensure the following:
+
+1. Always On Availability Groups feature is enabled on the SQL Server instance.
+2. The user account configured for the integration has `VIEW SERVER STATE` and `VIEW ANY DEFINITION` permissions. *Additionaly look at secion [Microsoft SQL Server permissions](#microsoft-sql-server-permissions)*.


Corrected spelling of 'Additionaly' to 'Additionally' and 'secion' to 'section'.

Suggested change

2. The user account configured for the integration has `VIEW SERVER STATE` and `VIEW ANY DEFINITION` permissions. *Additionaly look at secion [Microsoft SQL Server permissions](#microsoft-sql-server-permissions)*.

2. The user account configured for the integration has `VIEW SERVER STATE` and `VIEW ANY DEFINITION` permissions. *Additionally look at section [Microsoft SQL Server permissions](#microsoft-sql-server-permissions)*.

Copilot · 2026-02-06T09:50:24Z

packages/microsoft_sqlserver/_dev/build/docs/README.md

+**Prerequisites**: To collect Availability Groups metrics, ensure the following:
+
+1. Always On Availability Groups feature is enabled on the SQL Server instance.
+2. The user account configured for the integration has `VIEW SERVER STATE` and `VIEW ANY DEFINITION` permissions. *Additionaly look at secion [Microsoft SQL Server permissions](#microsoft-sql-server-permissions)*.


Corrected spelling of 'Additionaly' to 'Additionally' and 'secion' to 'section'.

Suggested change

2. The user account configured for the integration has `VIEW SERVER STATE` and `VIEW ANY DEFINITION` permissions. *Additionaly look at secion [Microsoft SQL Server permissions](#microsoft-sql-server-permissions)*.

2. The user account configured for the integration has `VIEW SERVER STATE` and `VIEW ANY DEFINITION` permissions. *Additionally look at section [Microsoft SQL Server permissions](#microsoft-sql-server-permissions)*.

Copilot · 2026-02-06T09:50:24Z

packages/microsoft_sqlserver/changelog.yml

 # newer versions go on top
+- version: "2.16.0"
+  changes:
+    - description: Add health metrics for Always On Availability Groups.  


Remove trailing whitespace at the end of the description line.

Suggested change

- description: Add health metrics for Always On Availability Groups.

- description: Add health metrics for Always On Availability Groups.

muthu-mps · 2026-02-06T09:56:18Z

packages/microsoft_sqlserver/data_stream/availability_groups/manifest.yml

+title: "Microsoft SQL Server Always On Availability Groups metrics"
+type: metrics
+streams:
+  - input: sql/metrics


Can you add enabled:false flag, so that this datastream is not enabled by default for data collection?

Added, Availability Groups metrics are disabled by default

muthu-mps

LGTM!

elasticmachine · 2026-02-06T10:42:27Z

💚 Build Succeeded

Buildkite Build
Commit: ab386c7

History

💚 Build #37584 succeeded 9574243
💚 Build #37193 succeeded 6790e27
💚 Build #36917 succeeded 41a66d8
💚 Build #36896 succeeded 80ad445
💔 Build #36893 failed 662489d
💔 Build #36847 failed 9d07ec8

elastic-vault-github-plugin-prod · 2026-02-06T11:19:35Z

Package microsoft_sqlserver - 2.16.0 containing this change is available at https://epr.elastic.co/package/microsoft_sqlserver/2.16.0/

jakubgalecki0 added 4 commits December 31, 2025 17:49

create new data_stream for always_on_availability

fd0b4ed

Merge branch 'elastic:main' into mssql-always-on

1ae4fc0

Merge branch 'elastic:main' into mssql-always-on

ad126fe

add replicas and databases metrics

8c82aa1

andrewkroh added the Integration:microsoft_sqlserver Microsoft SQL Server label Jan 8, 2026

jakubgalecki0 added 2 commits January 8, 2026 17:52

drop replicas and databases metrics

e3d4d0d

add sample_event

37c9f9f

jakubgalecki0 force-pushed the mssql-always-on branch from bc08f4f to 37c9f9f Compare January 9, 2026 13:41

andrewkroh added the documentation Improvements or additions to documentation. Applied to PRs that modify *.md files. label Jan 9, 2026

muthu-mps reviewed Jan 12, 2026

View reviewed changes

jakubgalecki0 added 2 commits January 13, 2026 19:45

change field type

d9d06b9

bump version

2bda7d1

jakubgalecki0 marked this pull request as ready for review January 14, 2026 11:02

jakubgalecki0 requested a review from a team as a code owner January 14, 2026 11:02

andrewkroh added the Team:Obs-InfraObs Observability Infrastructure Monitoring team [elastic/obs-infraobs-integrations] label Jan 14, 2026

muthu-mps requested a review from alaudazzi January 16, 2026 10:55

mykola-elastic reviewed Jan 20, 2026

View reviewed changes

mykola-elastic approved these changes Jan 20, 2026

View reviewed changes

bump version

3611abb

ishleenk17 reviewed Jan 20, 2026

View reviewed changes

muthu-mps reviewed Jan 21, 2026

View reviewed changes

gpop63 reviewed Jan 21, 2026

View reviewed changes

jakubgalecki0 added 6 commits January 23, 2026 18:38

change fields type

9d07ec8

Merge branch 'main' into mssql-always-on

662489d

remove agent.yml

80ad445

add dimension fields

41a66d8

Merge branch 'main' into mssql-always-on

6790e27

Merge branch 'main' into mssql-always-on

9574243

muthu-mps requested a review from Copilot February 6, 2026 09:49

Copilot AI reviewed Feb 6, 2026

View reviewed changes

muthu-mps reviewed Feb 6, 2026

View reviewed changes

fix typos & make disabled by default

ab386c7

muthu-mps approved these changes Feb 6, 2026

View reviewed changes

jakubgalecki0 merged commit 641ca79 into elastic:main Feb 6, 2026
11 checks passed

		@@ -1,4 +1,9 @@
		# newer versions go on top
		- version: "2.15.1"

		@@ -0,0 +1,31 @@
		metricsets: ["query"]
		# Specify hosts in the below format. TODO: hosts need to be updated to support multiple entries.

	2. The user account configured for the integration has `VIEW SERVER STATE` and `VIEW ANY DEFINITION` permissions. Additionaly look at secion [Microsoft SQL Server permissions](#microsoft-sql-server-permissions).
	2. The user account configured for the integration has `VIEW SERVER STATE` and `VIEW ANY DEFINITION` permissions. Additionally look at section [Microsoft SQL Server permissions](#microsoft-sql-server-permissions).

	- description: Add health metrics for Always On Availability Groups.
	- description: Add health metrics for Always On Availability Groups.

Conversation

jakubgalecki0 commented Jan 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Proposed commit message

Checklist

Author's Checklist

How to test this PR locally

Related issues

Screenshots

Uh oh!

elastic-vault-github-plugin-prod bot commented Jan 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🚀 Benchmarks report

Package microsoft_sqlserver 👍(1) 💚(1) 💔(1)

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gpop63 Jan 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Feb 6, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 6, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 6, 2026

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

muthu-mps left a comment

Choose a reason for hiding this comment

Uh oh!

elasticmachine commented Feb 6, 2026

💚 Build Succeeded

History

Uh oh!

Uh oh!

elastic-vault-github-plugin-prod bot commented Feb 6, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

jakubgalecki0 commented Jan 5, 2026 •

edited

Loading

elastic-vault-github-plugin-prod bot commented Jan 5, 2026 •

edited

Loading

Package `microsoft_sqlserver` 👍(1) 💚(1) 💔(1)

gpop63 Jan 21, 2026 •

edited

Loading