Skip to content

Add additional troubleshooting documentation for sqlserver connection issues #16080

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Dec 5, 2022
Merged

Add additional troubleshooting documentation for sqlserver connection issues #16080

Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
e80ce3e
Add additional troubleshooting documentation for sqlserver connection…
jmeunier28 Nov 30, 2022
a30adf3
Group together driver related troubleshooting steps
jmeunier28 Dec 1, 2022
6b29b0d
Review feedback
jmeunier28 Dec 1, 2022
740006f
Add section on Provider or driver not found
jmeunier28 Dec 1, 2022
7b69664
Add perm link anchors for sql server troubleshooting headers
jmeunier28 Dec 1, 2022
59de35d
review feedback
jmeunier28 Dec 5, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
126 changes: 94 additions & 32 deletions content/en/database_monitoring/setup_sql_server/troubleshooting.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,9 @@ description: Troubleshoot Database Monitoring setup for SQL Server

This page details common issues with setting up and using Database Monitoring with SQL Server, and how to resolve them. Datadog recommends staying on the latest stable Agent version and adhering to the latest [setup documentation][1], as it can change with Agent version releases.

## Diagnosing common connection issues
## Diagnosing common connection issues {#common-connection-issues}

### SQL Server unable to connect 'Login Failed for user'
### SQL Server unable to connect 'Login Failed for user' {#login-failed-for-user}

There are two ways that the agent can connect to a SQL Server instance:

Expand Down Expand Up @@ -41,7 +41,7 @@ If the `datadog` user is unable to log into the SQL Server instance, please ensu

Microsoft also provides a helpful doc on troubleshooting these types of errors, which can be [followed here][5].

### SQL Server Unable to connect due to “Invalid connection string attribute”
### SQL Server Unable to connect due to “Invalid connection string attribute” {#tcp-connection-error}

The following ADO Providers are supported on Windows: `SQLOLEDB`, `MSOLEDBSQL`, `MSOLEDBSQL19`, `SQLNCLI11`.

Expand Down Expand Up @@ -78,7 +78,21 @@ To troubleshoot:

If neither step produces meaningful help, or the error code you are seeing is not listed, Datadog recommends to use the `MSOLEDBSQL` driver or the `Microsoft ODBC Driver for SQL Server`. Either of these options will produce more meaningful error messages, which should help troubleshooting why the connection is failing.

### SSL Provider: The certificate chain was issued by an authority that is not trusted
### SQL Server 'Unable to connect: Adaptive Server is unavailable or does not exist' {#adaptive-server-unavailable}

This error can sometimes be the result of not properly setting the `host` field. For the integration, set the `host` field with the following syntax: `host:server,port`.

For example, if you've set `host` this way:

```
host: sqlserver-foo.cfxxae8cilce.us-east-1.rds.amazonaws.com
```
You must add the port, and instead set it as the following:
```
host: sqlserver-foo.cfxxae8cilce.us-east-1.rds.amazonaws.com,1433
```

### SSL Provider: The certificate chain was issued by an authority that is not trusted {#certificate-verify-fail}

This error is common after upgrading to the latest [MSOLEDBSQL][6] driver due to [breaking changes][7] that were introduced. In the latest version of the driver, all connections to the SQL instance are encrypted by default.

Expand Down Expand Up @@ -127,44 +141,91 @@ If you are using a driver **other than `MSOLEDBSQL` 2019**, this error can be re
driver: '{ODBC Driver 17 for SQL Server}'
```

### SQL Server unable to connect 'SSL Security error (18)'
### SQL Server unable to connect 'SSL Security error (18)' {#ssl-security-error}

This is a known issue for older versions of the SQL Server ODBC driver. You can check which version of the driver is being used by the agent by looking at the connection string in the error message.

For example, if you see `Provider=SQL Server` in the connection string of the error message, upgrading to a newer version of the ODBC driver will fix the error.

This issue is described in more detail in this [Microsoft blog post][9]

### Empty connection string {#empty-connection-string}

Datadog's SQL Server check relies on the `adodbapi` Python library, which has some limitations in the characters that it is able to use in making a connection string to a SQL Server. If your Agent experiences trouble connecting to your SQL Server, and if you find errors similar to the following in your Agent's collector.logs, your `sqlserver.yaml` may include some characters that cause issues with `adodbapi`.

```text
OperationalError: (KeyError('Python string format error in connection string->',), 'Error opening connection to ""')
```

At the moment, the only character known to cause this specific connectivity issue is the `%` character. If you need to use the `%` character in your `sqlserver.yaml` file, (for example, if your Datadog SQL Server user password includes a `%`), you must escape that character by including a double `%%` in place of each single `%`.

### SQL Server 'Unable to connect: Adaptive Server is unavailable or does not exist'
## Diagnosing common SQL Server driver issues {#common-driver-issues}

This error can sometimes just be the result of not properly setting the `host` field. For the integration, you should the `host` field with the following syntax: `host:server,port`.
### Data source name not found, and no default driver specified {#data-source-name-not-found}

For example:
This is a common error seen on Linux when using the default setting for the ODBC driver. This can happen due the [DSN][10], which is set for your driver in the `/etc/odbcinst.ini` file, not matching the name of the driver that is set in your agent config.

```
host: sqlserver-foo.cfxxae8cilce.us-east-1.rds.amazonaws.com
```
Should be set as:
```
host: sqlserver-foo.cfxxae8cilce.us-east-1.rds.amazonaws.com,1433
For example, if you wanted to use the default ODBC driver for the Agent (`{ODBC Driver 18 for SQL Server}`), your instance config should contain the following:

```yaml
connector: odbc
```

### Empty connection string
When the Agent starts and tries to establish a connection to your SQL Server instance, it looks for the `/etc/odbcinst.ini` file to find the path to the driver binaries.

Datadog's SQL Server check relies on the adodbapi Python library, which has some limitations in the characters that it is able to use in making a connection string to a SQL Server. If your Agent experiences trouble connecting to your SQL Server, and if you find errors similar to the following in your Agent's collector.logs, your `sqlserver.yaml` probably includes some character that causes issues with adodbapi.
For example, this `/etc/odbcinst.ini` file sets the driver:

```text
OperationalError: (KeyError('Python string format error in connection string->',), 'Error opening connection to ""')
```text
$ cat /etc/odbcinst.ini
[ODBC Driver 18 for SQL Server]
Description=Microsoft ODBC Driver 18 for SQL Server
Driver=/opt/microsoft/msodbcsql/lib64/libmsodbcsql-13.1.so.7.0
UsageCount=1
```

The DSN in the above example is `[ODBC Driver 18 for SQL Server]`, which matches the default driver name the Agent is using. If the DSN for your driver does not match the name of driver the Agent is using, you will get the `Data source not found` error.

It is possible to set the `dsn` in your instance config to match what is set in your `/etc/odbcinst.ini` file. For example:

```text
$ cat /etc/odbcinst.ini
[Custom]
Description=Microsoft ODBC Driver 18 for SQL Server
Driver=/opt/microsoft/msodbcsql/lib64/libmsodbcsql-13.1.so.7.0
UsageCount=1
```

In your instance config, you would then set the `dsn` field:

```yaml
connector: odbc
dsn: "Custom"
```

At the moment, the only character known to cause this specific connectivity issue is the `%` character. If you want to use the "%" character in your `sqlserver.yaml`, that is if your Datadog SQL Server user password includes a `%`), you need to escape that character by including a double `%%` in place of each single `%`.
### Provider or driver not found {#provider-not-found}

This error message can vary in wording across the different drivers, but typically it looks like the following for `ODBC`:

1. `Can't open lib .* file not found`
2. `Data source name not found.* and no default driver specified`

And for `MSOLEDBSQL` providers the error message looks like:

```text
Provider cannot be found. It may not be properly installed.
```

This means that the driver or provider is not properly installed on the host where the Agent is running. You should ensure that you have followed all the installation directions for the driver you have chosen to use.

It's possible that the Agent is not finding the driver. This is more common with ODBC drivers on Linux. See the [connecting to SQL Server on a Linux host](#connecting-to-sql-server-on-a-linux-host) section for more instructions on how to install the ODBC driver on Linux.

For help choosing a driver, see the [picking a SQL Server driver section](#picking-a-sql-server-driver) on how to properly configure your driver with the agent.

### Connecting to SQL Server on a Linux host

To connect SQL Server (either hosted on Linux or Windows) to a Linux host:

1. Install the [Microsoft ODBC Driver][10] for your Linux distribution.
1. Install the [Microsoft ODBC Driver][11] for your Linux distribution.
If you are unsure of the driver name to use, you can find it enclosed in brackets at the top of `/etc/odbcinst.ini`.

```text
Expand Down Expand Up @@ -195,22 +256,20 @@ To connect SQL Server (either hosted on Linux or Windows) to a Linux host:
password: <PASSWORD>
```

## Other common questions

### Picking a SQL Server driver
### Picking a SQL Server driver {#picking-a-driver}

In order for the agent to connect to the SQL Server instance, you must install either the [Microsoft ODBC driver][11] or the [OLE DB driver][12].
In order for the agent to connect to the SQL Server instance, you must install either the [Microsoft ODBC driver][12] or the [OLE DB driver][13].

Depending on which driver you pick, this will influence what you set for the [connector][13] field in your instance config.
The driver you choose, determines what you set for the [connector][14] field in your instance config.

For example, for the [Microsoft ODBC driver][11]:
For example, for the [Microsoft ODBC driver][12]:

```yaml
connector: odbc
driver: '{ODBC Driver 18 for SQL Server}'
```

For the [OLE DB driver][12]:
For the [OLE DB driver][13]:

```yaml
connector: adodbapi
Expand All @@ -221,7 +280,7 @@ These values will be used to map to the `Provider` part of the connection string

So for example, if you set `adoprovider: MSOLEDBSQL`, then the connection string would include `Provider=MSOLEDBSQL`. This should match the name of the driver version you have installed.

In the latest version of the [Microsoft OLE DB driver][12], the driver name was changed from `MSOLEDBSQL` to `MSOLEDBSQL19`, which means this should appear in your instance config like so:
In the latest version of the [Microsoft OLE DB driver][13], the driver name was changed from `MSOLEDBSQL` to `MSOLEDBSQL19`, which means this should appear in your instance config like so:

```yaml
connector: adodbapi
Expand All @@ -230,6 +289,8 @@ In the latest version of the [Microsoft OLE DB driver][12], the driver name was

It is recommended to stay up to date with the latest available version of the driver you select.

## Other common issues

### SQL Server user tag is missing on the Query Metrics and Plan Samples

The `user` tag is no longer supported for Query Metrics and Plan Samples due to technical limitations in SQL Server which prevent the collection of the correct user running queries.
Expand All @@ -249,7 +310,8 @@ In versions of the agent older than 7.40.0, there exists a bug where `PROCEDURE`
[7]: https://techcommunity.microsoft.com/t5/sql-server-blog/ole-db-driver-19-0-for-sql-server-released/ba-p/3170362
[8]: https://learn.microsoft.com/en-us/sql/connect/oledb/release-notes-for-oledb-driver-for-sql-server?view=sql-server-ver16#1863
[9]: https://community.hostek.com/t/ssl-security-error-for-microsoft-sql-driver/348
[10]: https://docs.microsoft.com/en-us/sql/connect/odbc/linux/installing-the-microsoft-odbc-driver-for-sql-server-on-linux
[11]: https://learn.microsoft.com/en-us/sql/connect/odbc/download-odbc-driver-for-sql-server?view=sql-server-ver16
[12]: https://learn.microsoft.com/en-us/sql/connect/oledb/oledb-driver-for-sql-server?view=sql-server-ver16
[13]: https://github.com/DataDog/integrations-core/blob/master/sqlserver/assets/configuration/spec.yaml#L201-L208
[10]: https://learn.microsoft.com/en-us/sql/integration-services/import-export-data/connect-to-an-odbc-data-source-sql-server-import-and-export-wizard?view=sql-server-ver16
[11]: https://docs.microsoft.com/en-us/sql/connect/odbc/linux/installing-the-microsoft-odbc-driver-for-sql-server-on-linux
[12]: https://learn.microsoft.com/en-us/sql/connect/odbc/download-odbc-driver-for-sql-server?view=sql-server-ver16
[13]: https://learn.microsoft.com/en-us/sql/connect/oledb/oledb-driver-for-sql-server?view=sql-server-ver16
[14]: https://github.com/DataDog/integrations-core/blob/master/sqlserver/assets/configuration/spec.yaml#L201-L208