From 86e36d21d3a80832d1cd7f8494e2f1a8ae40da7a Mon Sep 17 00:00:00 2001 From: Matt Quinn Date: Wed, 4 Sep 2024 15:02:58 -0400 Subject: [PATCH 1/4] docs(insights): detail MongoDB support for Queries module --- .../sdk/telemetry/traces/modules/queries.mdx | 14 ++++++++++++-- .../telemetry/traces/span-data-conventions.mdx | 1 + docs/product/insights/queries.mdx | 17 ++++++++++++----- 3 files changed, 25 insertions(+), 7 deletions(-) diff --git a/develop-docs/sdk/telemetry/traces/modules/queries.mdx b/develop-docs/sdk/telemetry/traces/modules/queries.mdx index 19a575cb7a88e..e2c9a53110a76 100644 --- a/develop-docs/sdk/telemetry/traces/modules/queries.mdx +++ b/develop-docs/sdk/telemetry/traces/modules/queries.mdx @@ -2,7 +2,7 @@ title: Queries Module --- -The SDK should auto-instrument all database queries, and each query to the database should result in a span. This includes queries made manually, or via ORM. The Queries module supports SQL databases like PostgreSQL and MySQL. It does not support databases like Neo4j, ClickHouse, or MongoDB. Regardless of that, the SDK should instrument database queries as fully and correctly as possible to ensure correct behavior and future compatibility. +The SDK should auto-instrument all database queries, and each query to the database should result in a span. This includes queries made manually, or via ORM. The Queries module supports SQL databases like PostgreSQL and MySQL, as well as MongoDB. It does not support other databases like Neo4j or ClickHouse. Regardless of that, the SDK should instrument database queries as fully and correctly as possible to ensure correct behavior and future compatibility. ## Span Attributes @@ -29,11 +29,21 @@ SELECT * FROM users WHERE id = ?; The `?` parameter is a placeholder in the query. The SDK must scrub out all values and replace them with a placeholder. Supported placeholder values are `%s`, `?`, `:c0` (e.g., `:c0`, `:c1`) and `$1` (e.g., `$1`, `$2`). +For MongoDB, the description should contain valid JSON. If replacing values with a placeholder, the string `"?"` should be used. + +```json +{"find": "documents", "filter": { "wordCount": { "$gte": "?" } } } +``` + ### Span Data For full support, each database span should have a `data` attribute. `data` is itself a key-value lookup of attributes. Refer to [Database Span Data Conventions](/sdk/performance/span-data-conventions/#database) for a full list of attributes that database spans should have. We recommend that the SDK provide _all_ of those attributes for every span. However, the _minimal_ requirement is that the SDK provides the `db.system` attribute. -## Instrumentation Example +#### MongoDB Data + +MongoDB support requires a few extra data attributes. In addition to a `db.system` value or `mongodb`, the `db.operation` and `db.collection.name` attributes are also required to contain the command and collection name respectively. + +## SQL Instrumentation Example Consider a hypothetical Python ORM: diff --git a/develop-docs/sdk/telemetry/traces/span-data-conventions.mdx b/develop-docs/sdk/telemetry/traces/span-data-conventions.mdx index 9da3e3d6de8b3..761a4c9fb329d 100644 --- a/develop-docs/sdk/telemetry/traces/span-data-conventions.mdx +++ b/develop-docs/sdk/telemetry/traces/span-data-conventions.mdx @@ -68,6 +68,7 @@ Below describes the conventions for the Span interface for the `data` field on t | ----------------------- | ------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------- | | `db.system` | string | An identifier for the database management system (DBMS) product being used. See [OpenTelemetry docs for a list of well-known identifiers](https://github.com/open-telemetry/semantic-conventions/blob/main/docs/database/database-spans.md#notes-and-well-known-identifiers-for-dbsystem). | `postgresql` | | `db.operation` | string | The name of the operation being executed, e.g. the MongoDB command name such as findAndModify, or the SQL keyword. Based on [OpenTelemetry's common db attributes](https://github.com/open-telemetry/semantic-conventions/blob/main/docs/database/database-spans.md#common-attributes) | `SELECT` | +| `db.collection.name` | string | The name of the collection (or table, etc) being queried. | `users` | | `db.name` | string | This attribute is used to report the name of the database being accessed. For commands that switch the database, this should be set to the target database (even if the command fails). | `customers` | | `server.address` | string | Name of the database host. | `example.com` | | `server.port` | int | Logical server port number host. | `8080` | diff --git a/docs/product/insights/queries.mdx b/docs/product/insights/queries.mdx index ca32b94da56dd..e26ff7a1f62e4 100644 --- a/docs/product/insights/queries.mdx +++ b/docs/product/insights/queries.mdx @@ -38,13 +38,15 @@ Query monitoring works best with up-to-date SDK versions. The following minimum ### Span Eligibility -Sentry tries to extract metrics for all SQL-like dialects. NoSQL databases like MongoDB, graph databases like Neo4j, and other non-SQL database systems are not currently eligible for this feature. +Sentry tries to extract metrics for all SQL-like dialects, as well as MongoDB. Other NoSQL databases like Elasticsearch, graph databases like Neo4j, and other non-SQL database systems are not currently eligible for this feature. + +MongoDB support is currently in Alpha. Alpha features are still in-progress and may have bugs. We recognize the irony. If you are using automatic instrumentation, query monitoring should work without any configuration. If you've manually instrumented Sentry, you'll need to make sure that your spans conform to our standards for the best experience: - The span `op` field is set to an [eligible value](https://develop.sentry.dev/sdk/performance/span-operations/#database). -- The span's description contains the full, parameterized SQL query (e.g. `"SELECT * FROM users WHERE id = ?"`). Supported placeholder values are `%s`, `?`, `:c0` (e.g. `:c0`, `:c1`) and `$1` (e.g. `$1`, `$2`). -- The `db.system` span data value is set to the [correct identifier](https://github.com/open-telemetry/semantic-conventions/blob/main/docs/attributes-registry/db.md#generic-database-attributes) (e.g., `"postgresql"` or `"mysql"`). +- The span's description contains the full, parameterized query (e.g. `"SELECT * FROM users WHERE id = ?"`). Supported placeholder values for SQL are `%s`, `?`, `:c0` (e.g. `:c0`, `:c1`) and `$1` (e.g. `$1`, `$2`). MongoDB span descriptions should contain valid JSON, and placeholders should be the string `"?"`. +- The `db.system` span data value is set to the [correct identifier](https://github.com/open-telemetry/semantic-conventions/blob/main/docs/attributes-registry/db.md#generic-database-attributes) (e.g., `"postgresql"` or `"mongodb"`). ## Queries Page @@ -58,19 +60,24 @@ The query description table below shows a list of queries, along with their volu By default, this table is sorted by most [time spent](#what-is-time-spent). This means that queries at the top are usually really slow, very frequent, or both. You can change this to sort by queries per minute to see the most frequently run queries, or by average duration to see the slowest queries. -You can also use the dropdowns above to filter the table for specific SQL commands (such as `SELECT` and `UPDATE`) and tables queried. +You can also use the dropdowns above to filter all data for specific commands (such as `SELECT` and `UPDATE`) and tables/collections queried. To view more details, click on a query from the table to open its **Query Summary** page. ### Query Parameterization -In some cases, Sentry processes queries and simplifies them to improve readability. For example: +In some cases, Sentry processes queries and simplifies them to improve readability. For example, for SQL queries: - Table names are removed from column selections if possible (e.g., `SELECT "users"."name", "users"."id", "users.age" FROM users` becomes `SELECT name, id, age FROM users`) - Long lists of `SELECT` arguments are collapsed (e.g., `SELECT id, name, age, city, country, phone FROM users` becomes `SELECT .. FROM users`) - Long lists of `VALUES` arguments are collapsed (e.g., `INSERT INTO users (id, email, name, age) VALUES (%s %s %s %s)` becomes `INSERT INTO users (..) VALUES (%s)`) - `CASE / WHEN` statements are collapsed +For MongoDB queries: + +- Leaf values are replaced with the placeholder `"?"` +- A maximum depth is applied to deeply nested query objects + You can see the full query by hovering on a truncated description, or clicking it to see its **Query Summary** page. ## Query Summary Page From 403e562c082fcba54e93e578a1ee7c6703fac561 Mon Sep 17 00:00:00 2001 From: Matt Quinn Date: Wed, 4 Sep 2024 15:22:14 -0400 Subject: [PATCH 2/4] clarify system value Co-authored-by: George Gritsouk <989898+gggritso@users.noreply.github.com> --- develop-docs/sdk/telemetry/traces/modules/queries.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/develop-docs/sdk/telemetry/traces/modules/queries.mdx b/develop-docs/sdk/telemetry/traces/modules/queries.mdx index e2c9a53110a76..4e22099db51da 100644 --- a/develop-docs/sdk/telemetry/traces/modules/queries.mdx +++ b/develop-docs/sdk/telemetry/traces/modules/queries.mdx @@ -41,7 +41,7 @@ For full support, each database span should have a `data` attribute. `data` is i #### MongoDB Data -MongoDB support requires a few extra data attributes. In addition to a `db.system` value or `mongodb`, the `db.operation` and `db.collection.name` attributes are also required to contain the command and collection name respectively. +MongoDB support requires a few extra data attributes. In addition to a `db.system` value of `"mongodb"`, the `db.operation` and `db.collection.name` attributes are also required to contain the command and collection name respectively. ## SQL Instrumentation Example From 921772e2d3e92ec12ca8d38342497f70c9bf2d53 Mon Sep 17 00:00:00 2001 From: Matt Quinn Date: Wed, 4 Sep 2024 15:22:29 -0400 Subject: [PATCH 3/4] improve json spacing Co-authored-by: George Gritsouk <989898+gggritso@users.noreply.github.com> --- develop-docs/sdk/telemetry/traces/modules/queries.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/develop-docs/sdk/telemetry/traces/modules/queries.mdx b/develop-docs/sdk/telemetry/traces/modules/queries.mdx index 4e22099db51da..838809be899e2 100644 --- a/develop-docs/sdk/telemetry/traces/modules/queries.mdx +++ b/develop-docs/sdk/telemetry/traces/modules/queries.mdx @@ -32,7 +32,7 @@ The `?` parameter is a placeholder in the query. The SDK must scrub out all valu For MongoDB, the description should contain valid JSON. If replacing values with a placeholder, the string `"?"` should be used. ```json -{"find": "documents", "filter": { "wordCount": { "$gte": "?" } } } +{ "find": "documents", "filter": { "wordCount": { "$gte": "?" } } } ``` ### Span Data From 1b327cabda1035da1bbe0e2bc157957c2a712c7f Mon Sep 17 00:00:00 2001 From: Matt Quinn Date: Tue, 15 Oct 2024 11:13:31 -0400 Subject: [PATCH 4/4] Alpha -> Early Adopter MongoDB is in EA now --- docs/product/insights/queries.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/product/insights/queries.mdx b/docs/product/insights/queries.mdx index e26ff7a1f62e4..cdc78aa768c5b 100644 --- a/docs/product/insights/queries.mdx +++ b/docs/product/insights/queries.mdx @@ -40,7 +40,7 @@ Query monitoring works best with up-to-date SDK versions. The following minimum Sentry tries to extract metrics for all SQL-like dialects, as well as MongoDB. Other NoSQL databases like Elasticsearch, graph databases like Neo4j, and other non-SQL database systems are not currently eligible for this feature. -MongoDB support is currently in Alpha. Alpha features are still in-progress and may have bugs. We recognize the irony. +MongoDB support is currently available to [Early Adopters](/organization/early-adopter-features/). Early Adopter features are still in-progress and may have bugs. We recognize the irony. If you are using automatic instrumentation, query monitoring should work without any configuration. If you've manually instrumented Sentry, you'll need to make sure that your spans conform to our standards for the best experience: