Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -129,7 +129,11 @@ import LogsIntro from '../../../reuse/apps/opentelemetry/send-logs-intro.md';
<TabItem value="Linux">

1. Copy the yaml file to `/etc/otelcol-sumo/conf.d/` folder in the RabbitMQ instance which needs to be monitored.
2. Restart the collector using:
2. Move the `env` file to the following directory:
```sh
/etc/otelcol-sumo/env/
```
3. Restart the collector using:
```sh
  sudo systemctl restart otelcol-sumo
```
Expand Down
130 changes: 94 additions & 36 deletions docs/integrations/databases/opentelemetry/mongodb-opentelemetry.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,15 +11,17 @@ import TabItem from '@theme/TabItem';

<img src={useBaseUrl('img/integrations/databases/mongodb.png')} alt="Thumbnail icon" width="120"/><img src={useBaseUrl('img/send-data/otel-color.svg')} alt="Thumbnail icon" width="45"/>

[MongoDB](https://www.mongodb.com/why-use-mongodb#:~:text=MongoDB%20is%20a%20document%20database,development%20teams%20using%20agile%20methodologies.) is a source-available cross-platform document-oriented database program. The Sumo Logic app for MongoDB supports logs and metrics from the open source version of MongoDB. The App is tested on the 4.4.4 version of MongoDB.
[MongoDB](https://www.mongodb.com/why-use-mongodb#:~:text=MongoDB%20is%20a%20document%20database,development%20teams%20using%20agile%20methodologies.) is a source-available cross-platform document-oriented database program. The Sumo Logic app for MongoDB supports logs and metrics from the open source version of MongoDB. The app is tested on the 7.0.14 version of MongoDB.

MongoDB logs are sent to Sumo Logic through OpenTelemetry [filelog receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/filelogreceiver).
MongoDB logs are sent to Sumo Logic through OpenTelemetry [filelog receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/filelogreceiver). The OpenTelemetry collector runs on the same host as MongoDB and uses the [MongoDB Receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/mongodbreceiver) and the [Sumo Logic OpenTelemetry Exporter](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/exporter/sumologicexporter) to send the metrics to Sumo Logic.

<img src='https://sumologic-app-data-v2.s3.amazonaws.com/dashboards/MongoDB-OpenTelemetry/MongoDB-Schematics.png' alt="Schematics" />

## Log types
## Log and metrics types

The MongoDB logs are generated in files as configured in the configuration file `/var/log/mongodb/mongodb.log`. For more details on MongoDB logs, see [this](https://docs.mongodb.com/manual/reference/log-messages/) link.
This app supports logs and metrics for MongoDB instance. The MongoDB logs are generated in files as configured in this configuration file `/var/log/mongodb/mongodb.log`. For more details on MongoDB logs, refer to the [MongoDB documentation](https://docs.mongodb.com/manual/reference/log-messages/).

The app supports metrics generated by the [MongoDB Receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/receiver/mongodbreceiver/documentation.md).

## Fields creation in Sumo Logic for MongoDB

Expand All @@ -29,19 +31,28 @@ Following are the [Fields](/docs/manage/fields/) which will be created as part o
- **`db.system`**. Has fixed value of **mongodb**.
- **`deployment.environment`**. User configured. This is the deployment environment where the Mongodb cluster resides. For example: dev, prod or qa.
- **`sumo.datasource`**. has a fixed value of **mongodb**.
* **`db.node.name`**. Has the value of host name of the machine which is being monitored.

## Prerequisites

### For metrics collection

The MongoDB receiver fetches stats from a MongoDB instance using the [golang mongo driver](https://github.com/mongodb/mongo-go-driver). Stats are collected via MongoDB's `dbStats` and `serverStatus` commands. The MongoDB receiver extracts values from the result and converts them to OpenTelemetry metrics.

### For logs collection

By default, MongoDB logs are stored in a log file.

1. Configure logging verbosity in MongoDB : MongoDB logs have six levels of verbosity. All logging settings are located in [MongoDB.conf](https://docs.mongodb.com/manual/reference/method/db.setLogLevel/). To select a level, set loglevel to one of:
1. Configure logging verbosity in MongoDB.<br/>MongoDB logs have six levels of verbosity. All logging settings are located in [MongoDB.conf](https://docs.mongodb.com/manual/reference/method/db.setLogLevel/). To select a level, set loglevel to one of:
- 0 is the MongoDB's default log verbosity level, to include [Informational](https://docs.mongodb.com/manual/reference/log-messages/#std-label-log-severity-levels) messages.
- 1 to 5 increases the verbosity level to include[ Debug](https://docs.mongodb.com/manual/reference/log-messages/#std-label-log-severity-levels) messages.
2. Configure MongoDB to log to a Local file: Configuring MongoDB logs to go to log files. By default, MongoDB logs are stored in `/var/log/mongodb/mongodb.log`. The default directory for log files is listed in the MongoDB.conf file. To configure the log output destination to a log file, use one of the following settings, either in the [configuration file](https://docs.mongodb.com/manual/reference/configuration-options/) or command-line:
2. You need to set the [profiling_level](https://www.mongodb.com/docs/manual/reference/method/db.setProfilingLevel/) to a value of 1. <br/>But by default it is set to 0. You can also configure the value of [slowms](https://www.mongodb.com/docs/manual/reference/method/db.setProfilingLevel/#std-label-set-profiling-level-options-slowms) to a value greater than or equal to 100. Thus can be done using the below command in mongosh:
`db.setProfilingLevel(1,100)`
3. Configure MongoDB to log to a local file. <br/>By default, MongoDB logs are stored in `/var/log/mongodb/mongodb.log`. The default directory for log files is listed in the `MongoDB.conf` file. To configure the log output destination to a log file, use one of the following settings, either in the [configuration file](https://docs.mongodb.com/manual/reference/configuration-options/) or command-line:
- Configuration file: The [systemLog.destination](https://docs.mongodb.com/manual/reference/configuration-options/#mongodb-setting-systemLog.destination) option for file.
- Command-line:
- The [--logpath](https://docs.mongodb.com/manual/reference/program/mongod/#std-option-mongod.--logpath) option for [mongod](https://docs.mongodb.com/manual/reference/program/mongod/#mongodb-binary-bin.mongod) for file.
- The [--logpath](https://docs.mongodb.com/manual/reference/program/mongos/#std-option-mongos.--logpath) option for [mongos](https://docs.mongodb.com/manual/reference/program/mongos/#mongodb-binary-bin.mongos) for file.
- Command-line:
- The [--logpath](https://docs.mongodb.com/manual/reference/program/mongod/#std-option-mongod.--logpath) option for [mongod](https://docs.mongodb.com/manual/reference/program/mongod/#mongodb-binary-bin.mongod) for file.
- The [--logpath](https://docs.mongodb.com/manual/reference/program/mongos/#std-option-mongos.--logpath) option for [mongos](https://docs.mongodb.com/manual/reference/program/mongos/#mongodb-binary-bin.mongos) for file.

import LogsCollectionPrereqisites from '../../../reuse/apps/logs-collection-prereqisites.md';

Expand Down Expand Up @@ -79,9 +90,22 @@ import SetupColl from '../../../reuse/apps/opentelemetry/set-up-collector.md';

### Step 2: Configure integration

In this step the user needs to provide the path to the mongo db log file configured as part of above steps. Typically the logs are located at the location: `/var/log/mongodb/mongodb.log`.
OpenTelemetry works with a [configuration](https://opentelemetry.io/docs/collector/configuration/) YAML file with all the details concerning the data that needs to be collected. For example, it specifies the location of a log file that is read and sent to the Sumo Logic platform.

In this step, you will configure the YAML file required for MongoDB collection.

Below are the inputs required:

- **`Endpoint (no default)`**. The hostname and port of the MognoDB instance, separated by a colon. (For example: `localhost:27017`.)
- **`Logs Path`**. Provide the path to the mongo db log file configured as part of above steps. Typically the logs are located at the location: `/var/log/mongodb/mongodb.log`.
- **`username (optional)`**. If authentication is required, the user can with clusterMonitor permissions can be provided here.
- **`password (optional)`**. If authentication is required, the password can be provided here.

You can add any custom fields which you want to tag along with the data ingested in Sumo. Click on the **Download YAML File** button to get the YAML file.

You can add any custom fields which you want to tag along with the data ingested in Sumo. Click on the **Download YAML File** button to get the yaml file.
Click on the **Download YAML File** button to get the YAML file.

For Linux platform, click **Download Environment Variables File** to get the file with the password which is supposed to be set as environment variable.

<img src='https://sumologic-app-data-v2.s3.amazonaws.com/dashboards/MongoDB-OpenTelemetry/MongoDB-YAML.png' style={{border:'1px solid gray'}} alt="YAML" />

Expand All @@ -105,16 +129,20 @@ import LogsIntro from '../../../reuse/apps/opentelemetry/send-logs-intro.md';

<TabItem value="Linux">

1. Copy the yaml file to `/etc/otelcol-sumo/conf.d/` folder in the Mongodb instance which needs to be monitored.
2. restart the collector using:
1. Copy the YAML file to `/etc/otelcol-sumo/conf.d/` folder in the Mongodb instance which needs to be monitored.
2. Place Env file in the following directory:
```sh
/etc/otelcol-sumo/env/
```
3. restart the collector using:
```sh
 sudo systemctl restart otelcol-sumo
```

</TabItem>
<TabItem value="Windows">

1. Copy the yaml file to `C:\ProgramData\Sumo Logic\OpenTelemetry Collector\config\conf.d` folder in the machine which needs to be monitored.
1. Copy the YAML file to `C:\ProgramData\Sumo Logic\OpenTelemetry Collector\config\conf.d` folder in the machine which needs to be monitored.
2. Restart the collector using: 
```sh
Restart-Service -Name OtelcolSumo
Expand All @@ -123,7 +151,7 @@ Restart-Service -Name OtelcolSumo
</TabItem>
<TabItem value="macOS">

1. Copy the yaml file to `/etc/otelcol-sumo/conf.d/` folder in the Mongodb instance which needs to be monitored.
1. Copy the YAML file to `/etc/otelcol-sumo/conf.d/` folder in the Mongodb instance which needs to be monitored.
2. Restart the otelcol-sumo process using:
```sh
 otelcol-sumo --config /etc/otelcol-sumo/sumologic.yaml --config "glob:/etc/otelcol-sumo/conf.d/*.yaml" 
Expand Down Expand Up @@ -161,7 +189,7 @@ import LogsOutro from '../../../reuse/apps/opentelemetry/send-logs-outro.md';

## Sample log messages

```sql
```json
{
   "t":{
       "$date":"2021-05-21T10:22:57.373+00:00"
Expand Down Expand Up @@ -195,9 +223,7 @@ import LogsOutro from '../../../reuse/apps/opentelemetry/send-logs-outro.md';

## Sample queries

Dashboard: MongoDB - Errors and Warnings, Panel: Errors by Component

```sql
```sql title="Errors by Component( MongoDB - Errors and Warnings)"
deployment.environment=* db.cluster.name=* sumo.datasource=mongodb  | json "log" as _rawlog nodrop
| if (isEmpty(_rawlog), _raw, _rawlog) as _raw
| json field=_raw "t.$date" as timestamp
Expand All @@ -211,17 +237,19 @@ deployment.environment=* db.cluster.name=* sumo.datasource=mongodb  | json "log

## Viewing MongoDB dashboards

If no relevant data was received within the time range of the Panel, the Panel will be empty.
import ViewDashboards from '../../../reuse/apps/view-dashboards.md';

<ViewDashboards/>

### Overview

The **MongoDB - Overview** dashboard provides an at-a-glance view of MongoDB health, performance and problems causing errors.
The **MongoDB - Overview** dashboard provides an at-a-glance view of MongoDB health, performance, and problems causing errors.

Use this dashboard to:

- Identify Slow CRUD and DB commands.
- Gain insights into Errors logs by component and context.
- Number of up servers.
- Identify slow CRUD and DB commands.
- Gain insights into errors logs by component and context.
- Know the number of up servers.

<img src='https://sumologic-app-data-v2.s3.amazonaws.com/dashboards/MongoDB-OpenTelemetry/MongoDB-Overview.png' alt="Overview" />

Expand All @@ -231,7 +259,7 @@ The **MongoDB - Errors and Warnings** dashboard shows errors and warnings by the

Use this dashboard to:

- Determine components producing multiple errors or warnings.
- Determine components producing multiple errors or warnings.

<img src='https://sumologic-app-data-v2.s3.amazonaws.com/dashboards/MongoDB-OpenTelemetry/MongoDB-Errors-and-Warnings.png' alt="Access" />

Expand All @@ -241,8 +269,8 @@ The **MongoDB - Logins and Connections** dashboard shows geo location of client

Use this dashboard to:

- Determine potential hacking attempts.
- Determine location of attacks.
- Determine potential hacking attempts.
- Determine location of attacks.

<img src='https://sumologic-app-data-v2.s3.amazonaws.com/dashboards/MongoDB-OpenTelemetry/MongoDB-Logins-and-Connections.png' alt="Logins and Connections" />

Expand All @@ -252,29 +280,59 @@ The **MongoDB - Query Logs** dashboard shows read and write query trends.

Use this dashboard to:

- Monitor abnormal spikes in Query volume.
- Identify the read versus write ratio of your application queries. Adjusting indexes to improve query performance.
- Monitor abnormal spikes in query volume.
- Identify read versus write ratio of your application queries. This helps you to adjust indexes to improve query performance.

<img src='https://sumologic-app-data-v2.s3.amazonaws.com/dashboards/MongoDB-OpenTelemetry/MongoDB-Query-Logs.png' alt="Query Logs" />

### Sharding

The **MongoDB - Sharding** dashboard shows sharding related errors, events, failures, and number of chunks moving between shards.

Use this dashboard to:

- Identify sharding errors and warnings.
- Gain insights into chunk operations.

<img src='https://sumologic-app-data-v2.s3.amazonaws.com/dashboards/MongoDB-OpenTelemetry/MongoDB-Sharding.png' alt="Sharding" />

### Replication Logs

The **MongoDB - Replication Logs** dashboard shows replica deletes/updates/inserts trend and replica state.

Use this dashboard to:

- Monitor replication state and replication events like inserts/updates/commands per second.
- Track Replication Oplog window to identify replication delay.
- Identify replication errors and warnings.
- Gain insights into replication operations.

<img src='https://sumologic-app-data-v2.s3.amazonaws.com/dashboards/MongoDB-OpenTelemetry/MongoDB-Replication-Logs.png' alt="Replication Logs" />

### Sharding
### Resource

The **MongoDB - Sharding** dashboard dashboard shows sharding related errors, events, failures and number of chunks moving between shards.
The **MongoDB - Resource** dashboard shows resource utilization by the MongoDB component.

Use this dashboard to:
* Determine memory and disk usage.
* Identify potential resource constraints and issues.

- Identify Sharding errors and warnings.
- Gain insights into Chunk operations.
<img src='https://sumologic-app-data-v2.s3.amazonaws.com/dashboards/MongoDB-OpenTelemetry/MongoDB-Resource.png' alt="Resource" />

<img src='https://sumologic-app-data-v2.s3.amazonaws.com/dashboards/MongoDB-OpenTelemetry/MongoDB-Sharding.png' alt="Sharding" />
### Operations

The **MongoDB - Operations** dashboard shows MongoDB queries analytics using metrics.

Use this dashboard to:
* Know different kind of operation count like query, insert, and delete.
* Determine the operation time taken by different queries.

<img src='https://sumologic-app-data-v2.s3.amazonaws.com/dashboards/MongoDB-OpenTelemetry/MongoDB-Operations.png' alt="Operations" />

### Replication

The **MongoDB - Replication** dashboard displays the replication events, errors, warnings, and nodes information.

Use this dashboard to:
* Know different kind of operation count executed on replicas.
* Operation count like query, insert, and delete.

<img src='https://sumologic-app-data-v2.s3.amazonaws.com/dashboards/MongoDB-OpenTelemetry/MongoDB-Replication.png' alt="Replication" />
32 changes: 20 additions & 12 deletions docs/integrations/databases/opentelemetry/redis-opentelemetry.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,9 +75,9 @@ import SetupColl from '../../../reuse/apps/opentelemetry/set-up-collector.md';

### Step 2: Configure integration

OpenTelemetry works with a [configuration](https://opentelemetry.io/docs/collector/configuration/) yaml file with all the details concerning the data that needs to be collected. For example, it specifies the location of a log file that is read and sent to the Sumo Logic platform.
OpenTelemetry works with a [configuration](https://opentelemetry.io/docs/collector/configuration/) YAML file with all the details concerning the data that needs to be collected. For example, it specifies the location of a log file that is read and sent to the Sumo Logic platform.

In this step, you will configure the yaml file required for Redis Collection.
In this step, you will configure the YAML file required for Redis Collection.

Below are the inputs required:

Expand All @@ -86,7 +86,11 @@ Below are the inputs required:

The log file path configured to capture redis logs must be given here. The files are typically located in `/var/log/redis/redis-server.log`. If you are using a customized path, check the [`redis.conf`](https://download.redis.io/redis-stable/redis.conf) file for this information.

You can add any custom fields which you want to tag along with the data ingested in Sumo. Click on the **Download YAML File** button to get the yaml file.
You can add any custom fields which you want to tag along with the data ingested in Sumo. Click on the **Download YAML File** button to get the YAML file.

Click on the **Download YAML File** button to get the YAML file.

For Linux platform, click **Download Environment Variables File** to get the file with the password which is supposed to be set as environment variable.

<img src='https://sumologic-app-data-v2.s3.amazonaws.com/dashboards/Redis-OpenTelemetry/Redis-YAML.png' style={{border:'1px solid gray'}} alt="Configuration" />

Expand All @@ -113,20 +117,24 @@ import LogsIntro from '../../../reuse/apps/opentelemetry/send-logs-intro.md';

<TabItem value="Linux">

1. Copy the yaml at `/etc/otelcol-sumo/conf.d/` folder in the Redis instance that needs to be monitored.
2. Restart the otelcol-sumo process using:
```sh
sudo systemctl restart otelcol-sumo
```
1. Copy the YAML at `/etc/otelcol-sumo/conf.d/` folder in the Redis instance that needs to be monitored.
2. Move the env file in the following directory:
```sh
/etc/otelcol-sumo/env/
```
3. Restart the otelcol-sumo process using:
```sh
sudo systemctl restart otelcol-sumo
```

</TabItem>
<TabItem value="macOS">

1. Copy the yaml at `/etc/otelcol-sumo/conf.d/` folder in the Redis instance that needs to be monitored.
1. Copy the YAML at `/etc/otelcol-sumo/conf.d/` folder in the Redis instance that needs to be monitored.
2. Restart the otelcol-sumo process using:
```sh
otelcol-sumo --config /etc/otelcol-sumo/sumologic.yaml --conf "glob:/etc/otelcol-sumo/conf.d/*.yaml"
```
```sh
otelcol-sumo --config /etc/otelcol-sumo/sumologic.yaml --conf "glob:/etc/otelcol-sumo/conf.d/*.yaml"
```

</TabItem>
<TabItem value="Chef">
Expand Down