Skip to content

Changes in oracle otel app for addition of metric collection, metric … #4694

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 13 commits into from
Oct 28, 2024
Merged

Changes in oracle otel app for addition of metric collection, metric … #4694

Changes from all commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
acef4ab
Changes in oracle otel app for addition of metric collection, metric …
sumoanema Oct 25, 2024
2db7a56
Spell corrections
sumoanema Oct 28, 2024
0209e71
Minor fixes
JV0812 Oct 28, 2024
ccf5863
Update docs/integrations/databases/opentelemetry/oracle-opentelemetry.md
JV0812 Oct 28, 2024
7b1229e
minor fix
JV0812 Oct 28, 2024
84e6192
Merge branch 'oracleotelupdate' of https://github.com/SumoLogic/sumol…
JV0812 Oct 28, 2024
3d77d34
Update docs/integrations/databases/opentelemetry/oracle-opentelemetry.md
JV0812 Oct 28, 2024
ecfb274
Modifying description for log file paths needed for config
sumoanema Oct 28, 2024
d2f9ead
Update docs/integrations/databases/opentelemetry/oracle-opentelemetry.md
JV0812 Oct 28, 2024
4462d03
Update docs/integrations/databases/opentelemetry/oracle-opentelemetry.md
JV0812 Oct 28, 2024
f3ac4c0
Update docs/integrations/databases/opentelemetry/oracle-opentelemetry.md
JV0812 Oct 28, 2024
a97ca83
Update docs/integrations/databases/opentelemetry/oracle-opentelemetry.md
JV0812 Oct 28, 2024
df7c41c
Merge branch 'main' into oracleotelupdate
sumoanema Oct 28, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
120 changes: 86 additions & 34 deletions docs/integrations/databases/opentelemetry/oracle-opentelemetry.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,40 +11,50 @@ import TabItem from '@theme/TabItem';

<img src={useBaseUrl('img/integrations/databases/oracle.png')} alt="Thumbnail icon" width="100"/> <img src={useBaseUrl('img/send-data/otel-color.svg')} alt="Thumbnail icon" width="45"/>

The [Oracle](https://docs.oracle.com/database/121/CNCPT/intro.htm#CNCPT001) app is a logs based app. Preconfigured dashboards and searches provide insight into the listeners, sys/xml audit logs, alerts, performance, and security.
The [Oracle](https://docs.oracle.com/database/121/CNCPT/intro.htm#CNCPT001) app is a logs and metrics based app. Preconfigured dashboards and searches provide insight into the listeners, sys/xml audit logs, alerts, performance, and security. It also gives insight around count of rollback, commits, transaction, process, session, hard parse, and DML locks.

This app is tested with the following Oracle versions:

- Non-Kubernetes: Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production - Version 19.3.0.0.0
- Non-Kubernetes: Oracle Database 23 Release 23.4.0.24.05.

Oracle logs are sent to Sumo Logic through OpenTelemetry [filelog receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/filelogreceiver).
Oracle logs are sent to Sumo Logic through OpenTelemetry [filelog receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/filelogreceiver) and metrics are collected through [Oracledb receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/oracledbreceiver)

<img src='https://sumologic-app-data-v2.s3.amazonaws.com/dashboards/Oracle-OpenTelemetry/Oracle-Schematics.png' alt="Schematics" />

## Log types

- Alert Logs
- Listener Logs
- Audit Logs

## Fields creation in Sumo Logic for Oracle

Following are the tags which that be created as part of Oracle App install if not already present. 

- `db.cluster.name`. User configured. Enter a name to identify this Oracle cluster. This cluster name will be shown in the Sumo Logic dashboards.
- `db.system`. Has a fixed value of **oracle**.
- `sumo.datasource`. Has a fixed value of **oracle**.
- `db.system`. Has a fixed value of **oracle**.
- `db.cluster.name`. User configured. Enter a name to identify this Oracle cluster. This cluster name will be shown in the Sumo Logic dashboards.
- `db.node.name`. Has the value of host name of the machine which is being monitored.

## Prerequisites

### For metrics collection

You will need to assign these permissions to the database user used to fetch metrics:

```
GRANT SELECT ON V_$SESSION TO <username>;
GRANT SELECT ON V_$SYSSTAT TO <username>;
GRANT SELECT ON V_$RESOURCE_LIMIT TO <username>;
GRANT SELECT ON DBA_TABLESPACES TO <username>;
GRANT SELECT ON DBA_DATA_FILES TO <username>;
GRANT SELECT ON DBA_TABLESPACE_USAGE_METRICS TO <username>;
```

### For logs collection

Here are the steps required to configure log collection for Oracle running on a non-Kubernetes environment.

* [Enable Oracle logging](#enable-oracle-logging)
* [Verify log files path](#verify-local-logs-file-directories-and-path)
* [Set up Oracle performance metrics script](#performance-metrics-script-setup)
* [Configure three local log file Sources](#step-2-configure-integration)

### Enable Oracle logging
#### Enable Oracle logging

If logging is not enabled, you can configure it by following the steps below.

Expand All @@ -54,9 +64,13 @@ If logging is not enabled, you can configure it by following the steps below.
lsnrctl command [listener_name]
lsnrctl set log_status on
```
- **Audit Log**. Follow [this](https://docs.oracle.com/cd/E11882_01/server.112/e10575/tdpsg_auditing.htm#TDPSG50000) guide to enable Audit Logs.
- **Audit Log**. Traditional auditing is deprecated in Oracle Database 21c. If you are using version below 21c follow [this](https://docs.oracle.com/cd/E11882_01/server.112/e10575/tdpsg_auditing.htm#TDPSG50000) guide to enable Audit Logs. For version 21c and above, follow [this](https://docs.oracle.com/en/database/oracle/oracle-database/19/dbseg/administering-the-audit-trail.html#GUID-662AA54B-D878-4B78-94D3-733256B3F37C) to enable exporting for unified audit trail through syslog for Oracle on Linux.

### Verify local logs file directories and path
:::note
Currently this app does not support collection of unified audit trail logs collected in Oracle on Windows as Windows event logs.
:::

#### Verify local logs file directories and path

- **Oracle Alert Logs**. For 11g and later releases (12c, 18c, 19c). By default, Oracle logs are stored in
`$ORACLE_BASE/diag/rdbms/$DB_UNIQUE_NAME/$ORACLE_SID/trace/`. The default directory for log files is stored in `BACKGROUND_DUMP_DEST` parameter. You can query the value of `BACKGROUND_DUMP_DEST`, an initialization parameter, where you can find Oracle alert log by executing the command below:
Expand All @@ -67,18 +81,20 @@ If logging is not enabled, you can configure it by following the steps below.
```
[oracle@sumolab alert]$ lsnrctl status
```
- **Oracle Audit Logs**. By default, Oracle logs are stored in
```
$ORACLE_BASE/app/oracle/admin/orcl/adump
```
- **Oracle Audit Logs**.
- **For Oracle version below 21c**. By default, Oracle logs are stored in
```
$ORACLE_BASE/app/oracle/admin/orcl/adump
```
The default directory for log files is stored as the value of `audit_file_dest`. In order to display it, run the following command: `SQL> show parameter audit`.

The default directory for log files is stored as the value of `audit_file_dest`. In order to display it, run the following command: `SQL> show parameter audit`.
Audit Logs should be in either `XML`, `EXTENDED`, or `{{OS }}` format for the app to work.

Audit Logs should be in either `XML`, `EXTENDED`, or `{{OS }}` format for the app to work.
The location of these logs will be required when you set up the app through the app catalog.

The location of these logs will be required when you set up the app through the app catalog.
- **For Oracle version 21c and above**. Once unified audit trail is redirected to syslog, it will start getting written to destination set in `syslog.conf`.

### Performance Metrics Script Setup
#### Performance metrics script setup

To set up the performance metrics script on Linux and Windows for the Oracle app:

Expand Down Expand Up @@ -163,21 +179,26 @@ import SetupColl from '../../../reuse/apps/opentelemetry/set-up-collector.md';
### Step 2: Configure integration

In this step, you will configure the yaml required for Oracle Collection.
Below are the inputs required:

Path of the log file configured to capture oracle logs needs to be given here. Here is the list of logs which are required by the application. 

- Alert Logs
- Listener Logs
- Audit Logs
- Performance metric script-based logs
- **`Endpoint (no default)`**. Endpoint used to connect to the OracleDB server. Must be in the format of `host:port`.
- **`Alert Logs`**. Path of the log file configured to capture oracle alert logs.
- **`Listener Logs`**. Path of the log file configured to capture oracle listener logs .
- **`Audit Logs`**. Path of the log file configured to capture oracle audit logs.
- **`Performance metric script-based logs`**. Path of the log file configured to capture log generated through script.
- **`username`**. Username for the OracleDB connection.
- **`password`**. Password for the OracleDB connection. Special characters are allowed.
- **`service`**. OracleDB Service that the receiver should connect to.
- **`Collection Interval`**. You can specify at what interval should the metrics be collected.

You can get the location of these logs by following the instructions in the prerequisite step.
You can add any custom fields which you want to tag along with the data ingested in Sumo Logic.

Once the details are filled, click on the **Download YAML File** button to get the yaml file.
For Linux platform, click **Download Environment Variables File** to get the file with the password which is supposed to be set as environment variable.

<img src='https://sumologic-app-data-v2.s3.amazonaws.com/dashboards/Oracle-OpenTelemetry/Oracle-YAML.png' style={{border:'1px solid gray'}} alt="YAML" />

### Step 3: Send logs to Sumo Logic
### Step 3: Send logs and metrics to Sumo Logic

import LogsIntro from '../../../reuse/apps/opentelemetry/send-logs-intro.md';

Expand Down Expand Up @@ -259,19 +280,19 @@ import LogsOutro from '../../../reuse/apps/opentelemetry/send-logs-outro.md';

<LogsOutro/>

## Sample log messages
## Sample log message

Sample Log Message in Non-Kubernetes environment:
Sample log message in Non-Kubernetes environment:

```sh title="Time"
20-Jan-2023 11:02:56 * (CONNECT_DATA=(CID=(PROGRAM=null)(HOST=__jdbc__)(USER=null))(SERVICE_NAME=sumo.cmdb01.com)) * (ADDRESS=(PROTOCOL=TCP)(HOST=124.243.25.82)(PORT=56486)) * establish * sumo.cmdb01.com * 12514

TNS-12514: TNS:listener does not currently know of service requested in connect descriptor
```

## Sample queries
## Sample log query

This sample Query is from the **Oracle - Overview** dashboard > **DB Connection** panel.
This sample query is from the **Oracle - Overview** dashboard > **DB Connection** panel.

```sql title="Query String"
 %"db.cluster.name"=* %"deployment.environment"=* %"sumo.datasource"=oracle establish ("SID=" or "SERVICE_NAME=")  | json "log" as _rawlog nodrop 
Expand All @@ -283,8 +304,27 @@ This sample Query is from the **Oracle - Overview** dashboard > **DB Connection*
| count as %"ConnectionCount"
```

## Sample metrics

```json
{"queryId":"A","oracledb.instance.name":"localhost:1521/FREE","_source":"oracle/oracledb","cloud.platform":"aws_ec2","_metricId":"lZCclGCpxjgi0_E4r4v5ww","host.image.id":"ami-000314be38df8042e","_sourceName":"oracle","cloud.availability_zone":"us-east-1b","host.id":"i-05ce9f3824aa73d10","host.group":"anemaoracle","os.type":"linux","sumo.datasource":"oracle","db.system":"oracle","db.node.name":"ip-10-0-29-37","cloud.account.id":"956882708938","_sourceCategory":"OTC Metric Input","deployment.environment":"anemaoracle","_contentType":"OpenTelemetry","tablespace_name":"SYSTEM","host.type":"m5.large","host.name":"ip-10-0-29-37","metric":"oracledb.tablespace_size.usage","_collectorId":"00005AF310D08C7C","_sourceId":"0000000000000000","unit":"By","db.cluster.name":"test","cloud.region":"us-east-1","cloud.provider":"aws","_sourceHost":"ip-10-0-29-37","_collector":"ip-10-0-29-37","max":1104150528,"min":1104150528,"avg":1104150528,"sum":3312451584,"latest":1104150528,"count":3}
```

## Sample metrics query

This sample query is from the **Oracle - Overview** dashboard > **Session Count** panel.

```sql title="Session Count"
sumo.datasource=oracle metric=oracledb.sessions.usage deployment.environment=* db.cluster.name=* db.node.name=* | sum
```

## Viewing Oracle dashboards

:::note
- **Oracle - Sys Audit Log**, **Oracle - Sys Audit Log - Logon Analysis**, **Oracle - XML Audit Log - Logon Analysis**, and **Oracle - XML Audit Log - SQL Statement Analysis** dashboards will be populated only when collecting logs for Oracle database version below 21c.
- **Oracle - Unified Audit Syslogs** dashboard will populate for audit log collected for Oracle database version 21c and above.
:::

### Overview

See the overview of Oracle listener process activity, including successful DB connections, TNS error information, SID and Service Name usage, and top usage, in terms of ports, database users, user hosts, client hosts, and user programs as derived from the Oracle Listener log.
Expand Down Expand Up @@ -504,3 +544,15 @@ Maximum Wait Time (sec) by User. A line chart that shows, for each user, the ses
Top Session Wait Time Events. A table that shows the top 10 event types associated with session waits, and the count of each event type.

Recent Jobs in the database. A table of information about recent database jobs, including when each job ran, low long it ran, and when it will next run.


### Unified Audit Syslog

<img src='https://sumologic-app-data-v2.s3.amazonaws.com/dashboards/Oracle-OpenTelemetry/Oracle-Unified-Audit-Syslog.png' alt="Monitor Performance by DB Script" />
See information derived from the syslog audit trail, including successful and failed activities, successful and failed logon attempts. In addition to this dashboard gives insight around logon status trend, top current and database users.

### Performance Details

<img src='https://sumologic-app-data-v2.s3.amazonaws.com/dashboards/Oracle-OpenTelemetry/Oracle-Performance-Details.png' alt="Monitor Performance by DB Script" />
The Oracle - Performance Details dashboard gives insight about - count of rollback, commits, transaction, process, session.
In addition to this it helps monitoring physical and logical reads, PGA allocated. This dashboard is based on the [metrics collected by Oracle DB opentelemetry receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/receiver/oracledbreceiver/documentation.md).
Loading