diff --git a/docs/images/explore-logs-openobserve-plugin.png b/docs/images/explore-logs-openobserve-plugin.png new file mode 100644 index 00000000..139c0839 Binary files /dev/null and b/docs/images/explore-logs-openobserve-plugin.png differ diff --git a/docs/images/openobserve-data-source-config.png b/docs/images/openobserve-data-source-config.png new file mode 100644 index 00000000..19232389 Binary files /dev/null and b/docs/images/openobserve-data-source-config.png differ diff --git a/docs/images/openobserve-plugin.png b/docs/images/openobserve-plugin.png new file mode 100644 index 00000000..db7253f7 Binary files /dev/null and b/docs/images/openobserve-plugin.png differ diff --git a/docs/operator-guide/.pages b/docs/operator-guide/.pages index f45b4a6a..e3b58f20 100644 --- a/docs/operator-guide/.pages +++ b/docs/operator-guide/.pages @@ -3,7 +3,7 @@ nav: - Systemd: systemd.md - SIMD: simd.md - Mimalloc: mimalloc.md - - Grafana plugin: grafana_plugin.md + - Grafana plugin: openobserve-plugin-for-grafana.md - Etcd maintenance: etcd.md - Etcd restore: etcd_restore.md - Nginx proxy: nginx_proxy.md diff --git a/docs/operator-guide/grafana_plugin.md b/docs/operator-guide/grafana_plugin.md deleted file mode 100644 index 80a01083..00000000 --- a/docs/operator-guide/grafana_plugin.md +++ /dev/null @@ -1,227 +0,0 @@ ---- -title: OpenObserve Grafana Plugin -weight: 4450 -description: >- - Use the OpenObserve Grafana plugin to visualize logs and metrics if you're - already using Grafana. Includes setup steps for Kubernetes and non-K8s - installs. ---- -# OpenObserve Grafana Plugin - -## What is Grafana? - -Grafana is a popular open-source dashboarding and visualization platform. Originally designed for time series data, it has evolved into a comprehensive tool that can pull data from multiple sources and create unified dashboards for logs, metrics, and traces. It's widely used for monitoring and observability across organizations. - -## Do You Need Grafana with OpenObserve? - -**Short Answer: No, but you might want it.** - -OpenObserve comes with a powerful built-in GUI that handles all your visualization needs, including: - -- [Logs analysis and search](../features/logs.md) -- [Metrics monitoring](../features/metrics.md) -- [Distributed tracing](../features/distributed-tracing.md) -- [Frontend monitoring](../features/frontend.md) -- [Interactive dashboards](../user-guide/dashboards/dashboards-in-openobserve.md) -- [Alerting and notifications](../user-guide/alerts/alerts.md) - -**When to Use the Grafana Plugin:** - -You should consider using OpenObserve's Grafana plugin if you: - -1. **Already use Grafana** for other monitoring needs (e.g., Prometheus metrics) -2. **Have existing Grafana dashboards** you want to keep -3. **Need to consolidate** OpenObserve data with other data sources in a single Grafana instance - -!!! warning "Plugin Maintenance Status" - This Grafana plugin is not actively maintained. It may work with current Grafana and OpenObserve versions, but compatibility isn’t guaranteed. Test thoroughly before production use. For best results, use OpenObserve’s built-in visualizations. - -## Getting Started - -The following guide will walk you through installing and configuring the plugin in a Kubernetes environment. The steps can be adapted for non-Kubernetes deployments. - -**Quick Start:** If you are already familiar with grafana plugin installation, you can download the plugin from [here](https://zincsearch-releases.s3.us-west-2.amazonaws.com/zo_gp/zo_gp.tar.gz) and get started. Feel free to skip the configuration section, you can directly jump [here](#using-grafana-plugin) - -## Install Grafana - -Grafana requires a persistent store to store its data and configuration. While configuration can be stored in a configmap or secret, data needs to be stored in a database. Grafana supports sqlite, mysql and postgres. Most installations I have seen in the wild use a single node grafana installation using sqlite. I have also seen that many of these use a kubernetes `deployment` . - -If you are using a single node grafana installation using sqlite then you should use `statefulset` instead of `deployment` so you do not lose your data when the pod restarts. If you are using mysql/postgres then you can use `deployment` as the data is stored in the database. - -You would also need a `grafana.ini` config file to configure grafana. You can use the below minimalistic working grafana.ini file to start. You can add more configuration as needed. - - -### Configuration - -```ini title="grafana.ini" linenums="1" hl_lines="4 9" -[date_formats] -default_timezone = UTC -[server] -root_url = https://grafana.yourdomain.com - -[plugins] -enable_alpha = true -app_tls_skip_verify_insecure = false -allow_loading_unsigned_plugins = zinclabs_openobserve -``` - -`Line 4` should be updated with the root url of your grafana installation. This is the url that you will use to access grafana. e.g. `https://grafana.yourdomain.com` - -`Line 9` is the one that is important where we specify that grafana should use the unsigned plugin `zinclabs_openobserve`. This is the plugin that we will install using the init container in the statefulset. - -Once you have created the file, you can create a kubernetes secret using the below command. - - -```bash linenums="1" -kubectl create secret generic grafana-config --from-file=grafana.ini -``` - - -### Deployment - -Now let's install grafana. - -```yaml title="grafana_statefulset.yaml" linenums="1" -apiVersion: apps/v1 -kind: StatefulSet -metadata: - name: grafana -spec: - serviceName: "grafana" - replicas: 1 - selector: - matchLabels: - app: grafana - template: - metadata: - labels: - app: grafana - spec: - securityContext: - fsGroup: 2000 - runAsUser: 10000 - runAsGroup: 3000 - initContainers: - - name: openobserve-plugin-loader - image: wbitt/network-multitool - imagePullPolicy: IfNotPresent - command: - [ - "sh", - "-c", - "curl -o /var/lib/grafana/plugins/zo_gp.tar.gz https://zincsearch-releases.s3.us-west-2.amazonaws.com/zo_gp/zo_gp.tar.gz && cd /var/lib/grafana/plugins && tar -zxvf zo_gp.tar.gz", - ] - volumeMounts: - - name: grafana-base - mountPath: /var/lib/grafana - - name: grafana-plugins - mountPath: /var/lib/grafana/plugins - containers: - - name: grafana - image: grafana/grafana:latest - ports: - - containerPort: 3000 - name: grafana - volumeMounts: - - name: grafana-base - mountPath: /var/lib/grafana - - name: grafana-plugins - mountPath: /var/lib/grafana/plugins - - name: grafana-config - mountPath: /etc/grafana - volumes: - - name: grafana-base - persistentVolumeClaim: - claimName: grafana-base - - name: grafana-plugins - persistentVolumeClaim: - claimName: grafana-plugins - - name: grafana-config - secret: - defaultMode: 420 - secretName: grafana-config - volumeClaimTemplates: - - metadata: - name: grafana-base - spec: - accessModes: ["ReadWriteOnce"] - resources: - requests: - storage: 1Gi - - metadata: - name: grafana-plugins - spec: - accessModes: ["ReadWriteOnce"] - resources: - requests: - storage: 1Gi ---- -apiVersion: v1 -kind: Service -metadata: - name: grafana -spec: - ports: - - port: 3000 - targetPort: 3000 - selector: - app: grafana - -``` - -We are using an `init container` in this case to download and configure openobserve plugin for the grafana container. - - -Now let's install the statefulset using the below command. - -```bash linenums="1" -kubectl apply -f grafana_statefulset.yaml -``` - -## Using Grafana plugin - - -### Set up data source - -Once grafana starts you could go to the data sources section and search for `OpenObserve` - - - -Next let's add OpenObserve data source server details. - -You need to do following: - -1. Update URL -1. Enable Basic Auth -1. Provide user id and password for basic auth details. - -Once you have updated the above, click on `Save and Test` button. If everything is correct you should see a success message. - - - - -### Explore logs - -Now let's explore some logs. Click on Explore menu item on the left and select OpenObserve as the data source. - - - - - -Select appropriate: -1. organization -1. stream -1. time range - -and click on `Run Query` button. You should see the logs from the stream. - - - -You should now be able to see the results. - - - -If you want to explore metrics from OpenObserve in Grafana, you can set up OpenObserve as a Prometheus-compatible data source using an endpoint like https://api.openobserve.ai/api/org_name/prometheus. You do not need the plugin for this, as Grafana supports Prometheus natively. - - - diff --git a/docs/operator-guide/index.md b/docs/operator-guide/index.md index a73e4d2a..444a386f 100644 --- a/docs/operator-guide/index.md +++ b/docs/operator-guide/index.md @@ -3,7 +3,7 @@ Learn more: - [Systemd](systemd) - [SIMD](simd) - [Mimalloc](mimalloc) -- [Grafana plugin](grafana_plugin) +- [Grafana plugin](openobserve-plugin-for-grafana) - [Etcd maintenance](etcd) - [Etcd restore](etcd_restore) - [Nginx proxy](nginx_proxy) diff --git a/docs/operator-guide/openobserve-plugin-for-grafana.md b/docs/operator-guide/openobserve-plugin-for-grafana.md new file mode 100644 index 00000000..4fcc512e --- /dev/null +++ b/docs/operator-guide/openobserve-plugin-for-grafana.md @@ -0,0 +1,211 @@ +--- +title: OpenObserve Plugin for Grafana +weight: 4450 +description: >- + Use the OpenObserve plugin in Grafana to visualize logs and metrics if you are + already using Grafana. Includes setup steps for Kubernetes and non-K8s + installs. +--- +# OpenObserve Plugin for Grafana +This guide walks you through installing and configuring the openobserve plugin in Grafana within a Kubernetes environment. If you are not using Kubernetes, you can adapt these steps for your environment. + +## Overview +Grafana is an open-source platform for creating dashboards and visualizations. +OpenObserve provides its own interface for [logs](../features/logs.md), [metrics](../features/metrics.md), [tracing](../features/distributed-tracing.md), [frontend monitoring](../features/frontend.md), [dashboards](../user-guide/dashboards/dashboards-in-openobserve.md), and [alerting and notifications](../user-guide/alerts/alerts.md). + +The OpenObserve plugin for Grafana is an optional integration. It enables OpenObserve to be added as a Grafana data source, making it possible to use existing Grafana dashboards or combine OpenObserve data with other monitoring systems. + +!!! note "Plugin maintenance status" + OpenObserve actively maintains this Grafana plugin. + +!!! note "Quick start" + If you are familiar with the Grafana plugin installation process, proceed to download the plugin from [here](https://zincsearch-releases.s3.us-west-2.amazonaws.com/zo_gp/zo_gp.tar.gz) and move to [this](#use-grafana-plugin) step. + + +## Install the OpenObserve plugin in Grafana + +??? "Prerequisite" + **Storage Requirements**
+ Grafana requires persistent storage for two things: configuration and data. + + - Configuration can be stored in a ConfigMap or Secret. + - Data must be stored in a database. Grafana supports SQLite, MySQL, and PostgreSQL. + Most Grafana installations run on a single node with SQLite. + + + **Deployment method**
+ In Kubernetes, you deploy applications using either a Deployment or StatefulSet. Since Grafana needs persistent storage for its data, your choice depends on which database you use: + + - **SQLite** stores data files inside the pod itself. Use a `StatefulSet` to preserve this data when the pod restarts. + - **MySQL or PostgreSQL** store data in an external database. Use a `Deployment` since the data persists outside the pod. + + !!! note "Note" + This guide uses a StatefulSet with SQLite. + +??? "Step 1: Create the configuration file" + Create a `grafana.ini` file with the following configurations. + + ```ini title="grafana.ini" linenums="1" hl_lines="4 9" + [date_formats] + default_timezone = UTC + [server] + root_url = https://grafana.yourdomain.com + + [plugins] + enable_alpha = true + app_tls_skip_verify_insecure = false + allow_loading_unsigned_plugins = openobserve + ``` + + **Important**: + + - In `Line 4`, you must update the `root_url` with the root URL of your Grafana installation. This is the URL that you will use to access grafana. For example, `https://grafana.yourdomain.com`. + - In `Line 9`, you must specify that Grafana should use the unsigned plugin `openobserve`.You will install this plugin using the `init container` in the `statefulset`. + + +??? "Step 2: Create a Kubernetes secret" + Once you have created the file, you can create a Kubernetes secret using the below command. + + ```bash linenums="1" + kubectl create secret generic grafana-config --from-file=grafana.ini + ``` + This secret is mounted into the Grafana container in the following StatefulSet YAML. + +??? "Step 3: Create `grafana_statefulset.yaml`" + This file includes an init container that downloads and configures the `openobserve` plugin for the Grafana container. + + ```yaml title="grafana_statefulset.yaml" linenums="1" + apiVersion: apps/v1 + kind: StatefulSet + metadata: + name: grafana + spec: + serviceName: "grafana" + replicas: 1 + selector: + matchLabels: + app: grafana + template: + metadata: + labels: + app: grafana + spec: + securityContext: + fsGroup: 2000 + runAsUser: 10000 + runAsGroup: 3000 + initContainers: + - name: openobserve-plugin-loader + image: wbitt/network-multitool + imagePullPolicy: IfNotPresent + command: + [ + "sh", + "-c", + "curl -o /var/lib/grafana/plugins/zo_gp.tar.gz https://zincsearch-releases.s3.us-west-2.amazonaws.com/zo_gp/zo_gp.tar.gz && cd /var/lib/grafana/plugins && tar -zxvf zo_gp.tar.gz", + ] + volumeMounts: + - name: grafana-base + mountPath: /var/lib/grafana + - name: grafana-plugins + mountPath: /var/lib/grafana/plugins + containers: + - name: grafana + image: grafana/grafana:latest + ports: + - containerPort: 3000 + name: grafana + volumeMounts: + - name: grafana-base + mountPath: /var/lib/grafana + - name: grafana-plugins + mountPath: /var/lib/grafana/plugins + - name: grafana-config + mountPath: /etc/grafana + volumes: + - name: grafana-base + persistentVolumeClaim: + claimName: grafana-base + - name: grafana-plugins + persistentVolumeClaim: + claimName: grafana-plugins + - name: grafana-config + secret: + defaultMode: 420 + secretName: grafana-config + volumeClaimTemplates: + - metadata: + name: grafana-base + spec: + accessModes: ["ReadWriteOnce"] + resources: + requests: + storage: 1Gi + - metadata: + name: grafana-plugins + spec: + accessModes: ["ReadWriteOnce"] + resources: + requests: + storage: 1Gi + --- + apiVersion: v1 + kind: Service + metadata: + name: grafana + spec: + ports: + - port: 3000 + targetPort: 3000 + selector: + app: grafana + + ``` +??? "Step 4: Deploy the `openobserve` plugin" + Run the following `kubectl apply` command to deploy the plugin: + ```bash linenums="1" + kubectl apply -f grafana_statefulset.yaml + ``` + +## Use the OpenObserve plugin in Grafana + +??? "Step 1: Set up data source" + From the Grafana interface, go to the **Data Sources** section and search for `openobserve`. + ![openobserve-plugin](../images/openobserve-plugin.png) + +??? "Step 2: Add OpenObserve data source server details" + 1. In the data source configuration screen, update the following fields: + + - In the **HTTP** section, add the **URL**. + - In the **Auth** section, enable **Basic Auth** toggle. + - Under **Basic Auth Details**, add the User ID and password. + 2. Click **Save and Test** to save the changes.
+ ![openobserve-data-source-config](../images/openobserve-data-source-config.png) + If everything is correct you should see a success message. + +??? "Step 3: Explore logs" + 1. Click the **Explore** menu and select `openobserve` as the data source. + 2. Select appropriate organization, stream, and time range. + 3. Click **Run Query**.
+ You should now be able to see the results. + ![alt text](../images/explore-logs-openobserve-plugin.png) + + +!!! note "Note" + If you want to explore metrics from OpenObserve in Grafana, you can set up OpenObserve as a Prometheus-compatible data source using an endpoint like [https://api.openobserve.ai/api/org_name/prometheus](https://api.openobserve.ai/api/org_name/prometheus). You do not need the plugin for this, as Grafana supports Prometheus natively. + + + + + + + + + + + + + + + + diff --git a/overrides/css/output.css b/overrides/css/output.css index d0423eb2..413bd930 100644 --- a/overrides/css/output.css +++ b/overrides/css/output.css @@ -457,7 +457,6 @@ legend { padding: 0; } -ol, ul, menu { list-style: none;