Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file added docs/images/action-tab.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/enterprise-branding.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/filter-pipeline.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/pipeline-list-view.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/search-pipeline.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/sort-columns.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/theme-config.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
1 change: 1 addition & 0 deletions docs/user-guide/management/.pages
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
nav:
- General Settings: general-settings.md
- Management Overview: index.md
- Streaming Search: streaming-search.md
- Streaming Aggregation: aggregation-cache.md
Expand Down
55 changes: 55 additions & 0 deletions docs/user-guide/management/general-settings.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
## General settings
This page explains how to use the settings available under General Settings in OpenObserve. These settings allow you to configure scrape interval, theme colors, and Enterprise branding.

## What you can do in general settings
General settings provides the following configuration options:

- Scrape interval
- Theme configuration for light and dark modes
- Enterprise branding features such as custom logo text and custom logo

!!! note "Note"
`_meta` organizations can access all settings. Regular organizations see only the options allowed for their specific license and permissions.

### Scrape interval
> This setting is available for all organizations.

![theme-config](../../images/theme-config.png)
The scrape interval defines how often the monitoring system collects metrics. Enter the number of seconds and select Save.

### Theme configuration
Theme configuration allows you to manage the default colors for light mode and dark mode for your organization. When you save these values, they are applied across the UI for all users in the organization.
![theme-config](../../images/theme-config.png)

You can set the following fields:

- Light mode color
- Dark mode color

Select **Save** to apply the changes.

### Theme configuration using environment variables
You can also configure the default theme colors at the system level. These values become the initial defaults for all organizations.

- `ZO_DEFAULT_THEME_LIGHT_MODE_COLOR`: Sets the default color used in light mode. Default:"#3F7994"
- `ZO_DEFAULT_THEME_DARK_MODE_COLOR`: Sets the default color used in dark mode. Default: "#5B9FBE"

If these variables are set, the UI loads these colors automatically unless overridden by an organization through the General Settings page.

### Enterprise branding
Enterprise branding options appear only in the `_meta` org when Enterprise features are enabled.
![enterprise-branding](../../images/enterprise-branding.png)
<br>

**Custom logo text**
Use this field to replace the default OpenObserve text shown in the header. Enter the required text and save the changes.
<br>

**Custom logo**
Upload a custom branding logo to replace the default OpenObserve logo in the header.

- Supported file formats: PNG, JPG, JPEG, GIF, BMP
- Maximum file size: 20 KB
- Maximum dimensions: 150 by 30 pixels

After uploading the file, select the save icon to apply the logo.
79 changes: 41 additions & 38 deletions docs/user-guide/management/lincense.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ The license defines the daily ingestion capacity for your OpenObserve installati

!!! note "Who can access"

OpenObserve provides access to license information based on two roles. Your role decides what you can see and what actions you can perform.
OpenObserve provides access to license information based on the following roles. Your role decides what you can see and what actions you can perform.

| **Role** | **What you can do** | **What you cannot do** |
| ---------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
Expand All @@ -17,41 +17,44 @@ The license defines the daily ingestion capacity for your OpenObserve installati
## How to manage licenses
To perform the following tasks, you must have appropriate permission. Review the **Who can access** section to know more.

### View license information
![about-page](../../images/about-page-license.png)

1. Click the Help (?) icon in the top-right corner.
2. Click **About**.
2. In the **Enterprise License Details** section, review license status, ingestion limit, and daily usage.

### Request a new license

1. In the **Enterprise License Details** section of the **About** page, click **Manage License**.
![about-page](../../images/about-page-license.png)
2. It takes you to the License page under **Management** in the `_meta` organization.
![license-management](../../images/license-management.png)
3. Click **Request New License**.
4. Complete the form by following the guidelines mentioned below and click **Submit Request**.

!!! note "Guideline for completing the license request form"
Use the information below to complete each field accurately when requesting a new license through the License Server page.
![license-server](../../images/license-server.png)

- **Installation ID**: The Installation ID field is usually prefilled by the system. If it is not prefilled, copy it from the License page under Management in the meta organization.
- **Contact Name and Email Address**: Enter the name and email address of the primary contact person for your organization. This person will receive the license key and future communication related to license renewal or verification.
- **Address**: Provide the official address of your organization. This must be a valid, complete postal address because it is used for verification and record keeping.
- **Use case**: Describe how you plan to use OpenObserve. Provide clear details about your expected data volume or operational requirements.
**Ingestion limit**: Select a limit based on the actual ingestion volume of your installation. To determine the correct value, review the usage stream in OpenObserve to identify your peak daily ingestion. Choose a limit that is higher than your highest observed ingestion volume to avoid data loss or ingestion failures.
- **Additional email addresses**: List additional recipients who should receive notifications related to license issuance or updates.
- **Base URLs**: Enter the base URLs used by your OpenObserve installation. The License Server accepts multiple base URLs because some installations may operate from more than one domain. Add each URL that users use to access your OpenObserve instance.


### Add a new license key
![add-new-license-key](../../images/add-new-license-key.png)

1. In the `_meta` organization, navigate to **Management** from the top-right corner.
2. Select **License**.
3. Select **Add New License Key**.
4. Paste the license key in the **Update License Key** field.
5. Select **Update License**.
??? "View license information"
### View license information
![about-page](../../images/about-page-license.png)

1. Click the Help (?) icon in the top-right corner.
2. Click **About**.
2. In the **Enterprise License Details** section, review license status, ingestion limit, and daily usage.

??? "Request a new license"
### Request a new license

1. In the **Enterprise License Details** section of the **About** page, click **Manage License**.
![about-page](../../images/about-page-license.png)
2. It takes you to the License page under **Management** in the `_meta` organization.
![license-management](../../images/license-management.png)
3. Click **Request New License**.
4. Complete the form by following the guidelines mentioned below and click **Submit Request**.

!!! note "Guideline for completing the license request form"
Use the information below to complete each field accurately when requesting a new license through the License Server page.
![license-server](../../images/license-server.png)

- **Installation ID**: The Installation ID field is usually prefilled by the system. If it is not prefilled, copy it from the License page under Management in the meta organization.
- **Contact Name and Email Address**: Enter the name and email address of the primary contact person for your organization. This person will receive the license key and future communication related to license renewal or verification.
- **Address**: Provide the official address of your organization. This must be a valid, complete postal address because it is used for verification and record keeping.
- **Use case**: Describe how you plan to use OpenObserve. Provide clear details about your expected data volume or operational requirements.
- **Ingestion limit**: Select a limit based on the actual ingestion volume of your installation. To determine the correct value, review the usage stream in OpenObserve to identify your peak daily ingestion. Choose a limit that is higher than your highest observed ingestion volume to avoid data loss or ingestion failures.
- **Additional email addresses**: List additional recipients who should receive notifications related to license issuance or updates.
- **Base URLs**: Enter the base URLs used by your OpenObserve installation. The License Server accepts multiple base URLs because some installations may operate from more than one domain. Add each URL that users use to access your OpenObserve instance.


??? "Add a new license key"
### Add a new license key
![add-new-license-key](../../images/add-new-license-key.png)

1. In the `_meta` organization, navigate to **Management** from the top-right corner.
2. Select **License**.
3. Select **Add New License Key**.
4. Paste the license key in the **Update License Key** field.
5. Select **Update License**.

Original file line number Diff line number Diff line change
Expand Up @@ -91,7 +91,7 @@ Use this guide to create and run real-time pipelines. Real-time pipelines proces
A destination defines where the processed events are written. You can forward data to another stream or to an external system.

**To add a stream destination:**
![stream-destination](../../images/stream-destination.png)
![external-destination](../../images/external-destination.png)

1. Drag a **Stream** node into the editor.
2. In the **Associate Stream** panel, configure the destination stream.
Expand All @@ -113,7 +113,7 @@ Use this guide to create and run real-time pipelines. Real-time pipelines proces


**To add an external destination:**
![external-destination](../../images/external-destination.png)
![stream-destination](../../images/stream-destination.png)

1. Drag a **Remote** node into the editor.
2. In the **External Destination** panel, either select an existing destination or enable **Create new Destination** and fill in the required details such as name, URL, method, output format, and headers.
Expand Down
126 changes: 72 additions & 54 deletions docs/user-guide/pipelines/manage-pipelines.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,88 +3,106 @@ description: >-
Manage, filter, search, edit, pause, or delete real-time and scheduled
pipelines in OpenObserve using the intuitive pipeline list and actions panel.
---
This page describes how to manage real-time and scheduled pipelines in OpenObserve. You can filter, search, view, edit, pause, resume, or delete pipelines from a single interface.
This page describes how to manage real-time and scheduled pipelines in OpenObserve.

## Pipeline management options
You can filter, search, view, edit, pause, resume, or delete pipelines from a single interface.
![pipeline management](../../images/manage_pipelines.png)

## Filter pipelines
Use the filter options **All**, **Real-Time**, and **Scheduled** to view only the type of pipelines you want to work with.
??? "Filter pipelines"
## Filter pipelines
Use the filter options **All**, **Real-Time**, and **Scheduled** to view only the type of pipelines you want to work with.
![filter-pipeline](../../images/filter-pipeline.png)

## Search pipelines
Use the search bar to locate pipelines by name or attributes.
??? "Search pipelines"
## Search pipelines
Use the search bar to locate pipelines by name or attributes.
![search-pipeline](../../images/search-pipeline.png)

## Pipeline list view
The pipeline list displays key information for each pipeline.
??? "Pipeline list view"
## Pipeline list view
The pipeline list displays key information for each pipeline.
![pipeline-list-view](../../images/pipeline-list-view.png)

- **Pipeline Name**: Identifies the pipeline.
- **Type**: Indicates whether the pipeline is real-time or scheduled.
- **Stream Name**: Shows the associated source stream.
- **Stream Type**: Indicates the type of data in the stream such as logs, metrics, traces, or enrichment tables.
- **Frequency**: Displays how often the scheduled pipeline runs.
- **Period**: Shows the time duration used for each execution window of a scheduled pipeline.
- **Cron**: Indicates whether the pipeline is configured using a cron expression.
- **Pipeline Name**: Identifies the pipeline.
- **Type**: Indicates whether the pipeline is real-time or scheduled.
- **Stream Name**: Shows the associated source stream.
- **Stream Type**: Indicates the type of data in the stream such as logs, metrics, traces, or enrichment tables.
- **Frequency**: Displays how often the scheduled pipeline runs.
- **Period**: Shows the time duration used for each execution window of a scheduled pipeline.
- **Cron**: Indicates whether the pipeline is configured using a cron expression.

## Actions tab
??? "Actions tab"
## Actions tab
Each pipeline row includes the following actions.
![action-tab](../../images/action-tab.png)

Each pipeline row includes the following actions.
- Start or pause pipeline
- Edit pipeline
- Export pipeline configuration
- Delete pipeline
- View pipeline details

- Start or pause pipeline
- Edit pipeline
- Export pipeline configuration
- Delete pipeline
- View pipeline details
### Pause and resume pipelines

### Pause and resume pipelines
You can pause both real-time and scheduled pipelines. <br>

You can pause both real-time and scheduled pipelines. <br>
**Pausing a pipeline:**

**Pausing a pipeline:**
- When paused, the pipeline stops executing on its scheduled intervals.
- The system preserves the exact timestamp when the pause occurred.
- Pipeline configuration and state are maintained during the pause.

- When paused, the pipeline stops executing on its scheduled intervals.
- The system preserves the exact timestamp when the pause occurred.
- Pipeline configuration and state are maintained during the pause.
**Resuming a real-time pipeline**:

**Resuming a real-time pipeline**:
- Real-time pipelines resume instantly with one click.
- No dialog or additional options are shown.
- Processing continues from the current moment.

- Real-time pipelines resume instantly with one click.
- No dialog or additional options are shown.
- Processing continues from the current moment.

**Resuming a scheduled pipeline**:

**Resuming a scheduled pipeline**:
Scheduled pipelines display a Resume Pipeline Ingestion dialog with two options.
![Resume pipeline](../../images/resume-scheduled-pipeline.png)

Scheduled pipelines display a Resume Pipeline Ingestion dialog with two options.
![Resume pipeline](../../images/resume-scheduled-pipeline.png)
- **Continue from where it paused:**

- **Continue from where it paused:**
- Processes all data from the pause timestamp to the current time.
- Maintains complete data continuity with no gaps.
- May consume significant system resources for long pause periods.

- Processes all data from the pause timestamp to the current time.
- Maintains complete data continuity with no gaps.
- May consume significant system resources for long pause periods.
> **Note:** Use the **Continue from where it paused** option, when data completeness is critical.

> **Note:** Use the **Continue from where it paused** option, when data completeness is critical.
- **Start from now:**

- **Start from now:**
- Begins processing from the current timestamp.
- Creates a data gap between the pause and resume timestamps.
- Provides immediate resumption with minimal resource usage.

- Begins processing from the current timestamp.
- Creates a data gap between the pause and resume timestamps.
- Provides immediate resumption with minimal resource usage.
> **Note:** Use the **Start from now** option, when data gaps are acceptable.

> **Note:** Use the **Start from now** option, when data gaps are acceptable.
### Pipeline error
If a pipeline encounters an error, an error icon appears in the **Actions** column.
![pipeline-error](../../images/pipeline-error.png)
Selecting this icon opens a dialog that displays the full error summary.
![pipeline-error-view](../../images/pipeline-error-view.png)

### Pipeline error
If a pipeline encounters an error, an error icon appears in the **Actions** column.
![pipeline-error](../../images/pipeline-error.png)
Selecting this icon opens a dialog that displays the full error summary.
![pipeline-error-view](../../images/pipeline-error-view.png)
??? "Pipeline history"
## Pipeline history
You can view past executions, errors, and performance statistics in the Pipeline History page.

## Pipeline history
You can view past executions, errors, and performance statistics in the Pipeline History page.
The [Pipeline History](https://openobserve.ai/docs/user-guide/pipelines/pipeline-history/) page provides a complete timeline of runs with detailed execution information.

The [Pipeline History](https://openobserve.ai/docs/user-guide/pipelines/pipeline-history/) page provides a complete timeline of runs with detailed execution information.
??? "Import and export pipelines"
## Import and export pipelines
You can export any pipeline and import pipeline configurations through the Import and **Export** options. For more details, see [Import and export pipelines](https://openobserve.ai/docs/user-guide/pipelines/import-and-export-pipelines/).

??? "Sort columns"
## Sort columns
![sort-columns](../../images/sort-columns.png)

You can sort the Pipelines table by selecting any column header. The table supports ascending, descending, and alphabetical sorting depending on the data in that column.

## Import and export pipelines
You can export any pipeline and import pipeline configurations through the Import and **Export** options. For more details, see [Import and export pipelines](https://openobserve.ai/docs/user-guide/pipelines/import-and-export-pipelines/).

## Related links
- [Pipelines in OpenObserve](../pipelines/pipelines/)
Expand Down
4 changes: 2 additions & 2 deletions docs/user-guide/pipelines/pipelines.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
This page provides an overview of pipelines, their types, and how they work.<br>

## What pipelines do
Pipelines let you control what happens to your data after it is ingested into OpenObserve. They provide a clear way to read data from a source, apply changes, and send the processed output to a destination. Pipelines work continuously in the background and ensure that your data is transformed or routed exactly as required.
## Overview
Pipelines let you control what happens to your data after it is ingested into OpenObserve. They provide a clear way to read data from a source, apply changes, and send the processed output to a destination.
![Pipelines in OpenObserve](../../images/pipeline-new-editor.png)

!!! note "Who can access pipelines"
Expand Down
2 changes: 1 addition & 1 deletion docs/user-guide/traces/.pages
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,4 @@ nav:

- Traces Overview: index.md
- Traces in OpenObserve: traces.md

- Service Graph: service-graph.md
Loading