Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 16 additions & 7 deletions solutions/_snippets/edot-reference-arch.md
Original file line number Diff line number Diff line change
@@ -1,23 +1,32 @@
The following diagrams show the reference architecture for OpenTelemetry with Elastic, depending on your deployment model.

:::::{tab-set}
::::{tab-item} Serverless
:::::{applies-switch}

::::{applies-item} serverless:

:::{image} /solutions/images/observability-apm-otel-distro-serverless.png
:alt: APM data ingest path (Serverless)
:::

::::

::::{tab-item} ECH
::::{applies-item} ess:

:::{image} /solutions/images/observability-apm-otel-distro-ech.png
:alt: APM data ingest path (ECH)
:::
- {applies_to}`stack: preview 9.2`

:::{image} /solutions/images/ech-preview-motlp.png
:alt: APM data ingest path (ECH)
:::

- {applies_to}`stack: ga 9.1`

:::{image} /solutions/images/observability-apm-otel-distro-ech.png
:alt: APM data ingest path (ECH)
:::

::::

::::{tab-item} Self-managed
::::{applies-item} self:

:::{image} /solutions/images/observability-apm-otel-distro-self-managed.png
:alt: APM data ingest path (Self-managed)
Expand Down
Binary file added solutions/images/ech-preview-motlp.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
2 changes: 2 additions & 0 deletions solutions/observability/apm/get-started.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,8 @@ Follow these steps to send APM data to Elastic.
:::{include} /solutions/_snippets/obs-serverless-project.md
:::

:::::

:::::{step} Add data using EDOT or APM Agents

To send APM data to Elastic, you must install an Elastic Distribution of OpenTelemetry or an APM agent and configure it to send data to your project:
Expand Down
Original file line number Diff line number Diff line change
@@ -1,96 +1,66 @@
---
description: Learn how to use the Elastic Cloud Managed OTLP Endpoint to send logs, metrics, and traces to Elastic Observability.
description: Learn how to use the Elastic Cloud Managed OTLP Endpoint to send logs, metrics, and traces to Elastic Serverless and Elastic Cloud Hosted.
mapped_pages:
- https://www.elastic.co/guide/en/serverless/current/collect-data-with-native-otlp.html
applies_to:
serverless:
deployment:
ess:
stack: preview 9.2
---

# Quickstart: Send data to the {{motlp}}
# Quickstart: Send OTLP data to Elastic Serverless or Elastic Cloud Hosted

The {{motlp}} is a fully managed offering exclusively for Elastic Cloud users that simplifies OpenTelemetry data ingestion. It provides an endpoint for OpenTelemetry SDKs and Collectors to send telemetry data, with Elastic handling scaling, data processing, and storage. Refer to [{{motlp}}](opentelemetry://reference/motlp.md) for more information.
You can send OpenTelemetry data to Elastic Serverless and Elastic Cloud Hosted using the {{motlp}} endpoint.

This endpoint is designed for the following use cases:
The {{motlp}} provides an endpoint for OpenTelemetry SDKs and Collectors to send telemetry data, with Elastic handling scaling, data processing, and storage. Refer to [{{motlp}}](opentelemetry://reference/motlp.md) for more information.

The {{motlp}} is designed for the following use cases:

* Logs & Infrastructure Monitoring: Logs forwarded in OTLP format and host and Kubernetes metrics in OTLP format.
* APM: Application telemetry in OTLP format.

In this quickstart guide, you'll learn how to use the {{motlp}} to send logs, metrics, and traces to Elastic.

## Prerequisites

* An {{obs-serverless}} project. To learn more, refer to [create an Observability project](/solutions/observability/get-started.md).
* A system forwarding logs, metrics, or traces in OTLP (any EDOT Collector or SDK—EDOT or community).
Keep reading to learn how to use the {{motlp}} to send logs, metrics, and traces to your Serverless project or {{ech}} cluster.

## Get started
:::{note}
:applies_to: { ess:, stack: preview 9.2 }
The Managed OTLP endpoint might not be available in all {{ech}} regions during the Technical Preview.
:::

## Send data to Elastic

Follow these steps to send data to Elastic using the {{motlp}}.

::::::{stepper}

:::::{step} Check the requirements

To use the {{motlp}} you need the following:

* An Elastic Observability Serverless project. Security projects are not yet supported.
* An OTLP-compliant shipper capable of forwarding logs, metrics, or traces in OTLP format. This can include the OpenTelemetry Collector (EDOT, Contrib, or other distributions), OpenTelemetry SDKs (EDOT, upstream, or other distributions), or any other forwarder that supports the OTLP protocol.
:::::{step} Retrieve your endpoint and API key

:::::

:::::{step} Locate your {{motlp}}

To retrieve your {{motlp}} endpoint address and an API key, follow these steps:
To retrieve your {{motlp}} endpoint address and API key, follow these steps:

::::{applies-switch}
:::{applies-item} serverless:
1. In {{ecloud}}, create an Observability project or open an existing one.
2. Select your project's name and then select **Manage project**.
3. Locate the **Connection alias** and select **Edit**.
4. Copy the **Managed OTLP endpoint** URL.

% ## commented out until mOTLP on ECH is available
% ### Elastic Cloud on Elasticsearch ({{ech}})
% 1. Open your deployment in the Elastic Cloud console.
% 2. Navigate to **Integrations** and find **OpenTelemetry** or **Managed OTLP**.
% 3. Copy the endpoint URL shown.
% ## Self-Managed
% For self-managed environments, you can deploy and expose an OTLP-compatible endpoint using the EDOT Collector as a gateway. Refer to [EDOT deployment docs](https://www.elastic.co/docs/reference/opentelemetry/edot-collector/modes#edot-collector-as-gateway).
%
% :::{note}
% Please reach out to support, and then Engineering can look into increasing it based on the license tier or for experimentation purposes.
% :::

:::::

:::::{step} Create an API key
2. Go to **Add data**, select **Applications** and then select **OpenTelemetry**.
3. Copy the endpoint and authentication headers values.

Generate an API key with appropriate ingest privileges to authenticate OTLP traffic:

1. In {{ecloud}}, go to **Manage project** → **API Keys**.
2. Select **Create API Key**.
3. Name the key. For example, `otlp-client`.
4. Edit the optional security settings.
5. Select **Create API Key**.
6. Copy the key to the clipboard.

Add this key to your final API key string. For example:
Alternatively, you can retrieve the endpoint from the **Manage project** page and create an API key manually from the **API keys** page.
:::

```
Authorization: ApiKey <your-api-key>
```
:::{applies-item} ess:
{applies_to}`stack: preview 9.2`
1. In {{ecloud}}, create an {{ech}} deployment or open an existing one.
2. Go to **Add data**, select **Applications** and then select **OpenTelemetry**.
3. Copy the endpoint and authentication headers values.

:::{important}
The API key copied from Kibana does not include the `ApiKey` scheme. Always prepend `ApiKey ` before using it in your configuration or encoding it for Kubernetes secrets. For example:

- Correct: `Authorization: ApiKey abc123`
- Incorrect: `Authorization: abc123`
Alternatively, you can retrieve the endpoint from the **Manage project** page and create an API key manually from the **API keys** page.
:::
::::

:::::

:::::{step} Send data to the {{motlp}}
:::::{step} Configure your OTLP shipper

The final step is to use the {{motlp}} endpoint and your Elastic API key to send data to {{ecloud}}.
The final step is to configure your Collector or SDK to use the {{motlp}} endpoint and your Elastic API key to send data to {{ecloud}}.

::::{tab-set}

Expand All @@ -100,27 +70,21 @@ To send data to the {{motlp}} from the {{edot}} Collector or the contrib Collect
```yaml
exporters:
otlp:
endpoint: https://<motlp-endpoint> <1>
endpoint: https://<motlp-endpoint>
headers:
Authorization: ApiKey <your-api-key> <2>
Authorization: ApiKey <your-api-key>
```

1. The endpoint retrieved at [step 2](#locate-your-motlp)
2. The API key created at [step 3](#create-an-api-key)

Set the API key as an environment variable or directly in the configuration as shown in the example.
:::

:::{tab-item} OpenTelemetry SDK example
To send data to the {{motlp}} from {{edot}} SDKs or contrib SDKs, set the following variables in your application's environment:

```bash
export OTEL_EXPORTER_OTLP_ENDPOINT="https://<motlp-endpoint>" <1>
export OTEL_EXPORTER_OTLP_HEADERS="Authorization=ApiKey <your-api-key>" <2>
export OTEL_EXPORTER_OTLP_ENDPOINT="https://<motlp-endpoint>"
export OTEL_EXPORTER_OTLP_HEADERS="Authorization=ApiKey <your-api-key>"
```

1. The endpoint retrieved at [step 2](#locate-your-motlp)
2. The API key created at [step 3](#create-an-api-key)
:::

:::{tab-item} Kubernetes example
Expand All @@ -141,14 +105,11 @@ Mount the secret as an environment variable or file, then reference it in your O
```yaml
exporters:
otlp:
endpoint: https://<motlp-endpoint> <1>
endpoint: https://<motlp-endpoint>
headers:
Authorization: ${API_KEY} <2>
Authorization: ${API_KEY}
```

1. The endpoint retrieved at [step 2](#locate-your-motlp)
2. The API key created at [step 3](#create-an-api-key)

And in your deployment spec:

```yaml
Expand All @@ -173,15 +134,15 @@ When creating a Kubernetes secret, always encode the full string in Base64, incl

## Differences from the Elastic APM Endpoint

The Elastic Cloud Managed OTLP Endpoint ensures that OpenTelemetry data is stored without any schema translation, preserving both OpenTelemetry semantic conventions and resource attributes. It supports ingesting OTLP logs, metrics, and traces in a unified manner, ensuring consistent treatment across all telemetry data. This marks a significant improvement over the [existing functionality](/solutions/observability/apm/opentelemetry/index.md), which primarily focuses on traces and the APM use case.
The Elastic Cloud Managed OTLP Endpoint ensures that OpenTelemetry data is stored without any schema translation, preserving both OpenTelemetry semantic conventions and resource attributes. It supports ingesting OTLP logs, metrics, and traces in a unified manner, ensuring consistent treatment across all telemetry data.

## Troubleshoot
## Troubleshooting

The following sections provide troubleshooting information for the {{motlp}}.

### I don't have a Collector or SDK running
### You don't have a Collector or SDK running

Don't have a collector or SDK running? Spin up an EDOT collector in just a few steps:
Don't have a collector or SDK running? Spin up an EDOT collector in few steps:

* [Kubernetes Quickstart](/solutions/observability/get-started/opentelemetry/quickstart/serverless/k8s.md)
* [Hosts & VMs Quickstart](/solutions/observability/get-started/opentelemetry/quickstart/serverless/hosts_vms.md)
Expand All @@ -201,14 +162,14 @@ You must format your API key as `"Authorization": "ApiKey <api-key-value-here>"`

### Error: too many requests

The Managed OTLP endpoint has per-project rate limits in place. If you hit this limit, reach out to our [support team](https://support.elastic.co). Refer to [Rate limiting](opentelemetry://reference/motlp.md#rate-limiting) for more information.
The Managed OTLP endpoint has per-project rate limits in place. If you reach this limit, reach out to our [support team](https://support.elastic.co). Refer to [Rate limiting](opentelemetry://reference/motlp.md#rate-limiting) for more information.

## Provide feedback

Help improve the Elastic Cloud Managed OTLP Endpoint by sending us feedback in our [discussion forum](https://discuss.elastic.co/c/apm) or [community Slack](https://elasticstack.slack.com/signup#/domain-signup).

For EDOT collector feedback, open an issue in the [elastic-agent repository](https://github.com/elastic/elastic-agent/issues).

## What's next?
## What's next

Visualize your OpenTelemetry data. Learn more in [](/solutions/observability/otlp-visualize.md).
Loading