diff --git a/solutions/_snippets/edot-reference-arch.md b/solutions/_snippets/edot-reference-arch.md index 99963c5720..9324f9ff5a 100644 --- a/solutions/_snippets/edot-reference-arch.md +++ b/solutions/_snippets/edot-reference-arch.md @@ -1,7 +1,8 @@ The following diagrams show the reference architecture for OpenTelemetry with Elastic, depending on your deployment model. -:::::{tab-set} -::::{tab-item} Serverless +:::::{applies-switch} + +::::{applies-item} serverless: :::{image} /solutions/images/observability-apm-otel-distro-serverless.png :alt: APM data ingest path (Serverless) @@ -9,15 +10,23 @@ The following diagrams show the reference architecture for OpenTelemetry with El :::: -::::{tab-item} ECH +::::{applies-item} ess: -:::{image} /solutions/images/observability-apm-otel-distro-ech.png -:alt: APM data ingest path (ECH) -::: +- {applies_to}`stack: preview 9.2` + + :::{image} /solutions/images/ech-preview-motlp.png + :alt: APM data ingest path (ECH) + ::: + +- {applies_to}`stack: ga 9.1` + + :::{image} /solutions/images/observability-apm-otel-distro-ech.png + :alt: APM data ingest path (ECH) + ::: :::: -::::{tab-item} Self-managed +::::{applies-item} self: :::{image} /solutions/images/observability-apm-otel-distro-self-managed.png :alt: APM data ingest path (Self-managed) diff --git a/solutions/images/ech-preview-motlp.png b/solutions/images/ech-preview-motlp.png new file mode 100644 index 0000000000..b46ed9351e Binary files /dev/null and b/solutions/images/ech-preview-motlp.png differ diff --git a/solutions/observability/apm/get-started.md b/solutions/observability/apm/get-started.md index 1b0dbd141d..1b3c308b76 100644 --- a/solutions/observability/apm/get-started.md +++ b/solutions/observability/apm/get-started.md @@ -40,6 +40,8 @@ Follow these steps to send APM data to Elastic. :::{include} /solutions/_snippets/obs-serverless-project.md ::: +::::: + :::::{step} Add data using EDOT or APM Agents To send APM data to Elastic, you must install an Elastic Distribution of OpenTelemetry or an APM agent and configure it to send data to your project: diff --git a/solutions/observability/get-started/quickstart-elastic-cloud-otel-endpoint.md b/solutions/observability/get-started/quickstart-elastic-cloud-otel-endpoint.md index 3547c58c7a..2faa9ec8cf 100644 --- a/solutions/observability/get-started/quickstart-elastic-cloud-otel-endpoint.md +++ b/solutions/observability/get-started/quickstart-elastic-cloud-otel-endpoint.md @@ -1,28 +1,31 @@ --- -description: Learn how to use the Elastic Cloud Managed OTLP Endpoint to send logs, metrics, and traces to Elastic Observability. +description: Learn how to use the Elastic Cloud Managed OTLP Endpoint to send logs, metrics, and traces to Elastic Serverless and Elastic Cloud Hosted. mapped_pages: - https://www.elastic.co/guide/en/serverless/current/collect-data-with-native-otlp.html applies_to: serverless: + deployment: + ess: + stack: preview 9.2 --- -# Quickstart: Send data to the {{motlp}} +# Quickstart: Send OTLP data to Elastic Serverless or Elastic Cloud Hosted -The {{motlp}} is a fully managed offering exclusively for Elastic Cloud users that simplifies OpenTelemetry data ingestion. It provides an endpoint for OpenTelemetry SDKs and Collectors to send telemetry data, with Elastic handling scaling, data processing, and storage. Refer to [{{motlp}}](opentelemetry://reference/motlp.md) for more information. +You can send OpenTelemetry data to Elastic Serverless and Elastic Cloud Hosted using the {{motlp}} endpoint. -This endpoint is designed for the following use cases: +The {{motlp}} provides an endpoint for OpenTelemetry SDKs and Collectors to send telemetry data, with Elastic handling scaling, data processing, and storage. Refer to [{{motlp}}](opentelemetry://reference/motlp.md) for more information. + +The {{motlp}} is designed for the following use cases: * Logs & Infrastructure Monitoring: Logs forwarded in OTLP format and host and Kubernetes metrics in OTLP format. * APM: Application telemetry in OTLP format. -In this quickstart guide, you'll learn how to use the {{motlp}} to send logs, metrics, and traces to Elastic. - -## Prerequisites - -* An {{obs-serverless}} project. To learn more, refer to [create an Observability project](/solutions/observability/get-started.md). -* A system forwarding logs, metrics, or traces in OTLP (any EDOT Collector or SDK—EDOT or community). +Keep reading to learn how to use the {{motlp}} to send logs, metrics, and traces to your Serverless project or {{ech}} cluster. -## Get started +:::{note} +:applies_to: { ess:, stack: preview 9.2 } +The Managed OTLP endpoint might not be available in all {{ech}} regions during the Technical Preview. +::: ## Send data to Elastic @@ -30,67 +33,34 @@ Follow these steps to send data to Elastic using the {{motlp}}. ::::::{stepper} -:::::{step} Check the requirements - -To use the {{motlp}} you need the following: - -* An Elastic Observability Serverless project. Security projects are not yet supported. -* An OTLP-compliant shipper capable of forwarding logs, metrics, or traces in OTLP format. This can include the OpenTelemetry Collector (EDOT, Contrib, or other distributions), OpenTelemetry SDKs (EDOT, upstream, or other distributions), or any other forwarder that supports the OTLP protocol. +:::::{step} Retrieve your endpoint and API key -::::: - -:::::{step} Locate your {{motlp}} - -To retrieve your {{motlp}} endpoint address and an API key, follow these steps: +To retrieve your {{motlp}} endpoint address and API key, follow these steps: +::::{applies-switch} +:::{applies-item} serverless: 1. In {{ecloud}}, create an Observability project or open an existing one. -2. Select your project's name and then select **Manage project**. -3. Locate the **Connection alias** and select **Edit**. -4. Copy the **Managed OTLP endpoint** URL. - -% ## commented out until mOTLP on ECH is available -% ### Elastic Cloud on Elasticsearch ({{ech}}) -% 1. Open your deployment in the Elastic Cloud console. -% 2. Navigate to **Integrations** and find **OpenTelemetry** or **Managed OTLP**. -% 3. Copy the endpoint URL shown. -% ## Self-Managed -% For self-managed environments, you can deploy and expose an OTLP-compatible endpoint using the EDOT Collector as a gateway. Refer to [EDOT deployment docs](https://www.elastic.co/docs/reference/opentelemetry/edot-collector/modes#edot-collector-as-gateway). -% -% :::{note} -% Please reach out to support, and then Engineering can look into increasing it based on the license tier or for experimentation purposes. -% ::: - -::::: - -:::::{step} Create an API key +2. Go to **Add data**, select **Applications** and then select **OpenTelemetry**. +3. Copy the endpoint and authentication headers values. -Generate an API key with appropriate ingest privileges to authenticate OTLP traffic: - -1. In {{ecloud}}, go to **Manage project** → **API Keys**. -2. Select **Create API Key**. -3. Name the key. For example, `otlp-client`. -4. Edit the optional security settings. -5. Select **Create API Key**. -6. Copy the key to the clipboard. - -Add this key to your final API key string. For example: +Alternatively, you can retrieve the endpoint from the **Manage project** page and create an API key manually from the **API keys** page. +::: -``` -Authorization: ApiKey -``` +:::{applies-item} ess: +{applies_to}`stack: preview 9.2` +1. In {{ecloud}}, create an {{ech}} deployment or open an existing one. +2. Go to **Add data**, select **Applications** and then select **OpenTelemetry**. +3. Copy the endpoint and authentication headers values. -:::{important} -The API key copied from Kibana does not include the `ApiKey` scheme. Always prepend `ApiKey ` before using it in your configuration or encoding it for Kubernetes secrets. For example: - - - Correct: `Authorization: ApiKey abc123` - - Incorrect: `Authorization: abc123` +Alternatively, you can retrieve the endpoint from the **Manage project** page and create an API key manually from the **API keys** page. ::: +:::: ::::: -:::::{step} Send data to the {{motlp}} +:::::{step} Configure your OTLP shipper -The final step is to use the {{motlp}} endpoint and your Elastic API key to send data to {{ecloud}}. +The final step is to configure your Collector or SDK to use the {{motlp}} endpoint and your Elastic API key to send data to {{ecloud}}. ::::{tab-set} @@ -100,14 +70,11 @@ To send data to the {{motlp}} from the {{edot}} Collector or the contrib Collect ```yaml exporters: otlp: - endpoint: https:// <1> + endpoint: https:// headers: - Authorization: ApiKey <2> + Authorization: ApiKey ``` -1. The endpoint retrieved at [step 2](#locate-your-motlp) -2. The API key created at [step 3](#create-an-api-key) - Set the API key as an environment variable or directly in the configuration as shown in the example. ::: @@ -115,12 +82,9 @@ Set the API key as an environment variable or directly in the configuration as s To send data to the {{motlp}} from {{edot}} SDKs or contrib SDKs, set the following variables in your application's environment: ```bash -export OTEL_EXPORTER_OTLP_ENDPOINT="https://" <1> -export OTEL_EXPORTER_OTLP_HEADERS="Authorization=ApiKey " <2> +export OTEL_EXPORTER_OTLP_ENDPOINT="https://" +export OTEL_EXPORTER_OTLP_HEADERS="Authorization=ApiKey " ``` - -1. The endpoint retrieved at [step 2](#locate-your-motlp) -2. The API key created at [step 3](#create-an-api-key) ::: :::{tab-item} Kubernetes example @@ -141,14 +105,11 @@ Mount the secret as an environment variable or file, then reference it in your O ```yaml exporters: otlp: - endpoint: https:// <1> + endpoint: https:// headers: - Authorization: ${API_KEY} <2> + Authorization: ${API_KEY} ``` -1. The endpoint retrieved at [step 2](#locate-your-motlp) -2. The API key created at [step 3](#create-an-api-key) - And in your deployment spec: ```yaml @@ -173,15 +134,15 @@ When creating a Kubernetes secret, always encode the full string in Base64, incl ## Differences from the Elastic APM Endpoint -The Elastic Cloud Managed OTLP Endpoint ensures that OpenTelemetry data is stored without any schema translation, preserving both OpenTelemetry semantic conventions and resource attributes. It supports ingesting OTLP logs, metrics, and traces in a unified manner, ensuring consistent treatment across all telemetry data. This marks a significant improvement over the [existing functionality](/solutions/observability/apm/opentelemetry/index.md), which primarily focuses on traces and the APM use case. +The Elastic Cloud Managed OTLP Endpoint ensures that OpenTelemetry data is stored without any schema translation, preserving both OpenTelemetry semantic conventions and resource attributes. It supports ingesting OTLP logs, metrics, and traces in a unified manner, ensuring consistent treatment across all telemetry data. -## Troubleshoot +## Troubleshooting The following sections provide troubleshooting information for the {{motlp}}. -### I don't have a Collector or SDK running +### You don't have a Collector or SDK running -Don't have a collector or SDK running? Spin up an EDOT collector in just a few steps: +Don't have a collector or SDK running? Spin up an EDOT collector in few steps: * [Kubernetes Quickstart](/solutions/observability/get-started/opentelemetry/quickstart/serverless/k8s.md) * [Hosts & VMs Quickstart](/solutions/observability/get-started/opentelemetry/quickstart/serverless/hosts_vms.md) @@ -201,7 +162,7 @@ You must format your API key as `"Authorization": "ApiKey "` ### Error: too many requests -The Managed OTLP endpoint has per-project rate limits in place. If you hit this limit, reach out to our [support team](https://support.elastic.co). Refer to [Rate limiting](opentelemetry://reference/motlp.md#rate-limiting) for more information. +The Managed OTLP endpoint has per-project rate limits in place. If you reach this limit, reach out to our [support team](https://support.elastic.co). Refer to [Rate limiting](opentelemetry://reference/motlp.md#rate-limiting) for more information. ## Provide feedback @@ -209,6 +170,6 @@ Help improve the Elastic Cloud Managed OTLP Endpoint by sending us feedback in o For EDOT collector feedback, open an issue in the [elastic-agent repository](https://github.com/elastic/elastic-agent/issues). -## What's next? +## What's next Visualize your OpenTelemetry data. Learn more in [](/solutions/observability/otlp-visualize.md).