Add EDOT data streams documentation and clarify exporter/storage behavior

docs/reference/compatibility/data-streams.md

@@ -18,7 +18,9 @@
 
 This architecture is designed for scalable observability workloads. It supports dynamic attributes, reduces mapping complexity, and avoids issues like mapping explosions or manual dimension setup.
 
-EDOT uses Elasticsearch’s [Logs data stream (LogsDB)](docs-content://manage-data/data-store/data-streams/logs-data-stream.md) and [Time Series Data Streams (TSDS)](docs-content://manage-data/data-store/data-streams/time-series-data-stream-tsds.md) as storage backends. These are purpose-built to handle the scale and variety of observability data and improve the storage efficiency.
+EDOT uses {{es}}’s [Logs data stream (LogsDB)](docs-content://manage-data/data-store/data-streams/logs-data-stream.md) and [Time Series Data Streams (TSDS)](docs-content://manage-data/data-store/data-streams/time-series-data-stream-tsds.md) as storage backends. These are purpose-built to handle the scale and variety of observability data and improve the storage efficiency.
+
+This page provides a detailed comparison of EDOT data streams with classic {{product.apm}} and ECS-based integrations. For a practical reference on which data streams EDOT uses, exporter behavior, and storage engines, see [EDOT data streams](../data-streams.md).
 
 ## Logs and traces in LogsDB
 
@@ -29,7 +31,7 @@
 
 ## Metrics in TSDS
 
-Metric data is stored using Elasticsearch’s [TSDS](docs-content://manage-data/data-store/data-streams/time-series-data-stream-tsds.md). Benefits include:
+Metric data is stored using {{es}}’s [TSDS](docs-content://manage-data/data-store/data-streams/time-series-data-stream-tsds.md). Benefits include:
 
 * Efficient storage using columnar compression  
 * Fast aggregations 
@@ -40,19 +42,19 @@
 
 This table highlights key differences between classic Elastic APM data streams and EDOT with `mapping_mode: otel`:
 
-| Feature                   | Classic APM (ECS-based)                                                                                                          | EDOT (`mapping_mode: otel`)                                                                                                          |
+| Feature                   | Classic {{product.apm}} (ECS-based)                                                                                                          | EDOT (`mapping_mode: otel`)                                                                                                          |
 |---|---|---|
-| Index mode | General-purpose data streams (logs, traces, metrics) <br><br> TSDS is not supported for classic APM. | LogsDB (logs/traces), TSDS (metrics) |
+| Index mode | General-purpose data streams (logs, traces, metrics) <br><br> TSDS is not supported for classic {{product.apm}}. | LogsDB (logs/traces), TSDS (metrics) |
 | Mapping style | Nested objects are mapped as structured fields. Some exceptions exist, such as `labels.*` and `numeric_labels.*`, where dots in field names are replaced with underscores. <br><br> ECS supports multiple field types (keyword, long, double, date, boolean, and so on) as defined in the schema. | Native OpenTelemetry fields with `passthrough`, preserving types and structure. |
-| Attribute handling | Dynamic mapping. Custom attributes are stored under `labels.*` (strings) or `numeric_labels.*` (numbers); dots in field names are replaced with underscores. <br><br> See [Document examples - classic APM](#classic-apm) | Dynamic mapping with native types under `attributes.*`, preserving dots in field names. <br><br> See [Document examples - EDOT](#edot) |
+| Attribute handling | Dynamic mapping. Custom attributes are stored under `labels.*` (strings) or `numeric_labels.*` (numbers); dots in field names are replaced with underscores. <br><br> See [Document examples - classic {{product.apm}}](#classic-apm) | Dynamic mapping with native types under `attributes.*`, preserving dots in field names. <br><br> See [Document examples - EDOT](#edot) |
 
 ### Query compatibility with classic APM data streams
 
-EDOT is designed to make OpenTelemetry data queryable using many of the same field names as classic APM (ECS-based) data streams. This helps preserve compatibility with existing dashboards, saved searches, and queries.
+EDOT is designed to make OpenTelemetry data queryable using many of the same field names as classic {{product.apm}} (ECS-based) data streams. This helps preserve compatibility with existing dashboards, saved searches, and queries.
 
 Query compatibility is achieved through:
 
-* **`passthrough` fields:** Make nested OpenTelemetry fields available at the top level so they can be queried. For example, while the service name is stored at `resource.attributes.service.name`, you can query it as `service.name` (the same field name as the one used in the classic APM data stream).
+* **`passthrough` fields:** Make nested OpenTelemetry fields available at the top level so they can be queried. For example, while the service name is stored at `resource.attributes.service.name`, you can query it as `service.name` (the same field name as the one used in the classic {{product.apm}} data stream).
 * **Field aliases:** Map fields with different names in ECS and OpenTelemetry semantic conventions to a common query name to make migration easier.
 
 #### Limitations
@@ -120,7 +122,7 @@
 
 ## Comparison with ECS-based integrations
 
-While classic APM and EDOT represent two ingestion paths for application telemetry, Elastic’s integrations (for example Nginx, MySQL, Kubernetes) also produce ECS-based data streams for logs, metrics, and events. These use ECS mappings and integration-specific pipelines optimized for their domain.
+While classic {{product.apm}} and EDOT represent two ingestion paths for application telemetry, Elastic’s integrations (for example Nginx, MySQL, Kubernetes) also produce ECS-based data streams for logs, metrics, and events. These use ECS mappings and integration-specific pipelines optimized for their domain.
 
 | Stream type | Typical field layout | Custom attributes / dot notation |
 |--------------|----------------------|----------------------------------|
@@ -148,7 +150,7 @@
 
 ## Summary of all data stream types
 
-| Feature | Classic APM (ECS-based) | Integration ECS-based streams | EDOT (`mapping_mode: otel`) |
+| Feature | Classic {{product.apm}} (ECS-based) | Integration ECS-based streams | EDOT (`mapping_mode: otel`) |
 |----------|-------------------------|-------------------------------|-----------------------------|
 | **Index mode** | General-purpose data streams (logs, traces, metrics); TSDS not supported | ECS-style data streams (logs, metrics, events) using integrations | LogsDB for logs/traces, TSDS for metrics |
 | **Mapping style** | ECS object mappings; nested fields preserved. `labels.*` / `numeric_labels.*` flatten dots. | ECS mappings or integration-altered schemas (flattening, renaming). | OTel-native nested layout with `passthrough`, preserving types and structure. |

docs/reference/data-streams.md

@@ -0,0 +1,103 @@
+---
+navigation_title: Data streams
+description: Learn how EDOT stores traces, metrics, and logs in Elasticsearch. Understand OTel-native and ECS-compatible storage modes, exporter behavior, and differences between Managed OTLP Endpoint and local EDOT gateways.
+applies_to:
+  stack:
+  serverless:
+    observability:
+products:
+  - id: cloud-serverless
+  - id: observability
+  - id: edot-collector
+---
+
+# EDOT data streams
+
+{{edot}} writes telemetry data to two types of data stream schemas in {{es}}:
+
+- OTel-native data streams (default)
+- ECS-compatible data streams (for backwards compatibility with {{product.apm}})
+
+This page provides a practical reference for which data streams EDOT uses, exporter behavior, and storage engines. For a detailed comparison of OTel data streams with classic {{product.apm}} and ECS-based integrations, see [OTel data streams compared to classic {{product.apm}}](./compatibility/data-streams.md).
+
+To learn how to route OpenTelemetry (OTel) signals to custom data streams, see [Data stream routing](docs-content://solutions/observability/apm/opentelemetry/data-stream-routing.md).
+
+## Exporter behavior: `otel` and `ecs`
+
+EDOT may use two {{es}} exporters:
+
+- `otel`: sends data to OTel-native data streams  
+- `ecs`: sends data to ECS-compatible data streams  
+
+The `ecs` exporter ensures compatibility with legacy {{product.apm}} data formats.
+
+## Data streams used by a local deployment
+
+When EDOT runs as a local gateway or Collector, it writes telemetry data to different data streams depending on the storage mode and exporter configuration:
+
+### OTel-native mode
+
+This is the default mode. When ingesting standard OTel signals, EDOT writes to:
+
+| Signal | Data stream |
+|--------|-------------|
+| Traces | `traces-otel-*` |
+| Metrics | `metrics-otel-*` (TSDB-backed using `mode: time_series`) |
+| Logs | `logs-otel-*` |
+| Aggregated service destination metrics | `metrics-service_destination.[1m\|10m\|60m].otel-*` |
+
+These streams follow OTel naming conventions and field structures.
+
+### ECS-compatible mode
+
+When the `ecs` exporter is enabled, or when ingesting data from {{product.apm}} agents using `elasticapmintake`, EDOT writes to:
+
+- `traces-apm-*`
+- `traces-apm.sampled-*`
+- `metrics-apm.internal-*`
+- `metrics-apm.*-*`
+- `metrics-apm.service_destination.[interval]-*`
+- `logs-apm.error-*`
+- `logs-apm.app.*-*`
+
+These match legacy {{product.apm-server}} behavior.
+
+## Managed OTLP Endpoint
+
+The {{motlp}} follows the same stream selection logic as local EDOT deployments:
+
+- OTel-native streams for standard OTel telemetry  
+- ECS-compatible streams for {{product.apm}}-formatted events  
+
+The main difference is that {{motlp}} is a managed cloud service that handles ingestion and storage, whereas local EDOT deployments require you to manage the Collector infrastructure yourself. Ingest and storage behavior remain consistent between both approaches.
+
+## Field duplication
+
+You may notice fields like:
+
+- `resource.attributes.service.name`
+- `service.name`
+
+These are not true duplicates. Here's why:
+
+- OTel defines attributes at multiple levels (for example: resource, span).
+- EDOT stores resource attributes under `resource.attributes.*`, according to OTel specifications.
+- {{kib}} dashboards and {{product.apm}} UI rely on ECS-style top-level fields like `service.name`.
+
+To bridge these models, EDOT aliases or copies key fields so that:
+
+- Dashboards continue to function  
+- Filters and searches behave consistently  
+- OTel structure remains intact for downstream compatibility  
+
+This duplication is intentional.
+
+## Storage engines used by EDOT
+
+| Data type | Storage engine |
+|-----------|----------------|
+| Metrics   | TSDB (`mode: time_series`) |
+| Logs      | LogsDB |
+| Traces    | Standard time-series model |
+
+EDOT uses the same index templates and mappings as the rest of the Observability solution.

docs/reference/toc.yml

@@ -1,3 +1,3 @@
 project: 'Elastic Distributions of OpenTelemetry (EDOT)'
 toc:
   - file: index.md
@@ -19,6 +19,7 @@
       - file: compatibility/nomenclature.md
       - file: compatibility/data-streams.md 
   - file: motlp.md
+  - file: data-streams.md
   - file: central-configuration.md
   - file: edot-sdks/index.md
   - file: edot-cloud-forwarder/index.md