diff --git a/raw-migrated-files/docs-content/serverless/observability-apm-agents-opentelemetry-opentelemetry-native-support.md b/raw-migrated-files/docs-content/serverless/observability-apm-agents-opentelemetry-opentelemetry-native-support.md index 66111916ad..49ad7b66a1 100644 --- a/raw-migrated-files/docs-content/serverless/observability-apm-agents-opentelemetry-opentelemetry-native-support.md +++ b/raw-migrated-files/docs-content/serverless/observability-apm-agents-opentelemetry-opentelemetry-native-support.md @@ -8,8 +8,8 @@ This is one of several approaches you can use to integrate Elastic with OpenTele Elastic natively supports the OpenTelemetry protocol (OTLP). This means trace data and metrics collected from your applications and infrastructure can be sent directly to Elastic. -* Send data to Elastic from an upstream [OpenTelemetry Collector](../../../solutions/observability/apps/upstream-opentelemetry-collectors-language-sdks.md#observability-apm-agents-opentelemetry-opentelemetry-native-support-send-data-from-an-upstream-opentelemetry-collector) -* Send data to Elastic from an upstream [OpenTelemetry language SDK](../../../solutions/observability/apps/upstream-opentelemetry-collectors-language-sdks.md#observability-apm-agents-opentelemetry-opentelemetry-native-support-send-data-from-an-upstream-opentelemetry-sdk) +* Send data to Elastic from an upstream [OpenTelemetry Collector](../../../solutions/observability/apps/upstream-opentelemetry-collectors-language-sdks.md) +* Send data to Elastic from an upstream [OpenTelemetry language SDK](../../../solutions/observability/apps/upstream-opentelemetry-collectors-language-sdks.md) ## Send data from an upstream OpenTelemetry Collector [observability-apm-agents-opentelemetry-opentelemetry-native-support-send-data-from-an-upstream-opentelemetry-collector] @@ -105,7 +105,7 @@ java -javaagent:/path/to/opentelemetry-javaagent-all.jar \ `OTEL_EXPORTER_OTLP_HEADERS` : Authorization header that includes the Elastic APM API key: `"Authorization=ApiKey an_api_key"`. Note the required space between `ApiKey` and `an_api_key`. - For information on how to format an API key, refer to [Secure communication with APM agents](../../../solutions/observability/apps/use-apm-securely.md#observability-apm-keep-data-secure-secure-communication-with-apm-agents). + For information on how to format an API key, refer to [Secure communication with APM agents](../../../solutions/observability/apps/use-apm-securely.md). ::::{note} If you are using a version of the Python OpenTelemetry agent *before* 1.27.0, the content of the header *must* be URL-encoded. You can use the Python standard library’s `urllib.parse.quote` function to encode the content of the header. @@ -121,7 +121,7 @@ java -javaagent:/path/to/opentelemetry-javaagent-all.jar \ -You are now ready to collect traces and [metrics](../../../solutions/observability/apps/collect-metrics.md) before [verifying metrics](../../../solutions/observability/apps/collect-metrics.md#open-telemetry-verify-metrics) and [visualizing metrics](../../../solutions/observability/apps/collect-metrics.md#open-telemetry-visualize). +You are now ready to collect traces and [metrics](../../../solutions/observability/apps/collect-metrics.md) before [verifying metrics](../../../solutions/observability/apps/collect-metrics.md#apm-open-telemetry-verify-metrics) and [visualizing metrics](../../../solutions/observability/apps/collect-metrics.md#apm-open-telemetry-visualize). ## Proxy requests to Elastic [observability-apm-agents-opentelemetry-opentelemetry-native-support-proxy-requests-to-elastic] diff --git a/raw-migrated-files/docs-content/serverless/observability-apm-agents-opentelemetry.md b/raw-migrated-files/docs-content/serverless/observability-apm-agents-opentelemetry.md index 48ffd54308..e3ee3ae19c 100644 --- a/raw-migrated-files/docs-content/serverless/observability-apm-agents-opentelemetry.md +++ b/raw-migrated-files/docs-content/serverless/observability-apm-agents-opentelemetry.md @@ -15,10 +15,10 @@ For a complete overview of using OpenTelemetry with Elastic, explore [Elastic Di Elastic integrates with OpenTelemetry, allowing you to reuse your existing instrumentation to easily send observability data to the Elastic Stack. There are several ways to integrate OpenTelemetry with the Elastic Stack: -* [Elastic Distributions of OpenTelemetry language SDKs](../../../solutions/observability/apps/use-opentelemetry-with-apm.md#observability-apm-agents-opentelemetry-elastic-distributions-of-opentelemetry-language-sdks) -* [Upstream OpenTelemetry API/SDK + Elastic APM agent](../../../solutions/observability/apps/use-opentelemetry-with-apm.md#observability-apm-agents-opentelemetry-upstream-opentelemetry-apisdk-elastic-apm-agent) -* [Upstream OpenTelemetry Collector and language SDKs](../../../solutions/observability/apps/use-opentelemetry-with-apm.md#observability-apm-agents-opentelemetry-upstream-opentelemetry-collector-and-language-sdks) -* [AWS Lambda collector exporter](../../../solutions/observability/apps/use-opentelemetry-with-apm.md#observability-apm-agents-opentelemetry-aws-lambda-collector-exporter) +* [Elastic Distributions of OpenTelemetry language SDKs](../../../solutions/observability/apps/use-opentelemetry-with-apm.md) +* [Upstream OpenTelemetry API/SDK + Elastic APM agent](../../../solutions/observability/apps/use-opentelemetry-with-apm.md) +* [Upstream OpenTelemetry Collector and language SDKs](../../../solutions/observability/apps/use-opentelemetry-with-apm.md) +* [AWS Lambda collector exporter](../../../solutions/observability/apps/use-opentelemetry-with-apm.md) ## Elastic Distributions of OpenTelemetry language SDKs [observability-apm-agents-opentelemetry-elastic-distributions-of-opentelemetry-language-sdks] diff --git a/raw-migrated-files/docs-content/serverless/observability-apm-create-custom-links.md b/raw-migrated-files/docs-content/serverless/observability-apm-create-custom-links.md index 139fe9101c..c3f3474fa8 100644 --- a/raw-migrated-files/docs-content/serverless/observability-apm-create-custom-links.md +++ b/raw-migrated-files/docs-content/serverless/observability-apm-create-custom-links.md @@ -10,7 +10,7 @@ The **Editor** role or higher is required to create and manage custom links. To Elastic’s custom link feature allows you to easily create up to 500 dynamic links based on your specific APM data. Custom links can be filtered to only appear for relevant services, environments, transaction types, or transaction names. -Ready to dive in? Jump straight to the [examples](../../../solutions/observability/apps/create-custom-links.md#observability-apm-create-custom-links-custom-link-examples). +Ready to dive in? Jump straight to the [examples](../../../solutions/observability/apps/create-custom-links.md). ## Create a link [observability-apm-create-custom-links-create-a-link] diff --git a/raw-migrated-files/docs-content/serverless/observability-apm-dependencies.md b/raw-migrated-files/docs-content/serverless/observability-apm-dependencies.md index 81edee7164..c84c4a7162 100644 --- a/raw-migrated-files/docs-content/serverless/observability-apm-dependencies.md +++ b/raw-migrated-files/docs-content/serverless/observability-apm-dependencies.md @@ -1,6 +1,6 @@ # Dependencies [observability-apm-dependencies] -APM agents collect details about external calls made from instrumented services. Sometimes, these external calls resolve into a downstream service that’s instrumented — in these cases, you can utilize [distributed tracing](../../../solutions/observability/apps/trace-sample-timeline.md#observability-apm-trace-sample-timeline-distributed-tracing) to drill down into problematic downstream services. Other times, though, it’s not possible to instrument a downstream dependency — like with a database or third-party service. **Dependencies** gives you a window into these uninstrumented, downstream dependencies. +APM agents collect details about external calls made from instrumented services. Sometimes, these external calls resolve into a downstream service that’s instrumented — in these cases, you can utilize [distributed tracing](../../../solutions/observability/apps/trace-sample-timeline.md) to drill down into problematic downstream services. Other times, though, it’s not possible to instrument a downstream dependency — like with a database or third-party service. **Dependencies** gives you a window into these uninstrumented, downstream dependencies. :::{image} ../../../images/serverless-dependencies.png :alt: Dependencies view in the Applications UI diff --git a/raw-migrated-files/docs-content/serverless/observability-apm-observe-lambda-functions.md b/raw-migrated-files/docs-content/serverless/observability-apm-observe-lambda-functions.md index c89d59de60..cd1adda7a7 100644 --- a/raw-migrated-files/docs-content/serverless/observability-apm-observe-lambda-functions.md +++ b/raw-migrated-files/docs-content/serverless/observability-apm-observe-lambda-functions.md @@ -26,7 +26,7 @@ Cold start is also displayed in the trace waterfall, where you can drill-down in ### Latency distribution correlation [observability-apm-observe-lambda-functions-latency-distribution-correlation] -The [latency correlations](../../../solutions/observability/apps/find-transaction-latency-failure-correlations.md#observability-apm-find-transaction-latency-and-failure-correlations-find-high-transaction-latency-correlations) feature can be used to visualize the impact of Lambda cold starts on latency—just select the `faas.coldstart` field. +The [latency correlations](../../../solutions/observability/apps/find-transaction-latency-failure-correlations.md) feature can be used to visualize the impact of Lambda cold starts on latency—just select the `faas.coldstart` field. ## AWS Lambda function grouping [observability-apm-observe-lambda-functions-aws-lambda-function-grouping] diff --git a/raw-migrated-files/docs-content/serverless/observability-apm-service-overview.md b/raw-migrated-files/docs-content/serverless/observability-apm-service-overview.md index 0f1aab86c5..606aa47788 100644 --- a/raw-migrated-files/docs-content/serverless/observability-apm-service-overview.md +++ b/raw-migrated-files/docs-content/serverless/observability-apm-service-overview.md @@ -82,7 +82,7 @@ The **Dependencies** table displays a list of downstream services or external co The cold start rate chart is specific to serverless services, and displays the percentage of requests that trigger a cold start of a serverless function. A cold start occurs when a serverless function has not been used for a certain period of time. Analyzing the cold start rate can be useful for deciding how much memory to allocate to a function, or when to remove a large dependency. -The cold start rate chart is currently supported for [AWS Lambda](../../../solutions/observability/apps/observe-lambda-functions.md#observability-apm-observe-lambda-functions-cold-starts) functions and Azure functions. +The cold start rate chart is currently supported for [AWS Lambda](../../../solutions/observability/apps/observe-lambda-functions.md#apm-lambda-cold-start-info) functions and Azure functions. ## Instances [observability-apm-service-overview-instances] diff --git a/raw-migrated-files/docs-content/serverless/observability-apm-traces.md b/raw-migrated-files/docs-content/serverless/observability-apm-traces.md index 6c86c51b59..d62889a4f1 100644 --- a/raw-migrated-files/docs-content/serverless/observability-apm-traces.md +++ b/raw-migrated-files/docs-content/serverless/observability-apm-traces.md @@ -6,7 +6,7 @@ Traces link together related transactions to show an end-to-end performance of h :::: -**Traces** displays your application’s entry (root) transactions. Transactions with the same name are grouped together and only shown once in this table. If you’re using [distributed tracing](../../../solutions/observability/apps/trace-sample-timeline.md#observability-apm-trace-sample-timeline-distributed-tracing), this view is key to finding the critical paths within your application. +**Traces** displays your application’s entry (root) transactions. Transactions with the same name are grouped together and only shown once in this table. If you’re using [distributed tracing](../../../solutions/observability/apps/trace-sample-timeline.md), this view is key to finding the critical paths within your application. By default, transactions are sorted by *Impact*. Impact helps show the most used and slowest endpoints in your service — in other words, it’s the collective amount of pain a specific endpoint is causing your users. If there’s a particular endpoint you’re worried about, select it to view its [transaction details](../../../solutions/observability/apps/transactions-2.md#transaction-details). diff --git a/raw-migrated-files/docs-content/serverless/observability-apm-transaction-sampling.md b/raw-migrated-files/docs-content/serverless/observability-apm-transaction-sampling.md index 94986c8cdd..bda7eeb3ff 100644 --- a/raw-migrated-files/docs-content/serverless/observability-apm-transaction-sampling.md +++ b/raw-migrated-files/docs-content/serverless/observability-apm-transaction-sampling.md @@ -39,7 +39,7 @@ In the example in *Figure 2*, `Service A` initiates four transactions and has a In addition to setting the sample rate, you can also specify which *trace continuation strategy* to use. There are three trace continuation strategies: `continue`, `restart`, and `restart_external`. -The **`continue`** trace continuation strategy is the default and will behave similar to the examples in the [Distributed tracing section](../../../solutions/observability/apps/transaction-sampling.md#observability-apm-transaction-sampling-distributed-tracing). +The **`continue`** trace continuation strategy is the default and will behave similar to the examples in the [Distributed tracing section](../../../solutions/observability/apps/transaction-sampling.md). Use the **`restart_external`** trace continuation strategy on an Elastic-monitored service to start a new trace if the previous service did not have a `traceparent` header with `es` vendor data. This can be helpful if a transaction includes an Elastic-monitored service that is receiving requests from an unmonitored service. diff --git a/raw-migrated-files/docs-content/serverless/observability-apm-transactions.md b/raw-migrated-files/docs-content/serverless/observability-apm-transactions.md index 11a0db27df..eead30d403 100644 --- a/raw-migrated-files/docs-content/serverless/observability-apm-transactions.md +++ b/raw-migrated-files/docs-content/serverless/observability-apm-transactions.md @@ -37,7 +37,7 @@ The **Latency**, **Throughput***, ***Failed transaction rate***, ***Time spent b **Cold start rate** -: Only applicable to serverless transactions, this chart displays the percentage of requests that trigger a cold start of a serverless function. See [Cold starts](../../../solutions/observability/apps/observe-lambda-functions.md#observability-apm-observe-lambda-functions-cold-starts) for more information. +: Only applicable to serverless transactions, this chart displays the percentage of requests that trigger a cold start of a serverless function. See [Cold starts](../../../solutions/observability/apps/observe-lambda-functions.md#apm-lambda-cold-start-info) for more information. ## Transactions table [observability-apm-transactions-transactions-table] diff --git a/raw-migrated-files/docs-content/serverless/observability-apm-troubleshooting.md b/raw-migrated-files/docs-content/serverless/observability-apm-troubleshooting.md index 9e5791abc1..da935abca1 100644 --- a/raw-migrated-files/docs-content/serverless/observability-apm-troubleshooting.md +++ b/raw-migrated-files/docs-content/serverless/observability-apm-troubleshooting.md @@ -25,7 +25,7 @@ The most likely cause for this error is using an incompatible version of an {{ap ### HTTP 400: Event too large [event-too-large] -APM agents communicate with the Managed intake service by sending events in an HTTP request. Each event is sent as its own line in the HTTP request body. If events are too large, you can reduce the size of the events that your APM agents send by: [enabling span compression](../../../solutions/observability/apps/spans.md) or [reducing collected stack trace information](../../../solutions/observability/apps/reduce-storage.md#observability-apm-reduce-stacktrace). +APM agents communicate with the Managed intake service by sending events in an HTTP request. Each event is sent as its own line in the HTTP request body. If events are too large, you can reduce the size of the events that your APM agents send by: [enabling span compression](../../../solutions/observability/apps/spans.md) or [reducing collected stack trace information](../../../solutions/observability/apps/reduce-storage.md). ### HTTP 401: Invalid token [unauthorized] diff --git a/raw-migrated-files/docs-content/serverless/observability-monitor-synthetics.md b/raw-migrated-files/docs-content/serverless/observability-monitor-synthetics.md index cfa9c7b377..0e83f16d92 100644 --- a/raw-migrated-files/docs-content/serverless/observability-monitor-synthetics.md +++ b/raw-migrated-files/docs-content/serverless/observability-monitor-synthetics.md @@ -8,8 +8,8 @@ The Synthetics UI is for viewing result data from monitors created and managed d Synthetics periodically checks the status of your services and applications. Monitor the availability of network endpoints and services using the following types of monitors: -* [Lightweight HTTP/S, TCP, and ICMP monitors](../../../solutions/observability/apps/synthetic-monitoring.md#observability-monitor-synthetics-lightweight-https-tcp-and-icmp-monitors) -* [Browser monitors](../../../solutions/observability/apps/synthetic-monitoring.md#observability-monitor-synthetics-browser-monitors) +* [Lightweight HTTP/S, TCP, and ICMP monitors](../../../solutions/observability/apps/synthetic-monitoring.md) +* [Browser monitors](../../../solutions/observability/apps/synthetic-monitoring.md) :::{image} ../../../images/serverless-synthetics-monitor-page.png :alt: Synthetics UI diff --git a/raw-migrated-files/docs-content/serverless/observability-synthetics-command-reference.md b/raw-migrated-files/docs-content/serverless/observability-synthetics-command-reference.md index a9a523e699..99a2bb4607 100644 --- a/raw-migrated-files/docs-content/serverless/observability-synthetics-command-reference.md +++ b/raw-migrated-files/docs-content/serverless/observability-synthetics-command-reference.md @@ -6,7 +6,7 @@ navigation_title: "Use the CLI" -## `@elastic/synthetics` [elastic-synthetics-command] +## `@elastic/synthetics` [elastic-synthetics-command] Elastic uses the [@elastic/synthetics](https://www.npmjs.com/package/@elastic/synthetics) library to run synthetic browser tests and report the test results. The library also provides a CLI to help you scaffold, develop/run tests locally, and push tests to Elastic. @@ -57,7 +57,7 @@ You will not need to use most command line flags. However, there are some you ma Throttling can also be disabled in the configuration file using [`monitor.throttling`](../../../solutions/observability/apps/configure-synthetics-projects.md#synthetics-configuration-monitor). The value defined via the CLI will take precedence. - ::::{note} + ::::{note} Network throttling for browser based monitors is disabled. See this [documention](https://github.com/elastic/synthetics/blob/main/docs/throttling.md) for more details. :::: @@ -68,7 +68,7 @@ You will not need to use most command line flags. However, there are some you ma This is the same as setting [Playwright’s `headless` option](https://playwright.dev/docs/api/class-testoptions#test-options-headless) to `false` by running `--playwright-options '{"headless": false}'`. - ::::{note} + ::::{note} Headful mode should only be used locally to see the browser and interact with DOM elements directly for testing purposes. Do not attempt to run in headful mode when running through Elastic’s global managed testing infrastructure or {{private-location}}s as this is not supported. :::: @@ -77,20 +77,20 @@ You will not need to use most command line flags. However, there are some you ma `-h, --help` : Shows help for the `npx @elastic/synthetics` command. -::::{note} +::::{note} The `--pattern`, `--tags`, and `--match` flags for filtering are only supported when you run synthetic tests locally or push them to Elastic. Filtering is *not* supported in any other subcommands like `init` and `locations`. :::: -::::{note} +::::{note} For debugging synthetic tests locally, you can set an environment variable, `DEBUG=synthetics npx @elastic/synthetics`, to capture Synthetics agent logs. :::: -## `@elastic/synthetics init` [elastic-synthetics-init-command] +## `@elastic/synthetics init` [elastic-synthetics-init-command] Scaffold a new Synthetics project using Elastic Synthetics. @@ -100,10 +100,10 @@ This will create a template Node.js project that includes the synthetics agent, npx @elastic/synthetics init ``` -Read more about what’s included in a template Synthetics project in [Create a Synthetics project](../../../solutions/observability/apps/create-monitors-with-project-monitors.md#observability-synthetics-get-started-project-create-a-synthetics-project). +Read more about what’s included in a template Synthetics project in [Create a Synthetics project](../../../solutions/observability/apps/create-monitors-with-project-monitors.md). -## `@elastic/synthetics push` [elastic-synthetics-push-command] +## `@elastic/synthetics push` [elastic-synthetics-push-command] Create monitors in by using your local journeys. By default, running `push` command will use the `project` settings field from the `synthetics.config.ts` file, which is set up using the `init` command. However, you can override these settings using the CLI flags. @@ -111,7 +111,7 @@ Create monitors in by using your local journeys. By default, running `push` comm SYNTHETICS_API_KEY= npx @elastic/synthetics push --url --id ``` -::::{note} +::::{note} The `push` command includes interactive prompts to prevent you from accidentally deleting or duplicating monitors. You will see a prompt when: * You `push` a project that used to contain one or more monitors but either no longer contains previously running monitors or has any monitors. Select `yes` to delete the monitors associated with the project ID being pushed. @@ -120,7 +120,7 @@ The `push` command includes interactive prompts to prevent you from accidentally :::: -::::{note} +::::{note} If the journey contains external NPM packages other than the `@elastic/synthetics`, those packages will be bundled along with the journey code when the `push` command is invoked. However there are some limitations when using external packages: * Bundled journeys after compression should not be more than 1500 Kilobytes. @@ -188,7 +188,7 @@ If the journey contains external NPM packages other than the `@elastic/synthetic -## Tag monitors [observability-synthetics-command-reference-tag-monitors] +## Tag monitors [observability-synthetics-command-reference-tag-monitors] Synthetics journeys can be tagged with one or more tags. Use tags to filter journeys when running tests locally or pushing them to Elastic. @@ -215,7 +215,7 @@ tags: To apply tags to all browser and lightweight monitors, configure using the `monitor.tags` field in the `synthetics.config.ts` file. -## Filter monitors [observability-synthetics-command-reference-filter-monitors] +## Filter monitors [observability-synthetics-command-reference-filter-monitors] When running the `npx @elastic/synthetics push` command, you can filter the monitors that are pushed to Elastic using the following flags: @@ -236,7 +236,7 @@ npx @elastic/synthetics push --config synthetics.prod.config.ts --tags env:prod ``` -## `@elastic/synthetics locations` [elastic-synthetics-locations-command] +## `@elastic/synthetics locations` [elastic-synthetics-locations-command] List all available locations for running synthetics monitors. @@ -255,7 +255,7 @@ To list both locations on Elastic’s global managed infrastructure and {{privat : API key used for authentication. -## `@elastic/synthetics totp ` [observability-synthetics-command-reference-elasticsynthetics-totp-lesssecretgreater] +## `@elastic/synthetics totp ` [observability-synthetics-command-reference-elasticsynthetics-totp-lesssecretgreater] Generate a Time-based One-Time Password (TOTP) for multifactor authentication(MFA) in Synthetics. diff --git a/raw-migrated-files/docs-content/serverless/observability-synthetics-create-test.md b/raw-migrated-files/docs-content/serverless/observability-synthetics-create-test.md index 6d09ff93ef..ae489848cf 100644 --- a/raw-migrated-files/docs-content/serverless/observability-synthetics-create-test.md +++ b/raw-migrated-files/docs-content/serverless/observability-synthetics-create-test.md @@ -208,7 +208,7 @@ step('make an API request', async () => { The Elastic Synthetics `request` parameter is similar to [other request objects that are exposed by Playwright](https://playwright.dev/docs/api/class-apirequestcontext) with a few key differences: -* The Elastic Synthetics `request` parameter comes built into the library so it doesn’t have to be imported separately, which reduces the amount of code needed and allows you to make API requests in [inline journeys](../../../solutions/observability/apps/create-monitors-in-synthetics-app.md#observability-synthetics-get-started-ui-add-a-browser-monitor). +* The Elastic Synthetics `request` parameter comes built into the library so it doesn’t have to be imported separately, which reduces the amount of code needed and allows you to make API requests in [inline journeys](../../../solutions/observability/apps/create-monitors-in-synthetics-app.md#synthetics-get-started-ui-add-a-browser-monitor). * The top level `request` object exposed by Elastic Synthetics has its own isolated cookie storage unlike Playwright’s `context.request` and `page.request`, which share cookie storage with the corresponding [`BrowserContext`](https://playwright.dev/docs/api/class-browsercontext). * If you want to control the creation of the `request` object, you can do so by passing options via [`--playwright-options`](../../../solutions/observability/apps/use-synthetics-cli.md#elastic-synthetics-command) or in the [`synthetics.config.ts` file](../../../solutions/observability/apps/configure-synthetics-projects.md). diff --git a/raw-migrated-files/docs-content/serverless/observability-synthetics-get-started-project.md b/raw-migrated-files/docs-content/serverless/observability-synthetics-get-started-project.md index b309803204..59fa4e3742 100644 --- a/raw-migrated-files/docs-content/serverless/observability-synthetics-get-started-project.md +++ b/raw-migrated-files/docs-content/serverless/observability-synthetics-get-started-project.md @@ -82,7 +82,7 @@ Then, take a look at key files and directories inside your Synthetics project: * `synthetics.config.ts` contains settings for your Synthetics project. When you create a new Synthetics project, it will contain some basic configuration options that you can customize later. ::::{note} - The `synthetics.config.ts` in the sample Synthetics project uses a location on Elastic’s global managed testing infrastructure. Administrators can restrict access to Elastic’s global managed testing infrastructure. When you attempt to [`push` the sample monitors](../../../solutions/observability/apps/create-monitors-with-project-monitors.md#observability-synthetics-get-started-project-test-and-connect-to-your-observability-project), if you see an error stating that you don’t have permission to use Elastic managed global locations, refer to the [troubleshooting guide](../../../troubleshoot/observability/troubleshooting-synthetics.md#synthetics-troubleshooting-no-locations) for guidance. + The `synthetics.config.ts` in the sample Synthetics project uses a location on Elastic’s global managed testing infrastructure. Administrators can restrict access to Elastic’s global managed testing infrastructure. When you attempt to [`push` the sample monitors](../../../solutions/observability/apps/create-monitors-with-project-monitors.md), if you see an error stating that you don’t have permission to use Elastic managed global locations, refer to the [troubleshooting guide](../../../troubleshoot/observability/troubleshooting-synthetics.md#synthetics-troubleshooting-no-locations) for guidance. :::: diff --git a/raw-migrated-files/observability-docs/observability/apm-anomaly-rule.md b/raw-migrated-files/observability-docs/observability/apm-anomaly-rule.md index 6655c059c4..055a11be3a 100644 --- a/raw-migrated-files/observability-docs/observability/apm-anomaly-rule.md +++ b/raw-migrated-files/observability-docs/observability/apm-anomaly-rule.md @@ -6,7 +6,7 @@ navigation_title: "APM Anomaly" ::::{important} -To use the APM Anomaly rule, you have to enable [machine learning](../../../solutions/observability/apps/integrate-with-machine-learning.md#create-ml-integration), which requires an [appropriate license](https://www.elastic.co/subscriptions). +To use the APM Anomaly rule, you have to enable [machine learning](../../../solutions/observability/apps/integrate-with-machine-learning.md#observability-apm-integrate-with-machine-learning-enable-anomaly-detection), which requires an [appropriate license](https://www.elastic.co/subscriptions). :::: diff --git a/raw-migrated-files/observability-docs/observability/apm-lambda.md b/raw-migrated-files/observability-docs/observability/apm-lambda.md index 65f1406725..758c987169 100644 --- a/raw-migrated-files/observability-docs/observability/apm-lambda.md +++ b/raw-migrated-files/observability-docs/observability/apm-lambda.md @@ -31,7 +31,7 @@ Cold start is also displayed in the trace waterfall, where you can drill-down in ### Latency distribution correlation [apm-lambda-cold-start-latency] -The [latency correlations](../../../solutions/observability/apps/find-transaction-latency-failure-correlations.md#correlations-latency) feature can be used to visualize the impact of Lambda cold starts on latency—​just select the `faas.coldstart` field. +The [latency correlations](../../../solutions/observability/apps/find-transaction-latency-failure-correlations.md) feature can be used to visualize the impact of Lambda cold starts on latency—​just select the `faas.coldstart` field. :::{image} ../../../images/observability-lambda-correlations.png :alt: lambda correlations example diff --git a/raw-migrated-files/observability-docs/observability/apm-machine-learning-integration.md b/raw-migrated-files/observability-docs/observability/apm-machine-learning-integration.md index fe1f42f6ed..0d1c3019ff 100644 --- a/raw-migrated-files/observability-docs/observability/apm-machine-learning-integration.md +++ b/raw-migrated-files/observability-docs/observability/apm-machine-learning-integration.md @@ -66,6 +66,6 @@ To make machine learning as easy as possible to set up, the Applications UI will After enabling anomaly detection, service health may display as "Unknown". Here are some reasons why this can occur: -1. No machine learning job exists. See [Enable anomaly detection](../../../solutions/observability/apps/integrate-with-machine-learning.md#create-ml-integration) to enable anomaly detection and create a machine learning job. +1. No machine learning job exists. See [Enable anomaly detection](../../../solutions/observability/apps/integrate-with-machine-learning.md#observability-apm-integrate-with-machine-learning-enable-anomaly-detection) to enable anomaly detection and create a machine learning job. 2. There is no machine learning data for the job. If you just created the machine learning job you’ll need to wait a few minutes for data to be available. Alternatively, if the service or its enviroment are new, you’ll need to wait for more trace data. 3. No "request" or "page-load" transaction type exists for this service; service health is only available for these transaction types. diff --git a/raw-migrated-files/observability-docs/observability/synthetics-command-reference.md b/raw-migrated-files/observability-docs/observability/synthetics-command-reference.md index ee599236d7..1dd4486a8e 100644 --- a/raw-migrated-files/observability-docs/observability/synthetics-command-reference.md +++ b/raw-migrated-files/observability-docs/observability/synthetics-command-reference.md @@ -6,7 +6,7 @@ navigation_title: "Use the CLI" -## `@elastic/synthetics` [elastic-synthetics-command] +## `@elastic/synthetics` [elastic-synthetics-command] The {{synthetics-app}} uses the [@elastic/synthetics](https://www.npmjs.com/package/@elastic/synthetics) Node.js library to run synthetic browser tests and report the test results. The library also provides a CLI to help you scaffold, develop/run tests locally, and push tests to {{kib}}. @@ -44,7 +44,7 @@ You will not need to use most command line flags — they have been implemen `-c, --config ` -: Path to the configuration file. By default, test runner looks for a `synthetics.config.(js|ts)` file in the current directory. Synthetics configuration provides options to configure how your tests are run and pushed to {{kib}}. Allowed options are described in the [configuration file](../../../solutions/observability/apps/configure-synthetics-projects.md#synthetics-config-file). +: Path to the configuration file. By default, test runner looks for a `synthetics.config.(js|ts)` file in the current directory. Synthetics configuration provides options to configure how your tests are run and pushed to {{kib}}. Allowed options are described in the [configuration file](../../../solutions/observability/apps/configure-synthetics-projects.md). `--reporter ` : One of `json`, `junit`, `buildkite-cli`, or `default`. Use the JUnit or Buildkite reporter to provide easily parsed output to CI systems. @@ -58,7 +58,7 @@ You will not need to use most command line flags — they have been implemen Throttling can also be disabled in the configuration file using [`monitor.throttling`](../../../solutions/observability/apps/configure-synthetics-projects.md#synthetics-configuration-monitor). The value defined via the CLI will take precedence. -::::{note} +::::{note} Network throttling for browser based monitors is disabled. See this [documention](https://github.com/elastic/synthetics/blob/main/docs/throttling.md) for more details. :::: @@ -70,7 +70,7 @@ Network throttling for browser based monitors is disabled. See this [documention This is the same as setting [Playwright’s `headless` option](https://playwright.dev/docs/api/class-testoptions#test-options-headless) to `false` by running `--playwright-options '{"headless": false}'`. -::::{note} +::::{note} Headful mode should only be used locally to see the browser and interact with DOM elements directly for testing purposes. Do not attempt to run in headful mode when running through Elastic’s global managed testing infrastructure or {{private-location}}s as this is not supported. :::: @@ -79,20 +79,20 @@ Headful mode should only be used locally to see the browser and interact with DO `-h, --help` : Shows help for the `npx @elastic/synthetics` command. -::::{note} +::::{note} The `--pattern`, `--tags`, and `--match` flags for filtering are only supported when you run synthetic tests locally or push them to Kibana. Filtering is *not* supported in any other subcommands like `init` and `locations`. :::: -::::{note} +::::{note} For debugging synthetic tests locally, you can set an environment variable, `DEBUG=synthetics npx @elastic/synthetics`, to capture Synthetics agent logs. :::: -## `@elastic/synthetics init` [elastic-synthetics-init-command] +## `@elastic/synthetics init` [elastic-synthetics-init-command] Scaffold a new project using Elastic Synthetics. @@ -102,10 +102,10 @@ This will create a template Node.js project that includes the synthetics agent, npx @elastic/synthetics init ``` -Read more about what’s included in a template project in [Create a project](../../../solutions/observability/apps/create-monitors-with-project-monitors.md#synthetics-get-started-project-init). +Read more about what’s included in a template project in [Create a project](../../../solutions/observability/apps/create-monitors-with-project-monitors.md). -## `@elastic/synthetics push` [elastic-synthetics-push-command] +## `@elastic/synthetics push` [elastic-synthetics-push-command] Create monitors in {{kib}} by using your local journeys. By default, running `push` command will use the `project` settings field from the `synthetics.config.ts` file, which is set up using the `init` command. However, you can override these settings using the CLI flags. @@ -113,7 +113,7 @@ Create monitors in {{kib}} by using your local journeys. By default, running `pu SYNTHETICS_API_KEY= npx @elastic/synthetics push --url --id ``` -::::{note} +::::{note} The `push` command includes interactive prompts to prevent you from accidentally deleting or duplicating monitors. You will see a prompt when: * You `push` a project that used to contain one or more monitors but either no longer contains previously running monitors or has any monitors. Select `yes` to delete the monitors associated with the project ID being pushed. @@ -124,7 +124,7 @@ You can set `DEBUG=synthetics` environment variable to capture the deleted monit :::: -::::{note} +::::{note} If the journey contains external NPM packages other than the `@elastic/synthetics`, those packages will be bundled along with the journey code when the `push` command is invoked. However there are some limitations when using external packages: * Bundled journeys after compression should not be more than 1500 Kilobytes. @@ -137,7 +137,7 @@ If the journey contains external NPM packages other than the `@elastic/synthetic `--auth ` : API key used for [{{kib}} authentication](../../../deploy-manage/api-keys/elasticsearch-api-keys.md). You can also set the API key via the `SYNTHETICS_API_KEY` environment variable. - If you are pushing to a [{{private-location}}](../../../solutions/observability/apps/create-monitors-in-synthetics-app.md#private-locations), you must use an API key generated in 8.4 or higher. + If you are pushing to a [{{private-location}}](../../../solutions/observability/apps/create-monitors-in-synthetics-app.md), you must use an API key generated in 8.4 or higher. To create an API key, you must be logged into {{kib}} as a user with the privileges described in [Writer role](../../../solutions/observability/apps/writer-role.md). @@ -200,7 +200,7 @@ If the journey contains external NPM packages other than the `@elastic/synthetic -## Tagging and Filtering monitors [tagging-and-filtering] +## Tagging and Filtering monitors [tagging-and-filtering] Synthetics journeys can be tagged with one or more tags. Use tags to filter journeys when running tests locally or pushing them to {{kib}}. @@ -228,7 +228,7 @@ tags: To apply tags to all browser and lightweight monitors, configure using [`monitor.tags`](../../../solutions/observability/apps/configure-synthetics-projects.md#synthetics-configuration-monitor) field in the `synthetics.config.ts` file. -### Filtering monitors [_filtering_monitors] +### Filtering monitors [_filtering_monitors] When running the `npx @elastic/synthetics push` command, you can filter the monitors that are pushed to {{kib}} using the following flags: @@ -249,7 +249,7 @@ npx @elastic/synthetics push --config synthetics.prod.config.ts --tags env:prod ``` -## `@elastic/synthetics locations` [elastic-synthetics-locations-command] +## `@elastic/synthetics locations` [elastic-synthetics-locations-command] List all available locations for running synthetics monitors. @@ -267,13 +267,13 @@ To list both locations on Elastic’s global managed infrastructure and {{privat `--auth ` : API key used for [{{kib}} authentication](../../../deploy-manage/api-keys/elasticsearch-api-keys.md). -::::{note} +::::{note} If an administrator has disabled Elastic managed locations for the role you are assigned and you do *not* include `--url` and `--auth`, all global locations managed by Elastic will be listed. However, you will not be able to push to these locations with your API key and will see an error: *You don’t have permission to use Elastic managed global locations*. For more details, refer to the [troubleshooting docs](../../../troubleshoot/observability/troubleshooting-synthetics.md#synthetics-troubleshooting-public-locations-disabled). :::: -## `@elastic/synthetics totp ` [elastic-synthetics-totp-command] +## `@elastic/synthetics totp ` [elastic-synthetics-totp-command] Generate a Time-based One-Time Password (TOTP) for multifactor authentication (MFA) in Synthetics. diff --git a/raw-migrated-files/observability-docs/observability/synthetics-create-test.md b/raw-migrated-files/observability-docs/observability/synthetics-create-test.md index 4db1185955..87a727f539 100644 --- a/raw-migrated-files/observability-docs/observability/synthetics-create-test.md +++ b/raw-migrated-files/observability-docs/observability/synthetics-create-test.md @@ -209,7 +209,7 @@ step('make an API request', async () => { The Elastic Synthetics `request` parameter is similar to [other request objects that are exposed by Playwright](https://playwright.dev/docs/api/class-apirequestcontext) with a few key differences: -* The Elastic Synthetics `request` parameter comes built into the library so it doesn’t have to be imported separately, which reduces the amount of code needed and allows you to make API requests in [inline journeys](../../../solutions/observability/apps/create-monitors-in-synthetics-app.md#synthetics-get-started-ui-browser). +* The Elastic Synthetics `request` parameter comes built into the library so it doesn’t have to be imported separately, which reduces the amount of code needed and allows you to make API requests in [inline journeys](../../../solutions/observability/apps/create-monitors-in-synthetics-app.md#synthetics-get-started-ui-add-a-browser-monitor). * The top level `request` object exposed by Elastic Synthetics has its own isolated cookie storage unlike Playwright’s `context.request` and `page.request`, which share cookie storage with the corresponding [`BrowserContext`](https://playwright.dev/docs/api/class-browsercontext). * If you want to control the creation of the `request` object, you can do so by passing options via [`--playwright-options`](../../../solutions/observability/apps/use-synthetics-cli.md#elastic-synthetics-command) or in the [`synthetics.config.ts` file](../../../solutions/observability/apps/configure-synthetics-projects.md). diff --git a/raw-migrated-files/observability-docs/observability/synthetics-get-started-project.md b/raw-migrated-files/observability-docs/observability/synthetics-get-started-project.md index 5689f91c7c..293e5cf0dc 100644 --- a/raw-migrated-files/observability-docs/observability/synthetics-get-started-project.md +++ b/raw-migrated-files/observability-docs/observability/synthetics-get-started-project.md @@ -93,7 +93,7 @@ Then, take a look at key files and directories inside your project: * `synthetics.config.ts` contains settings for your project. When you create a new project, it will contain some basic configuration options that you can customize later. ::::{note} - The `synthetics.config.ts` in the sample project uses a location on Elastic’s global managed testing infrastructure. Administrators can restrict access to Elastic’s global managed testing infrastructure. When you attempt to [`push` the sample monitors](../../../solutions/observability/apps/create-monitors-with-project-monitors.md#synthetics-get-started-project-push), if you see an error stating that you don’t have permission to use Elastic managed global locations, refer to the [troubleshooting guide](../../../troubleshoot/observability/troubleshooting-synthetics.md#synthetics-troubleshooting-no-locations) for guidance. + The `synthetics.config.ts` in the sample project uses a location on Elastic’s global managed testing infrastructure. Administrators can restrict access to Elastic’s global managed testing infrastructure. When you attempt to [`push` the sample monitors](../../../solutions/observability/apps/create-monitors-with-project-monitors.md), if you see an error stating that you don’t have permission to use Elastic managed global locations, refer to the [troubleshooting guide](../../../troubleshoot/observability/troubleshooting-synthetics.md#synthetics-troubleshooting-no-locations) for guidance. :::: diff --git a/solutions/observability/apps/act-on-data.md b/solutions/observability/apps/act-on-data.md index f4e83165d8..8959a7b078 100644 --- a/solutions/observability/apps/act-on-data.md +++ b/solutions/observability/apps/act-on-data.md @@ -2,13 +2,16 @@ mapped_urls: - https://www.elastic.co/guide/en/observability/current/apm-act-on-data.html - https://www.elastic.co/guide/en/serverless/current/observability-apm-act-on-data.html + +navigation_title: "Act on data" --- -# Act on data +# Act on application data -% What needs to be done: Align serverless/stateful -% Use migrated content from existing pages that map to this page: +In addition to exploring visualizations in the Applications UI in {{kib}}, you can make your application data more actionable with: -% - [ ] ./raw-migrated-files/observability-docs/observability/apm-act-on-data.md -% - [ ] ./raw-migrated-files/docs-content/serverless/observability-apm-act-on-data.md \ No newline at end of file +| | | +| --- | --- | +| [Rules and alerts](create-apm-rules-alerts.md) | The Applications UI allows you to define rules to detect complex conditions within your APM data and trigger built-in actions when those conditions are met. | +| [Custom links](create-custom-links.md) | Build URLs that contain relevant metadata from a specific trace. For example, you can create a link that will take you to a page where you can open a new GitHub issue with context already auto-populated in the issue body. These links will be shown in the *Actions* context menu in selected areas of the Applications UI (for example, by transaction details). | \ No newline at end of file diff --git a/solutions/observability/apps/analyze-data-from-synthetic-monitors.md b/solutions/observability/apps/analyze-data-from-synthetic-monitors.md index ce191c7fc6..f4a8be32a8 100644 --- a/solutions/observability/apps/analyze-data-from-synthetic-monitors.md +++ b/solutions/observability/apps/analyze-data-from-synthetic-monitors.md @@ -2,33 +2,252 @@ mapped_urls: - https://www.elastic.co/guide/en/observability/current/synthetics-analyze.html - https://www.elastic.co/guide/en/serverless/current/observability-synthetics-analyze.html + +navigation_title: "Analyze monitor data" --- # Analyze data from synthetic monitors -% What needs to be done: Align serverless/stateful +The Synthetics UI gives you a high-level overview of your service’s availability and allows you to dig into details to diagnose what caused downtime. + +## Overview [synthetics-analyze-overview] + +The Synthetics **Overview** tab provides you with a high-level view of all the services you are monitoring to help you quickly diagnose outages and other connectivity issues within your network. + +To access this page, find `Synthetics` in the [global search field](../../../get-started/the-stack.md#kibana-navigation-search) and make sure you’re on the **Overview** tab. + +This overview includes a snapshot of the current status of all monitors, the number of errors that occurred over the last 6 hours, and the number of alerts over the last 12 hours. All monitors created using projects or using the UI will be listed below with information about the location, current status, and duration average. + +::::{note} +When you use a single monitor configuration to create monitors in multiple locations, each location is listed as a separate monitor as they run as individual monitors and the status and duration average can vary by location. + +:::: + + +:::{image} ../../../images/observability-synthetics-monitor-page.png +:alt: Synthetics UI +:class: screenshot +::: + +To get started with your analysis in the Overview tab, you can search for monitors or use the filter options including current status (up, down, or disabled), monitor type (for example, journey or HTTP), location, and more. + +Then click an individual monitor to see some details in a flyout. From there, you can click **Go to monitor** to go to an individual monitor’s page to see more details (as described below). + + +## All monitor types [synthetics-analyze-individual-monitors] + +When you go to an individual monitor’s page, you’ll see much more detail about the monitor’s performance over time. The details vary by monitor type, but for every monitor at the top of the page you’ll see: + +* The monitor’s **name** with a down arrow icon that you can use to quickly move between monitors. +* The **location** of the monitor. If the same monitor configuration was used to create monitors in multiple locations, you’ll also see a down arrow icon that you can use to quickly move between locations that use the same configuration. +* The latest **status** and when the monitor was **last run**. +* The **![Beaker icon](../../../images/observability-beaker.svg "") Run test manually** button that allows you to run the test on demand before the next scheduled run. + + ::::{note} + This is only available for monitors running on Elastic’s global managed testing infrastructure. It is not available for monitors running on {{private-location}}s. + + :::: + +* The **![Pencil icon](../../../images/observability-pencil.svg "") Edit monitor** button that allows you to edit the monitor’s configuration. + +:::{image} ../../../images/observability-synthetics-analyze-individual-monitor-header.png +:alt: Header at the top of the individual monitor page for all monitor types in the {synthetics-app} +:class: screenshot +::: + +Each individual monitor’s page has three tabs: Overview, History, and Errors. + + +### Overview [synthetics-analyze-individual-monitors-overview] + +The **Overview** tab has information about the monitor availability, duration, and any errors that have occurred since the monitor was created. The *Duration trends* chart displays the timing for each check that was performed in the last 30 days. This visualization helps you to gain insights into how quickly requests resolve by the targeted endpoint and gives you a sense of how frequently a host or endpoint was down. + +:::{image} ../../../images/observability-synthetics-analyze-individual-monitor-details.png +:alt: Details in the Overview tab on the individual monitor page for all monitor types in the {synthetics-app} +:class: screenshot +::: + + +### History [synthetics-analyze-individual-monitors-history] + +The **History** tab has information on every time the monitor has run. It includes some high-level stats and a complete list of all test runs. Use the calendar icon (![Calendar icon](../../../images/observability-calendar.svg "")) and search bar to filter for runs that occurred in a specific time period. + +For browser monitors, you can click on any run in the **Test runs** list to see the details for that run. Read more about what information is included the in [Details for one run](../../../solutions/observability/apps/analyze-data-from-synthetic-monitors.md#synthetics-analyze-one-run) section below. + +:::{image} ../../../images/observability-synthetics-analyze-individual-monitor-history.png +:alt: The History tab on the individual monitor page for all monitor types in the {synthetics-app} +:class: screenshot +::: + +If the monitor is configured to [retest on failure](../../../solutions/observability/apps/configure-synthetics-projects.md#synthetics-configuration-monitor), you’ll see retests listed in the **Test runs** table. Runs that are retests include a rerun icon (![Refresh icon](../../../images/observability-refresh.svg "")) next to the result badge. + +:::{image} ../../../images/observability-synthetics-retest.png +:alt: A failed run and a retest in the table of test runs in the {synthetics-app} +:class: screenshot +::: + + +### Errors [synthetics-analyze-individual-monitors-errors] + +The **Errors** tab has information on failed runs. If the monitor is configured to [retest on failure](../../../solutions/observability/apps/configure-synthetics-projects.md#synthetics-configuration-monitor), failed runs will only result in an error if both the initial run and the rerun fail. This can reduce noise related to transient problems. + +The Errors tab includes a high-level overview of all alerts and a complete list of all failures. Use the calendar icon (![Calendar icon](../../../images/observability-calendar.svg "")) and search bar to filter for runs that occurred in a specific time period. + +For browser monitors, you can click on any run in the **Error** list to open an **Error details** page that includes most of the same information that is included the in [Details for one run](../../../solutions/observability/apps/analyze-data-from-synthetic-monitors.md#synthetics-analyze-one-run) section below. + +:::{image} ../../../images/observability-synthetics-analyze-individual-monitor-errors.png +:alt: The Errors tab on the individual monitor page for all monitor types in the {synthetics-app} +:class: screenshot +::: + + +## Browser monitors [synthetics-analyze-journeys] + +For browser monitors, you can look at results at various levels of granularity: + +* See an overview of journey runs over time. +* Drill down into the details of a single run. +* Drill down further into the details of a single *step* within a journey. + + +### Journey runs over time [journey_runs_over_time] + +The journey page on the Overview tab includes: + +* An overview of the **last test run** including high-level information for each step. +* **Alerts** to date including both active and recovered alerts. +* **Duration by step** over the last 24 hours. +* A list of the **last 10 test runs** that link to the [details for each run](../../../solutions/observability/apps/analyze-data-from-synthetic-monitors.md#synthetics-analyze-one-run). + +:::{image} ../../../images/observability-synthetics-analyze-journeys-over-time.png +:alt: Individual journey page for browser monitors in the {synthetics-app} +:class: screenshot +::: + +From here, you can either drill down into: + +* The latest run of the full journey by clicking **![Inspect icon](../../../images/observability-inspect.svg "") View test run** or a past run in the list of **Last 10 test runs**. This will take you to the view described below in [Details for one run](../../../solutions/observability/apps/analyze-data-from-synthetic-monitors.md#synthetics-analyze-one-run). +* An individual step in this run by clicking the performance breakdown icon (![Performance breakdown icon](../../../images/observability-apmTrace.svg "")) next to one of the steps. This will take you to the view described below in [Details for one step](../../../solutions/observability/apps/analyze-data-from-synthetic-monitors.md#synthetics-analyze-one-step). + + +### Details for one run [synthetics-analyze-one-run] + +The page detailing one run for a journey includes more information on each step in the current run and opportunities to compare each step to the same step in previous runs. + +At the top of the page, see the *Code executed* and any *Console* output for each step. If the step failed, this will also include a *Stacktrace* tab that you can use to diagnose the cause of errors. + +Navigate through each step using **![Previous icon](../../../images/observability-arrowLeft.svg "") Previous** and **Next ![Next icon](../../../images/observability-arrowRight.svg "")**. + +:::{image} ../../../images/observability-synthetics-analyze-one-run-code-executed.png +:alt: Step carousel on a page detailing one run of a browser monitor in the {synthetics-app} +:class: screenshot +::: + +Scroll down to dig into the steps in this journey run. Click the ![Arrow right icon](../../../images/observability-arrowRight.svg "") icon next to the step number to show details. The details include metrics for the step in the current run and the step in the last successful run. Read more about step-level metrics below in [Timing](../../../solutions/observability/apps/analyze-data-from-synthetic-monitors.md#synthetics-analyze-one-step-timing) and [Metrics](../../../solutions/observability/apps/analyze-data-from-synthetic-monitors.md#synthetics-analyze-one-step-metrics). + +This is particularly useful to compare the metrics for a failed step to the last time it completed successfully when trying to diagnose the reason it failed. + +![Step list on a page detailing one run of a browser monitor in the Synthetics UI](../../../images/observability-synthetics-analyze-one-run-compare-steps.png "") + +Drill down to see even more details for an individual step by clicking the performance breakdown icon (![Performance breakdown icon](../../../images/observability-apmTrace.svg "")) next to one of the steps. This will take you to the view described below in [Details for one step](../../../solutions/observability/apps/analyze-data-from-synthetic-monitors.md#synthetics-analyze-one-step). + + +### Details for one step [synthetics-analyze-one-step] + +After clicking the performance breakdown icon (![Performance breakdown icon](../../../images/observability-apmTrace.svg "")) you’ll see more detail for an individual step. + + +#### Screenshot [synthetics-analyze-one-step-screenshot] + +By default the synthetics library will capture a screenshot for each step regardless of whether the step completed or failed. + +::::{note} +Customize screenshot behavior for all monitors in the [configuration file](../../../solutions/observability/apps/configure-synthetics-projects.md), for one monitor using [`monitor.use`](../../../solutions/observability/apps/configure-individual-browser-monitors.md), or for a run using the [CLI](../../../solutions/observability/apps/use-synthetics-cli.md#elastic-synthetics-command). + +:::: + + +Screenshots can be particularly helpful to identify what went wrong when a step fails because of a change to the UI. You can compare the failed step to the last time the step successfully completed. + +:::{image} ../../../images/observability-synthetics-analyze-one-step-screenshot.png +:alt: Screenshot for one step in a browser monitor in the {synthetics-app} +:class: screenshot +::: + + +#### Timing [synthetics-analyze-one-step-timing] + +The **Timing** visualization shows a breakdown of the time spent in each part of the resource loading process for the step including: + +* **Blocked**: The request was initiated but is blocked or queued. +* **DNS**: The DNS lookup to convert the hostname to an IP Address. +* **Connect**: The time it took the request to connect to the server. Lengthy connections could indicate network issues, connection errors, or an overloaded server. +* **TLS**: If your page is loading resources securely over TLS, this is the time it took to set up that connection. +* **Wait**: The time it took for the response generated by the server to be received by the browser. A lengthy Waiting (TTFB) time could indicate server-side issues. +* **Receive**: The time it took to receive the response from the server, which can be impacted by the size of the response. +* **Send**: The time spent sending the request data to the server. + +Next to each network timing metric, there’s an icon that indicates whether the value is higher (![Value is higher icon](../../../images/observability-sortUp.svg "")), lower (![Value is lower icon](../../../images/observability-sortDown.svg "")), or the same (![Value is the same](../../../images/observability-minus.svg "")) compared to the median of all runs in the last 24 hours. Hover over the icon to see more details in a tooltip. + +This gives you an overview of how much time is spent (and how that time is spent) loading resources. This high-level information may not help you diagnose a problem on its own, but it could act as a signal to look at more granular information in the [Network requests](../../../solutions/observability/apps/analyze-data-from-synthetic-monitors.md#synthetics-analyze-one-step-network) section. + +:::{image} ../../../images/observability-synthetics-analyze-one-step-timing.png +:alt: Network timing visualization for one step in a browser monitor in the {synthetics-app} +:class: screenshot +::: + + +#### Metrics [synthetics-analyze-one-step-metrics] + +The **Metrics** visualization gives you insight into the performance of the web page visited in the step and what a user would experience when going through the current step. Metrics include: + +* **First contentful paint (FCP)** focuses on the initial rendering and measures the time from when the page starts loading to when any part of the page’s content is displayed on the screen. +* **Largest contentful paint (LCP)** measures loading performance. To provide a good user experience, LCP should occur within 2.5 seconds of when the page first starts loading. +* **Cumulative layout shift (CLS)** measures visual stability. To provide a good user experience, pages should maintain a CLS of less than 0.1. +* **`DOMContentLoaded` event (DCL)** is triggered when the browser completes parsing the document. Helpful when there are multiple listeners, or logic is executed: `domContentLoadedEventEnd - domContentLoadedEventStart`. +* **Transfer size** represents the size of the fetched resource. The size includes the response header fields plus the response payload body. + +::::{note} +Largest contentful paint and Cumulative layout shift are part of Google’s [Core Web Vitals](https://web.dev/vitals/), an initiative that introduces a set of metrics that help categorize good and bad sites by quantifying the real-world user experience. + +:::: + + +Next to each metric, there’s an icon that indicates whether the value is higher (![Value is higher icon](../../../images/observability-sortUp.svg "")), lower (![Value is lower icon](../../../images/observability-sortDown.svg "")), or the same (![Value is the same](../../../images/observability-minus.svg "")) compared to all runs over the last 24 hours. Hover over the icon to see more details in a tooltip. + +:::{image} ../../../images/observability-synthetics-analyze-one-step-metrics.png +:alt: Metrics visualization for one step in a browser monitor in the {synthetics-app} +:class: screenshot +::: + + +#### Object weight and count [synthetics-analyze-one-step-object] -% Use migrated content from existing pages that map to this page: +The **Object weight** visualization shows the cumulative size of downloaded resources by type, and **Object count** shows the number of individual resources by type. -% - [ ] ./raw-migrated-files/observability-docs/observability/synthetics-analyze.md -% - [ ] ./raw-migrated-files/docs-content/serverless/observability-synthetics-analyze.md +This provides a different kind of analysis. For example, you might have a large number of JavaScript files, each of which will need a separate download, but they may be collectively small. This could help you identify an opportunity to improve efficiency by combining multiple files into one. -% Internal links rely on the following IDs being on this page (e.g. as a heading ID, paragraph ID, etc): +:::{image} ../../../images/observability-synthetics-analyze-one-step-object.png +:alt: Object visualization for one step in a browser monitor in the {synthetics-app} +:class: screenshot +::: -$$$synthetics-analyze-individual-monitors-overview$$$ -$$$synthetics-analyze-individual-monitors-history$$$ +#### Network requests [synthetics-analyze-one-step-network] -$$$synthetics-analyze-overview$$$ +The **Network requests** visualization is a waterfall chart that shows every request the page made when a user executed it. Each line in the chart represents an HTTP network request and helps you quickly identify what resources are taking the longest to load and in what order they are loading. -$$$synthetics-analyze-journeys$$$ +The colored bars within each line indicate the time spent per resource. Each color represents a different part of that resource’s loading process (as defined in the [Timing](../../../solutions/observability/apps/analyze-data-from-synthetic-monitors.md#synthetics-analyze-one-step-timing) section above) and includes the time spent downloading content for specific Multipurpose Internet Mail Extensions (MIME) types: HTML, JS, CSS, Media, Font, XHR, and Other. -$$$synthetics-analyze-one-run$$$ +Understanding each phase of a request can help you improve your site’s speed by reducing the time spent in each phase. -$$$synthetics-analyze-one-step-metrics$$$ +:::{image} ../../../images/observability-synthetics-analyze-one-step-network.png +:alt: Network requests waterfall visualization for one step in a browser monitor in the {synthetics-app} +:class: screenshot +::: -$$$synthetics-analyze-one-step-network$$$ +Without leaving the waterfall chart, you can view data points relating to each resource: resource details, request headers, response headers, and certificate headers. On the waterfall chart, select a resource name, or any part of each row, to display the resource details overlay. -$$$synthetics-analyze-one-step-timing$$$ +For additional analysis, whether to check the content of a CSS file or to view a specific image, click the ![External link icon](../../../images/observability-popout.svg "") icon located beside each resource, to view its content in a new tab. -$$$synthetics-analyze-one-step$$$ \ No newline at end of file +You can also navigate between steps and checks at the top of the page to view the corresponding waterfall charts. \ No newline at end of file diff --git a/solutions/observability/apps/application-performance-monitoring-apm.md b/solutions/observability/apps/application-performance-monitoring-apm.md index 40d704aa52..d15a32ea98 100644 --- a/solutions/observability/apps/application-performance-monitoring-apm.md +++ b/solutions/observability/apps/application-performance-monitoring-apm.md @@ -4,11 +4,20 @@ mapped_urls: - https://www.elastic.co/guide/en/serverless/current/observability-apm.html --- -# Application performance monitoring (APM) +# Application performance monitoring (APM) [apm] -% What needs to be done: Align serverless/stateful +Elastic APM is an application performance monitoring system built on the {{stack}}. It allows you to monitor software services and applications in real time, by collecting detailed performance information on response time for incoming requests, database queries, calls to caches, external HTTP requests, and more. This makes it easy to pinpoint and fix performance problems quickly. -% Use migrated content from existing pages that map to this page: +:::{image} ../../../images/observability-apm-app-landing.png +:alt: Applications UI in {kib} +:class: screenshot +::: -% - [ ] ./raw-migrated-files/observability-docs/observability/apm.md -% - [ ] ./raw-migrated-files/docs-content/serverless/observability-apm.md \ No newline at end of file +Elastic APM also automatically collects unhandled errors and exceptions. Errors are grouped based primarily on the stack trace, so you can identify new errors as they appear and keep an eye on how many times specific errors happen. + +Metrics are another vital source of information when debugging production systems. Elastic APM agents automatically pick up basic host-level metrics and agent-specific metrics, like JVM metrics in the Java Agent, and Go runtime metrics in the Go Agent. + + +## Give Elastic APM a try [give_elastic_apm_a_try] + +Use [Get started with application traces and APM](../../../solutions/observability/apps/fleet-managed-apm-server.md) to quickly spin up an APM deployment. Want to host everything yourself instead? See [Get started](../../../solutions/observability/apps/get-started-with-apm.md). \ No newline at end of file diff --git a/solutions/observability/apps/applications-ui-settings.md b/solutions/observability/apps/applications-ui-settings.md index 8b1337bd64..7c2dec5d1b 100644 --- a/solutions/observability/apps/applications-ui-settings.md +++ b/solutions/observability/apps/applications-ui-settings.md @@ -2,13 +2,71 @@ mapped_urls: - https://www.elastic.co/guide/en/observability/current/apm-settings-in-kibana.html - https://www.elastic.co/guide/en/serverless/current/observability-apm-kibana-settings.html + +navigation_title: "Settings" --- -# Applications UI settings +# Applications UI settings [observability-apm-kibana-settings] + + +::::{admonition} Required role +:class: note + +The **Editor** role or higher is required to modify settings. To learn more, refer to [Assign user roles and privileges](../../../deploy-manage/users-roles/cloud-organization/user-roles.md#general-assign-user-roles). + +:::: + +You can adjust Application settings to fine-tune your experience in the Applications UI. + + +## General settings [observability-apm-kibana-settings-general-settings] + +To change APM settings, select **Settings** from any **Applications** page. The following settings are available. + +`observability:apmAgentExplorerView` +: [beta] Enables the Agent explorer view. + +`observability:apmAWSLambdaPriceFactor` +: Set the price per Gb-second for your AWS Lambda functions. + +`observability:apmAWSLambdaRequestCostPerMillion` +: Set the AWS Lambda cost per million requests. + +`observability:apmEnableContinuousRollups` +: [beta] When continuous rollups are enabled, the UI will select metrics with the appropriate resolution. On larger time ranges, lower resolution metrics will be used, which will improve loading times. + +`observability:apmEnableServiceMetrics` +: [beta] Enables the usage of service transaction metrics, which are low cardinality metrics that can be used by certain views like the service inventory for faster loading times. + +`observability:apmLabsButton` +: Enable or disable the APM Labs button — a quick way to enable and disable technical preview features in APM. + +`observability:apmServiceGroupMaxNumberOfServices` +: Limit the number of services in a given service group. + +`observability:apmDefaultServiceEnvironment` +: Set the default environment for APM. When left empty, data from all environments will be displayed by default. + +`observability:apmEnableProfilingIntegration` +: Enable the Universal Profiling integration in APM. + +`observability:enableComparisonByDefault` +: Enable the comparison feature by default. + +`observability:enableInspectEsQueries` +: When enabled, allows you to inspect Elasticsearch queries in API responses. + +## APM Indices [apm-indices-settings] + +The Applications UI uses data views to query APM indices. In non-serverless versions, change the default APM indices that the Applications UI queries by opening the Applications UI and select **Settings** → **Indices**. Index settings in the Applications UI take precedence over those set in `kibana.yml`. + +APM indices are {{kib}} Spaces-aware; Changes to APM index settings will only apply to the currently enabled space. See [Control access to APM data](../../../solutions/observability/apps/control-access-to-apm-data.md) for more information. + + +## APM Labs [observability-apm-kibana-settings-apm-labs] -% What needs to be done: Align serverless/stateful +**APM Labs** allows you to easily try out new features that are technical preview. -% Use migrated content from existing pages that map to this page: +To enable APM labs, go to **Applications** → **Settings*** → ***General settings*** and toggle ***Enable labs button in APM**. Select **Save changes** and refresh the page. -% - [ ] ./raw-migrated-files/observability-docs/observability/apm-settings-in-kibana.md -% - [ ] ./raw-migrated-files/docs-content/serverless/observability-apm-kibana-settings.md \ No newline at end of file +After enabling **APM Labs** select **Labs** in the toolbar to see the technical preview features available to try out. \ No newline at end of file diff --git a/solutions/observability/apps/collect-application-data.md b/solutions/observability/apps/collect-application-data.md index 27e317c30f..d3ca0ab916 100644 --- a/solutions/observability/apps/collect-application-data.md +++ b/solutions/observability/apps/collect-application-data.md @@ -4,15 +4,63 @@ mapped_urls: - https://www.elastic.co/guide/en/serverless/current/observability-apm-send-data-to-elastic.html --- -# Collect application data -% What needs to be done: Align serverless/stateful +# Collect application data [apm-collect-application-data] -% Use migrated content from existing pages that map to this page: +% is required role serverless only? -% - [ ] ./raw-migrated-files/observability-docs/observability/apm-collect-application-data.md -% - [ ] ./raw-migrated-files/docs-content/serverless/observability-apm-send-data-to-elastic.md +::::{admonition} Required role +:class: note -% Internal links rely on the following IDs being on this page (e.g. as a heading ID, paragraph ID, etc): +The **Admin** role or higher is required to send APM data to Elastic. To learn more, refer to [Assign user roles and privileges](../../../deploy-manage/users-roles/cloud-organization/user-roles.md#general-assign-user-roles). -$$$apm-collect-data-availability$$$ \ No newline at end of file +:::: + +![documentation icon](../../../images/serverless-documentation.svg "") Want to get started quickly? See [Get started with traces and APM](../../../solutions/observability/apps/get-started-with-apm.md). + +## Language-specific options [_language_specific_options] + +Use Elastic APM agents or an OpenTelemetry language SDK to instrument a service in the language its written in: + +* [**Elastic APM agents**](../../../solutions/observability/apps/elastic-apm-agents.md): Elastic APM agents are instrumentation libraries written in the same language as your service. +* [**OpenTelemetry**](../../../solutions/observability/apps/use-opentelemetry-with-apm.md): OpenTelemetry is an open source set of APIs, SDKs, tooling, and integrations that enable the capture and management of telemetry data from your services and applications. + + * This option includes Elastic Distributions of OpenTelemetry, which are customized versions of [OpenTelemetry language SDKs](https://opentelemetry.io/docs/languages/) that are optimized to work with an Elastic backend. + + +**Not sure which method is right for you?** Compare the available options below. + + +### Capabilities [_capabilities] + +| | Elastic APM agent | Elastic Distribution of OpenTelemetry | +| --- | --- | --- | +| **Support level** | Fully supported | Mixed support
*Refer to the* [*availability table*](../../../solutions/observability/apps/collect-application-data.md#apm-collect-data-availability) | +| **Data protocol** | Elastic protocol | [OpenTelemetry protocol (OTLP)](https://opentelemetry.io/docs/specs/otel/protocol/) | +| **Central configuration** | Supported
*Refer to* [*APM agent central configuration*](../../../solutions/observability/apps/apm-agent-central-configuration.md) | Not supported | + +% Stateful only after this comment? + +### Availability [apm-collect-data-availability] + +| | | | +| --- | --- | --- | +| **Language** | **Elastic APM agent** | **Elastic Distributions of OpenTelemetry (EDOT)** | +| **Android** | Android agent | ![Not available](../../../images/observability-cross.svg "") | +| **Go** | Go agent | ![Not available](../../../images/observability-cross.svg "") | +| **iOS** | iOS agent | ![Not available](../../../images/observability-cross.svg "") | +| **Java** | Java agent | EDOT Java | +| **.NET** | .NET agent | [preview] EDOT .NET | +| **Node.js** | Node.js agent | [preview] EDOT Node.js | +| **PHP** | PHP agent | [preview] EDOT PHP | +| **Python** | Python agent | [preview] EDOT Python | +| **Ruby** | Ruby agent | ![Not available](../../../images/observability-cross.svg "") | + + +## Service-specific options [_service_specific_options] + +Elastic also offers several tools to help you collect data from specific services: + +* **Kubernetes**: The Elastic APM attacher for Kubernetes simplifies the instrumentation and configuration of your application pods. Read more in the [APM attacher for Kubernetes docs](https://www.elastic.co/guide/en/apm/attacher/current/apm-attacher.html). +* **AWS Lambda Functions**: Helps you monitor your AWS Lambda functions. Read more in the [APM Architecture for AWS Lambda docs](https://www.elastic.co/guide/en/apm/lambda/current/aws-lambda-arch.html). +* [8.15.0] **Jaeger**: Helps you to switch an existing Jaeger setup from the default Jaeger backend to the {{stack}}. Read more in [Integrate with Jaeger](../../../solutions/observability/apps/integrate-with-jaeger-deprecated.md). \ No newline at end of file diff --git a/solutions/observability/apps/collect-metrics.md b/solutions/observability/apps/collect-metrics.md index ad2f6d64b8..c207d5ad0f 100644 --- a/solutions/observability/apps/collect-metrics.md +++ b/solutions/observability/apps/collect-metrics.md @@ -4,21 +4,51 @@ mapped_urls: - https://www.elastic.co/guide/en/serverless/current/observability-apm-agents-opentelemetry-collect-metrics.html --- -# Collect metrics +# Collect metrics [apm-open-telemetry-collect-metrics] -% What needs to be done: Align serverless/stateful +::::{important} +When collecting metrics, please note that the [`DoubleValueRecorder`](https://www.javadoc.io/doc/io.opentelemetry/opentelemetry-api/latest/io/opentelemetry/api/metrics/DoubleValueRecorder.md) and [`LongValueRecorder`](https://www.javadoc.io/doc/io.opentelemetry/opentelemetry-api/latest/io/opentelemetry/api/metrics/LongValueObserver.md) metrics are not yet supported. +:::: -% Use migrated content from existing pages that map to this page: -% - [ ] ./raw-migrated-files/observability-docs/observability/apm-open-telemetry-collect-metrics.md -% - [ ] ./raw-migrated-files/docs-content/serverless/observability-apm-agents-opentelemetry-collect-metrics.md +Here’s an example of how to capture business metrics from a Java application. -% Internal links rely on the following IDs being on this page (e.g. as a heading ID, paragraph ID, etc): +```java +// initialize metric +Meter meter = GlobalMetricsProvider.getMeter("my-frontend"); +DoubleCounter orderValueCounter = meter.doubleCounterBuilder("order_value").build(); -$$$apm-open-telemetry-verify-metrics$$$ +public void createOrder(HttpServletRequest request) { -$$$apm-open-telemetry-visualize$$$ + // create order in the database + ... + // increment business metrics for monitoring + orderValueCounter.add(orderPrice); +} +``` -$$$open-telemetry-verify-metrics$$$ +See the [Open Telemetry Metrics API](https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/metrics/api.md) for more information. -$$$open-telemetry-visualize$$$ \ No newline at end of file + +## Verify OpenTelemetry metrics data [apm-open-telemetry-verify-metrics] + +Use **Discover** to validate that metrics are successfully reported to {{kib}}. + +1. Open your Observability instance. +2. Find **Discover** in the main menu or use the [global search field](../../../get-started/the-stack.md#kibana-navigation-search), and select the **Logs Explorer** tab. +3. Click **All logs** → **Data Views** then select **APM**. +4. Filter the data to only show documents with metrics: `processor.name :"metric"` +5. Narrow your search with a known OpenTelemetry field. For example, if you have an `order_value` field, add `order_value: *` to your search to return only OpenTelemetry metrics documents. + + +## Visualize in {{kib}} [apm-open-telemetry-visualize] + +Use **Lens** to create visualizations for OpenTelemetry metrics. Lens enables you to build visualizations by dragging and dropping data fields. It makes smart visualization suggestions for your data, allowing you to switch between visualization types. + +To get started with a new Lens visualization: + +1. Go to **Visualizations**. +2. Click **Create new visualization**. +3. Select **Lens**. + +For more information on using Lens, refer to the [Lens documentation](../../../explore-analyze/visualize/lens.md). \ No newline at end of file diff --git a/solutions/observability/apps/configure-synthetics-projects.md b/solutions/observability/apps/configure-synthetics-projects.md index 1c82ae232d..0d86ee32c1 100644 --- a/solutions/observability/apps/configure-synthetics-projects.md +++ b/solutions/observability/apps/configure-synthetics-projects.md @@ -4,27 +4,295 @@ mapped_urls: - https://www.elastic.co/guide/en/serverless/current/observability-synthetics-configuration.html --- -# Configure Synthetics projects +# Configure a Synthetics project [observability-synthetics-configuration] -% What needs to be done: Align serverless/stateful +Synthetic tests support the configuration of dynamic parameters that can be used in Synthetics projects. In addition, the Synthetics agent, which is built on top of Playwright, supports configuring browser and context options that are available in Playwright-specific methods, for example, `ignoreHTTPSErrors`, `extraHTTPHeaders`, and `viewport`. -% Use migrated content from existing pages that map to this page: +Create a `synthetics.config.js` or `synthetics.config.ts` file in the root of the Synthetics project and specify the options. For example: -% - [ ] ./raw-migrated-files/observability-docs/observability/synthetics-configuration.md -% - [ ] ./raw-migrated-files/docs-content/serverless/observability-synthetics-configuration.md +::::{tab-set} +:group: stack-serverless -% Internal links rely on the following IDs being on this page (e.g. as a heading ID, paragraph ID, etc): +:::{tab-item} Elastic Stack v9 +:sync: stack +```ts +import type { SyntheticsConfig } from '@elastic/synthetics'; -$$$synthetics-config-file$$$ +export default env => { + const config: SyntheticsConfig = { + params: { + url: 'https://www.elastic.co', + }, + playwrightOptions: { + ignoreHTTPSErrors: false, + }, + /** + * Configure global monitor settings + */ + monitor: { + schedule: 10, + locations: [ 'us_east' ], + }, + /** + * Project monitors settings + */ + project: { + id: 'my-project', + url: 'https://abc123', + space: 'custom-space', + }, + }; + if (env !== 'development') { + /** + * Override configuration specific to environment + * For example, config.params.url = "" + */ + } + return config; +}; +``` +::: -$$$synthetics-configuration-monitor-name$$$ +:::{tab-item} Serverless +:sync: serverless -$$$synthetics-configuration-monitor-tags$$$ +```ts +import type { SyntheticsConfig } from '@elastic/synthetics'; -$$$synthetics-configuration-monitor$$$ +export default env => { + const config: SyntheticsConfig = { + params: { + url: 'https://www.elastic.co', + }, + playwrightOptions: { + ignoreHTTPSErrors: false, + }, + /** + * Configure global monitor settings + */ + monitor: { + schedule: 10, + locations: [ 'us_east' ], + }, + /** + * Synthetic project monitors settings + */ + project: { + id: 'my-synthetics-project', + url: 'https://abc123', + }, + }; + if (env !== 'development') { + /** + * Override configuration specific to environment + * For example, config.params.url = "" + */ + } + return config; +}; +``` +::: -$$$synthetics-configuration-params$$$ +:::: -$$$synthetics-configuration-playwright-options$$$ +::::{note} +`env` in the example above is the environment you are pushing from *not* the environment where monitors will run. In other words, `env` corresponds to the configured `NODE_ENV`. -$$$synthetics-configuration-project$$$ \ No newline at end of file +:::: + + +The configuration file can either export an object, or a function that when called should return the generated configuration. To know more about configuring the tests based on environments, look at the [dynamic configuration](../../../solutions/observability/apps/work-with-params-secrets.md#synthetics-dynamic-configs) documentation. + + +## `params` [synthetics-configuration-params] + +A JSON object that defines any variables your tests require. Read more in [Work with params and secrets](../../../solutions/observability/apps/work-with-params-secrets.md). + + +## `playwrightOptions` [synthetics-configuration-playwright-options] + +For all available options, refer to the [Playwright documentation](https://playwright.dev/docs/test-configuration). + +::::{note} +Do not attempt to run in headful mode (using `headless:false`) when running through Elastic’s global managed testing infrastructure or Private Locations as this is not supported. + +:::: + + +Below are details on a few Playwright options that are particularly relevant to Elastic Synthetics including TLS client authentication, timeouts, timezones, and device emulation. + + +### TLS client authentication [synthetics-configuration-playwright-options-client-certificates] + +To enable TLS client authentication, set the [`clientCertificates`](https://playwright.dev/docs/api/class-testoptions#test-options-client-certificates) option in the configuration file: + +::::{note} +Path-based options `{certPath, keyPath, pfxPath}` are only supported on Private Locations, defer to in-memory alternatives `{cert, key, pfx}` when running on locations hosted by Elastic. + +:::: + + +```js +playwrightOptions: { + clientCertificates: [ + { + origin: 'https://example.com', + certPath: './cert.pem', + keyPath: './key.pem', + passphrase: 'mysecretpassword', + }, + { + origin: 'https://example.com', + cert: Buffer.from("-----BEGIN CERTIFICATE-----\n..."), + key: Buffer.from("-----BEGIN RSA PRIVATE KEY-----\n..."), + passphrase: 'mysecretpassword', + } + ], +} +``` + +::::{tip} +When using Synthetics monitor UI, `cert`, `key` and `pfx` can simply be specified using a string literal: + +```js +clientCertificates: [ + { + cert: "-----BEGIN CERTIFICATE-----\n...", + // Not cert: Buffer.from("-----BEGIN CERTIFICATE-----\n..."), + } +], +``` + +:::: + + + +### Timeouts [synthetics-configuration-playwright-options-timeouts] + +Playwright has two types of timeouts that are used in Elastic Synthetics: [action and navigation timeouts](https://playwright.dev/docs/test-timeouts#action-and-navigation-timeouts). + +Elastic Synthetics uses a default action and navigation timeout of 50 seconds. You can override this default using [`actionTimeout`](https://playwright.dev/docs/api/class-testoptions#test-options-action-timeout) and [`navigationTimeout`](https://playwright.dev/docs/api/class-testoptions#test-options-navigation-timeout) in `playwrightOptions`. + + +### Timezones and locales [synthetics-configuration-playwright-options-timezones] + +The Elastic global managed testing infrastructure does not currently set the timezone. For {{private-location}}s, the monitors will use the timezone of the host machine running the {{agent}}. This is not always desirable if you want to test how a web application behaves across different timezones. To specify what timezone to use when the monitor runs, you can use `playwrightOptions` on a per monitor or global basis. + +To use a timezone and/or locale for all monitors in the Synthetics project, set [`locale` and/or `timezoneId`](https://playwright.dev/docs/emulation#locale%2D%2Dtimezone) in the configuration file: + +```js +playwrightOptions: { + locale: 'en-AU', + timezoneId: 'Australia/Brisbane', +} +``` + +To use a timezone and/or locale for a *specific* monitor, add these options to a journey using [`monitor.use`](../../../solutions/observability/apps/configure-individual-browser-monitors.md). + + +### Device emulation [synthetics-config-device-emulation] + +Users can emulate a mobile device using the configuration file. The example configuration below runs tests in "Pixel 5" emulation mode. + +```js +import { SyntheticsConfig } from "@elastic/synthetics" +import { devices } from "playwright-chromium" + +const config: SyntheticsConfig = { + playwrightOptions: { + ...devices['Pixel 5'] + } +} + +export default config; +``` + + +## `project` [synthetics-configuration-project] + +Information about the Synthetics project. + +`id` (`string`) +: A unique id associated with your Synthetics project. It will be used for logically grouping monitors. + + If you used [`init` to create a Synthetics project](../../../solutions/observability/apps/use-synthetics-cli.md#elastic-synthetics-init-command), this is the `` you specified. + + +`url` (`string`) +: The URL for the Serverless Observability project or the {{kib}} URL for the deployment to which you want to upload the monitors. + +space (`string`) +: For {{kib}} deployments 9.0 or higher, the identifier of the target [{{kib}} space](../../../deploy-manage/manage-spaces.md) for the pushed monitors. Spaces help you organize pushed monitors. Pushes to the "default" space if not specified. + + +## `monitor` [synthetics-configuration-monitor] + +Default values to be applied to *all* monitors when using the [`@elastic/synthetics` `push` command](../../../solutions/observability/apps/use-synthetics-cli.md#elastic-synthetics-push-command). + +`id` (`string`) +: A unique identifier for this monitor. + +$$$synthetics-configuration-monitor-name$$$ `name` (`string`) +: A human readable name for the monitor. + +$$$synthetics-configuration-monitor-tags$$$ `tags` (`Array`) +: A list of tags that will be sent with the monitor event. Tags are displayed in the Synthetics UI and allow you to search monitors by tag. + +`schedule` (`number`) +: The interval (in minutes) at which the monitor should run. + +`enabled` (`boolean`) +: Enable or disable the monitor from running without deleting and recreating it. + +`locations` ([`Array`](https://github.com/elastic/synthetics/blob/v1.3.0/src/locations/public-locations.ts#L28-L37)) +: Where to deploy the monitor. Monitors can be deployed in multiple locations so that you can detect differences in availability and response times across those locations. + + To list available locations you can: + + * Run the [`elastic-synthetics locations` command](../../../solutions/observability/apps/use-synthetics-cli.md#elastic-synthetics-locations-command). + * Find `Synthetics` in the [global search field](../../../get-started/the-stack.md#kibana-navigation-search) and click **Create monitor**. Locations will be listed in *Locations*. + + +`privateLocations` (`Array`) +: The [{{private-location}}s](../../../solutions/observability/apps/monitor-resources-on-private-networks.md) to which the monitors will be deployed. These {{private-location}}s refer to locations hosted and managed by you, whereas `locations` are hosted by Elastic. You can specify a {{private-location}} using the location’s name. + + To list available {{private-location}}s you can: + + * Run the [`elastic-synthetics locations` command](../../../solutions/observability/apps/use-synthetics-cli.md#elastic-synthetics-locations-command) with the URL for the Observability project or the {{kib}} URL for the deployment from which to fetch available locations. + * Find `Synthetics` in the [global search field](../../../get-started/the-stack.md#kibana-navigation-search) and click **Create monitor**. {{private-location}}s will be listed in *Locations*. + + +`throttling` (`boolean` | [`ThrottlingOptions`](https://github.com/elastic/synthetics/blob/v1.3.0/src/common_types.ts#L194-L198)) +: Control the monitor’s download speeds, upload speeds, and latency to simulate your application’s behavior on slower or laggier networks. Set to `false` to disable throttling altogether. + +`screenshot` ([`ScreenshotOptions`](https://github.com/elastic/synthetics/blob/v1.3.0/src/common_types.ts#L192)) +: Control whether or not to capture screenshots. Options include `'on'`, `'off'`, or `'only-on-failure'`. + +`alert.status.enabled` (`boolean`) +: Enable or disable monitor status alerts. Read more about alerts in [Alerting](../../../solutions/observability/apps/configure-synthetics-settings.md#synthetics-settings-alerting). + +`retestOnFailure` (`boolean`) +: Enable or disable retesting when a monitor fails. Default is `true`. + + By default, monitors are automatically retested if the monitor goes from "up" to "down". If the result of the retest is also "down", an error will be created, and if configured, an alert sent. Then the monitor will resume running according to the defined schedule. + + Using `retestOnFailure` can reduce noise related to transient problems. + + +`fields` (`object`) +: A list of key-value pairs that will be sent with each monitor event. The `fields` are appended to {{es}} documents as `labels`, and those labels are displayed in {{kib}} in the *Monitor details* panel in the [individual monitor’s *Overview* tab](../../../solutions/observability/apps/analyze-data-from-synthetic-monitors.md#synthetics-analyze-individual-monitors-overview). + + For example: + + ```js + fields: { + foo: 'bar', + team: 'synthetics', + } + ``` + + +For information on configuring monitors individually, refer to: + +* [Configure individual browser monitors](../../../solutions/observability/apps/configure-individual-browser-monitors.md) for browser monitors +* [Configure lightweight monitors](../../../solutions/observability/apps/configure-lightweight-monitors.md) for lightweight monitors \ No newline at end of file diff --git a/solutions/observability/apps/configure-synthetics-settings.md b/solutions/observability/apps/configure-synthetics-settings.md index 5325a8f121..322e189fcf 100644 --- a/solutions/observability/apps/configure-synthetics-settings.md +++ b/solutions/observability/apps/configure-synthetics-settings.md @@ -1,20 +1,104 @@ ---- -mapped_urls: - - https://www.elastic.co/guide/en/observability/current/synthetics-settings.html - - https://www.elastic.co/guide/en/serverless/current/observability-synthetics-settings.html ---- +# Configure Synthetics settings [synthetics-settings] -# Configure Synthetics settings +There are several Synthetics settings you can adjust in Observability. -% What needs to be done: Align serverless/stateful -% Use migrated content from existing pages that map to this page: +## Alerting [synthetics-settings-alerting] -% - [ ] ./raw-migrated-files/observability-docs/observability/synthetics-settings.md -% - [ ] ./raw-migrated-files/docs-content/serverless/observability-synthetics-settings.md +Alerting enables you to detect complex conditions using **rules** across Observability and send a notification using **connectors**. -% Internal links rely on the following IDs being on this page (e.g. as a heading ID, paragraph ID, etc): +When you create a new synthetic monitor, new default synthetics rules will be applied. To edit the default rules: -$$$synthetics-settings-alerting$$$ +1. Click **Alerts and rules** in the top bar. +2. Select a rule to open a panel where you can edit the rule’s configuration: -$$$synthetics-settings-global-parameters$$$ \ No newline at end of file + * **Monitor status rule** for receiving notifications for errors and outages. + * **TLS certificate rule** for receiving notifications when one or more of your HTTP or TCP lightweight monitors has a TLS certificate expiring within a specified threshold or when it exceeds an age limit. + + +However, the automatically created Synthetics internal alert is intentionally preconfigured, and some configuration options can’t be changed. For example, you can’t change how often it checks the rule. + +If you need specific alerting behavior, set up a different rule. To view all existing rules or create a new rule: + +1. Click **Alerts and rules** in the top bar. +2. Click **Manage rules** to go to the *Rules* page. + +On the *Rules* page, you can manage the default synthetics rules including snoozing rules, disabling rules, deleting rules, and more. + +:::{image} ../../../images/observability-synthetics-settings-disable-default-rules.png +:alt: Rules page with default Synthetics rules +:class: screenshot +::: + +::::{note} +You can enable and disable default alerts for individual monitors in a few ways: + +* In the Synthetics UI when you [create a monitor](../../../solutions/observability/apps/create-monitors-in-synthetics-app.md). +* In the Synthetics UI *after* a monitor is already created, on the **Monitors** page or on the **Edit monitor** page for the monitor. +* In Synthetics projects when [configuring a lightweight monitor](../../../solutions/observability/apps/configure-lightweight-monitors.md). + +:::: + + +In the **Alerting** tab on the Synthetics Settings page, you can add and configure connectors. If you are running in Elastic Cloud, then an SMTP connector will automatically be configured, allowing you to easily set up email alerts. Read more about all available connectors in [Action types](../../../solutions/observability/incident-management/create-an-apm-anomaly-rule.md). + +:::{image} ../../../images/observability-synthetics-settings-alerting.png +:alt: Alerting tab on the Synthetics Settings page in {kib} +:class: screenshot +::: + + +## {{private-location}}s [synthetics-settings-private-locations] + +{{private-location}}s allow you to run monitors from your own premises. + +In the **{{private-location}}s** tab, you can add and manage {{private-location}}s. After you [Set up {{fleet-server}} and {{agent}}](../../../solutions/observability/apps/monitor-resources-on-private-networks.md#synthetics-private-location-fleet-agent) and [Connect to the {{stack}} or your serverless Observability project](../../../solutions/observability/apps/monitor-resources-on-private-networks.md#synthetics-private-location-connect), this is where you will add the {{private-location}} so you can specify it as the location for a monitor created using the Synthetics UI or a Synthetics project. + +:::{image} ../../../images/observability-synthetics-settings-private-locations.png +:alt: {{private-location}}s tab on the Synthetics Settings page in {kib} +:class: screenshot +::: + + +## Global parameters [synthetics-settings-global-parameters] + +Global parameters can be defined once and used across the configuration of lightweight and browser-based monitors. + +In the **Global parameters** tab, you can define variables and parameters. This is one of several methods you can use to define variables and parameters. To learn more about the other methods and which methods take precedence over others, see [Work with params and secrets](../../../solutions/observability/apps/work-with-params-secrets.md). + +:::{image} ../../../images/observability-synthetics-settings-global-parameters.png +:alt: Global parameters tab on the Synthetics Settings page in {kib} +:class: screenshot +::: + + +## Data retention [synthetics-settings-data-retention] + +When you set up a synthetic monitor, data from the monitor is saved in [Elasticsearch data streams](../../../manage-data/data-store/data-streams.md), an append-only structure in Elasticsearch. You can customize how long synthetics data is stored by creating your own index lifecycle policy and attaching it to the relevant custom Component Template in Stack Management. + +In the **Data retention** tab, use the links to jump to the relevant policy for each data stream. Learn more about the data included in each data stream in [Manage data retention](../../../solutions/observability/apps/manage-data-retention.md). + +:::{image} ../../../images/observability-synthetics-settings-data-retention.png +:alt: Data retention tab on the Synthetics Settings page in {kib} +:class: screenshot +::: + + +## Project API keys [synthetics-settings-api-keys] + +Project API keys are used to push {{project-monitors}} remotely from a CLI or CD pipeline. + +In the **Project API keys** tab, you can generate project API keys to use with your projects. Learn more about using API keys in [Use {{project-monitors-cap}}](../../../solutions/observability/apps/create-monitors-with-project-monitors.md). + +::::{important} +In an Elastic Stack v9 deployment, to create a Project API key you must be logged into {{kib}} as a user with the privileges described in [Writer role](../../../solutions/observability/apps/writer-role.md). + +In a serverless project, to create a Project API key you must be logged in as a user with [Editor](../../../solutions/observability/apps/grant-users-access-to-secured-resources.md) access. + +:::: + + +:::{image} ../../../images/observability-synthetics-settings-api-keys.png +:alt: Project API keys tab on the Synthetics Settings page in {kib} +:class: screenshot +::: \ No newline at end of file diff --git a/solutions/observability/apps/control-access-to-apm-data.md b/solutions/observability/apps/control-access-to-apm-data.md index 9ec5368b30..4a5227c864 100644 --- a/solutions/observability/apps/control-access-to-apm-data.md +++ b/solutions/observability/apps/control-access-to-apm-data.md @@ -19,7 +19,7 @@ This guide will explain how to separate your staging and production data. This c This guide assumes that you: * Are sending both staging and production APM data to an {{es}} cluster. -* Have configured the `environment` variable in your APM agent configurations. This variable sets the `service.environment` field in APM documents. You should have documents where `service.environment: production` and `service.environment: staging`. If this field is empty, see [service environment filter](filter-application-data.md#environment-selector) to learn how to set this value. +* Have configured the `environment` variable in your APM agent configurations. This variable sets the `service.environment` field in APM documents. You should have documents where `service.environment: production` and `service.environment: staging`. If this field is empty, see [service environment filter](filter-application-data.md#apm-filter-your-data-service-environment-filter) to learn how to set this value. ### Step 1: Create filtered aliases [_step_1_create_filtered_aliases] diff --git a/solutions/observability/apps/create-apm-rules-alerts.md b/solutions/observability/apps/create-apm-rules-alerts.md index 08784b1d32..335e0c9d34 100644 --- a/solutions/observability/apps/create-apm-rules-alerts.md +++ b/solutions/observability/apps/create-apm-rules-alerts.md @@ -2,17 +2,73 @@ mapped_urls: - https://www.elastic.co/guide/en/observability/current/apm-alerts.html - https://www.elastic.co/guide/en/serverless/current/observability-apm-alerts.html + +navigation_title: "Create rules and alerts" --- -# Create APM rules and alerts +# Create APM rules and alerts [apm-alerts] + + +The Applications UI allows you to define **rules** to detect complex conditions within your APM data and trigger built-in **actions** when those conditions are met. + + +## APM rules [apm_rules] + +The following APM rules are supported: + +| | | +| --- | --- | +| **APM Anomaly** | Alert when either the latency, throughput, or failed transaction rate of a service is anomalous.Anomaly rules can be set at the environment level, service level, and/or transaction type level. Read more in [APM Anomaly rule →](../../../solutions/observability/incident-management/create-an-apm-anomaly-rule.md) | +| **Error count threshold** | Alert when the number of errors in a service exceeds a defined threshold. Error count rules can be set at theenvironment level, service level, and error group level. Read more in [Error count threshold rule →](../../../solutions/observability/incident-management/create-an-error-count-threshold-rule.md) | +| **Failed transaction rate threshold** | Alert when the rate of transaction errors in a service exceeds a defined threshold. Read more in [Failed transaction rate threshold rule →](../../../solutions/observability/incident-management/create-failed-transaction-rate-threshold-rule.md) | +| **Latency threshold** | Alert when the latency or failed transaction rate is abnormal.Threshold rules can be as broad or as granular as you’d like, enabling you to define exactly when you want to be alerted—​whether that’s at the environment level, service name level, transaction type level, and/or transaction name level. Read more in [Latency threshold rule →](../../../solutions/observability/incident-management/create-latency-threshold-rule.md) | + +::::{tip} +For a complete walkthrough of the **Create rule** flyout panel, including detailed information on each configurable property, see Kibana’s [Create and manage rules](../../../explore-analyze/alerts-cases/alerts/create-manage-rules.md). + +:::: + +% Stateful only after this? + +## Rules and alerts in the Applications UI [_rules_and_alerts_in_the_applications_ui] + +View and manage rules and alerts in the Applications UI. + + +### View active alerts [apm-alert-view-active] + +Active alerts are displayed and grouped in multiple ways in the Applications UI. + + +#### View alerts by service group [apm-alert-view-group] + +If you’re using the [service groups](../../../solutions/observability/apps/services.md#service-groups) feature, you can view alerts by service group. From the service group overview page, click the red alert indicator to open the **Alerts** tab with a predefined filter that matches the filter used when creating the service group. + +:::{image} ../../../images/observability-apm-service-group.png +:alt: Example view of service group in the Applications UI in Kibana +:class: screenshot +::: + + +#### View alerts by service [apm-alert-view-service] + +Alerts can be viewed within the context of any service. After selecting a service, go to the **Alerts** tab to view any alerts that are active for the selected service. + +:::{image} ../../../images/observability-active-alert-service.png +:alt: View active alerts by service +:class: screenshot +::: + + +### Manage alerts and rules [apm-alert-manage] -% What needs to be done: Align serverless/stateful +From the Applications UI, select **Alerts and rules** → **Manage rules** to be taken to the {{kib}} **{{rules-ui}}** page. From this page, you can disable, mute, and delete APM alerts. -% Use migrated content from existing pages that map to this page: -% - [ ] ./raw-migrated-files/observability-docs/observability/apm-alerts.md -% - [ ] ./raw-migrated-files/docs-content/serverless/observability-apm-alerts.md +### More information [apm-alert-more-info] -% Internal links rely on the following IDs being on this page (e.g. as a heading ID, paragraph ID, etc): +See [Alerting](../../../explore-analyze/alerts-cases.md) for more information. -$$$apm-alert-view-group$$$ \ No newline at end of file +::::{note} +If you are using an **on-premise** Elastic Stack deployment with security, communication between Elasticsearch and Kibana must have TLS configured. More information is in the alerting [prerequisites](../../../explore-analyze/alerts-cases/alerts/alerting-setup.md#alerting-prerequisites). +:::: \ No newline at end of file diff --git a/solutions/observability/apps/create-custom-links.md b/solutions/observability/apps/create-custom-links.md index a852afdb38..503586739d 100644 --- a/solutions/observability/apps/create-custom-links.md +++ b/solutions/observability/apps/create-custom-links.md @@ -2,19 +2,202 @@ mapped_urls: - https://www.elastic.co/guide/en/observability/current/apm-custom-links.html - https://www.elastic.co/guide/en/serverless/current/observability-apm-create-custom-links.html + +navigation_title: "Create custom links" --- -# Create custom links +# Custom links in the Applications UI [apm-custom-links] + +::::{admonition} Required role +:class: note + +For serverless Observability projects, the **Editor** role or higher is required to create and manage custom links. To learn more, refer to [Assign user roles and privileges](../../../deploy-manage/users-roles/cloud-organization/user-roles.md#general-assign-user-roles). + +:::: + + +Elastic’s custom link feature allows you to easily create up to 500 dynamic links based on your specific APM data. Custom links can be filtered to only appear in the Applications UI for relevant services, environments, transaction types, or transaction names. + +Ready to dive in? Jump straight to the [examples](#custom-links-examples). + + +## Create a link [custom-links-create] + +Each custom link consists of a label, URL, and optional filter. The easiest way to create a custom link is from within the actions dropdown in the transaction detail page. This method will automatically apply filters, scoping the link to that specific service, environment, transaction type, and transaction name. + +Alternatively, you can create a custom link in the Applications UI by navigating to **Settings** > **Customize UI**, and selecting **Create custom link**. + + +### Label [custom-links-label] + +The name of your custom link. The actions context menu displays this text, so keep it as short as possible. + +::::{tip} +Custom links are displayed alphabetically in the actions menu. +:::: + + + +### URL [custom-links-url] + +The URL your link points to. URLs support dynamic field name variables, encapsulated in double curly brackets: `{{field.name}}`. These variables will be replaced with transaction metadata when the link is clicked. + +Because everyone’s data is different, you’ll need to examine your traces to see what metadata is available for use. To do this, select a trace in the Applications UI, and click **Metadata** in the **Trace Sample** table. + +:::{image} ../../../images/observability-example-metadata.png +:alt: Example metadata +:class: screenshot +::: + + +### Filters [custom-links-filters] + +Filter each link to only appear for specific services or transactions. You can filter on the following fields: + +* `service.name` +* `service.env` +* `transaction.type` +* `transaction.name` + +Multiple values are allowed when comma-separated. + + +## Custom link examples [custom-links-examples] + +Not sure where to start with custom links? Take a look at the examples below and customize them to your liking! + + +### Email [custom-links-examples-email] + +Email the owner of a service. + +| | | +| --- | --- | +| Label | `Email engineer` | +| Link | `mailto:@.com` | +| Filters | `service.name:` | + +**Example** + +This link opens an email addressed to the team or owner of `python-backend`. It will only appear on services with the name `python-backend`. + +| | | +| --- | --- | +| Label | `Email python-backend engineers` | +| Link | `mailto:python_team@elastic.co` | +| Filters | `service.name:python-backend` | + + +### GitHub issue [custom-links-examples-gh] + +Open a GitHub issue with pre-populated metadata from the selected trace sample. + +| | | +| --- | --- | +| Label | `Open an issue in ` | +| Link | `https://github.com///issues/new?title=&body=<BODY>` | +| Filters | `service.name:client` | + +**Example** + +This link opens a new GitHub issue in the apm-agent-rum repository. It populates the issue body with relevant metadata from the currently active trace. Clicking this link results in the following issue being created: + +:::{image} ../../../images/observability-create-github-issue.png +:alt: Example github issue +:class: screenshot +::: + +| | | +| --- | --- | +| Label | `Open an issue in apm-rum-js` | +| Link | `https://github.com/elastic/apm-agent-rum-js/issues/new?title=Investigate+APM+trace&body=Investigate+the+following+APM+trace%3A%0D%0A%0D%0Aservice.name%3A+{{service.name}}%0D%0Atransaction.id%3A+{{transaction.id}}%0D%0Acontainer.id%3A+{{container.id}}%0D%0Aurl.full%3A+{{url.full}}` | +| Filters | `service.name:client` | + +See the [GitHub automation documentation](https://help.github.com/en/github/managing-your-work-on-github/about-automation-for-issues-and-pull-requests-with-query-parameters) for a full list of supported query parameters. + + +### Jira task [custom-links-examples-jira] + +Create a Jira task with pre-populated metadata from the selected trace sample. + +| | | +| --- | --- | +| Label | `Open an issue in Jira` | +| Link | `https://<JIRA_BASE_URL>/secure/CreateIssueDetails!init.jspa?<ARGUMENTS>` | + +**Example** + +This link creates a new task on the Engineering board in Jira. It populates the issue body with relevant metadata from the currently active trace. Clicking this link results in the following task being created in Jira: + +:::{image} ../../../images/observability-create-jira-issue.png +:alt: Example jira issue +:class: screenshot +::: + +| | | +| --- | --- | +| Label | `Open a task in Jira` | +| Link | `https://test-site-33.atlassian.net/secure/CreateIssueDetails!init.jspa?pid=10000&issuetype=10001&summary=Created+via+APM&description=Investigate+the+following+APM+trace%3A%0D%0A%0D%0Aservice.name%3A+{{service.name}}%0D%0Atransaction.id%3A+{{transaction.id}}%0D%0Acontainer.id%3A+{{container.id}}%0D%0Aurl.full%3A+{{url.full}}` | + +See the [Jira application administration knowledge base](https://confluence.atlassian.com/jirakb/how-to-create-issues-using-direct-html-links-in-jira-server-159474.md) for a full list of supported query parameters. + + +### Dashboards [custom-links-examples-kib] + +Link to a custom dashboard. + +| | | +| --- | --- | +| Label | `Open transaction in custom visualization` | +| Link | `https://kibana-instance/app/kibana#/dashboard?_g=query:(language:kuery,query:'transaction.id:{{transaction.id}}'...` | + +**Example** + +This link opens the current `transaction.id` in a custom dashboard. There are no filters set. + +| | | +| --- | --- | +| Label | `Open transaction in Python drilldown viz` | +| URL | `https://kibana-instance/app/kibana#/dashboard?_g=(filters:!(),refreshInterval:(pause:!t,value:0),time:(from:now-24h,to:now))&_a=(description:'',filters:!(),fullScreenMode:!f,options:(hidePanelTitles:!f,useMargins:!t),panels:!((embeddableConfig:(),gridData:(h:15,i:cb79c1c0-1af8-472c-aaf7-d158a76946fb,w:24,x:0,y:0),id:c8c74b20-6a30-11ea-92ab-b5d3feff11df,panelIndex:cb79c1c0-1af8-472c-aaf7-d158a76946fb,type:visualization,version:'7.7')),query:(language:kuery,query:'transaction.id:{{transaction.id}}'),timeRestore:!f,title:'',viewMode:edit)` | + + +### Slack channel [custom-links-examples-slack] + +Open a specified slack channel. + +| | | +| --- | --- | +| Label | `Open SLACK_CHANNEL` | +| Link | `https://COMPANY_SLACK.slack.com/archives/SLACK_CHANNEL` | +| Filters | `service.name` : `SERVICE_NAME` | + +**Example** + +This link opens a company slack channel, #apm-user-support. It only appears when `transaction.name` is `GET user/login`. + +| | | +| --- | --- | +| Label | `Open #apm-user-support` | +| Link | `https://COMPANY_SLACK.slack.com/archives/efk52kt23k` | +| Filters | `transaction.name:GET user/login` | + -% What needs to be done: Align serverless/stateful +### Website [custom-links-examples-web] -% Use migrated content from existing pages that map to this page: +Open an internal or external website. -% - [ ] ./raw-migrated-files/observability-docs/observability/apm-custom-links.md -% - [ ] ./raw-migrated-files/docs-content/serverless/observability-apm-create-custom-links.md +| | | +| --- | --- | +| Label | `Open <WEBSITE>` | +| Link | `https://<COMPANY_SLACK>.slack.com/archives/<SLACK_CHANNEL>` | +| Filters | `service.name:<SERVICE_NAME>` | -% Internal links rely on the following IDs being on this page (e.g. as a heading ID, paragraph ID, etc): +**Example** -$$$custom-links-examples$$$ +This link opens more data on a specific `user.email`. It only appears on front-end transactions. -$$$observability-apm-create-custom-links-custom-link-examples$$$ \ No newline at end of file +| | | +| --- | --- | +| Label | `View user internally` | +| Link | `https://internal-site.company.com/user/{{user.email}}` | +| Filters | `service.name:client` | \ No newline at end of file diff --git a/solutions/observability/apps/create-monitors-in-synthetics-app.md b/solutions/observability/apps/create-monitors-in-synthetics-app.md index e8b16ba9b5..20ba2c3d05 100644 --- a/solutions/observability/apps/create-monitors-in-synthetics-app.md +++ b/solutions/observability/apps/create-monitors-in-synthetics-app.md @@ -2,21 +2,129 @@ mapped_urls: - https://www.elastic.co/guide/en/observability/current/synthetics-get-started-ui.html - https://www.elastic.co/guide/en/serverless/current/observability-synthetics-get-started-ui.html + +navigation_title: "Use the Synthetics UI" --- -# Create monitors in the Synthetics app +# Create monitors in the Synthetics UI [synthetics-get-started-ui] + +You can create synthetic monitors directly in the UI by opening an Observability project and navigating to **Synthetics**. + +:::{image} ../../../images/observability-synthetics-get-started-ui.png +:alt: Diagram showing which pieces of software are used to configure monitors +::: + +This is one of [two approaches](../../../solutions/observability/apps/get-started.md) you can use to set up a synthetic monitor. + + +## Prerequisites [synthetics-get-started-ui-prerequisites] + +For **serverless Observability projects**, you must be signed in as a user with [Editor](../../../solutions/observability/apps/grant-users-access-to-secured-resources.md) access. + +For **Elastic Stack deployments**, you must be signed into {{kib}} as a user with at least [synthetics write permissions](../../../solutions/observability/apps/writer-role.md), and Monitor Management must be enabled by an administrator as described in [Setup role](../../../solutions/observability/apps/setup-role.md). + +You should decide where you want to run the monitors before getting started. You can run monitors on one or both of the following: + +* **Elastic’s global managed testing infrastructure**: With Elastic’s global managed testing infrastructure, you can create and run monitors in multiple locations without having to manage your own infrastructure. Elastic takes care of software updates and capacity planning for you. +* **{{private-location}}s**: {{private-location}}s allow you to run monitors from your own premises. To use {{private-location}}s you must create a {{private-location}} before continuing. For step-by-step instructions, refer to [Monitor resources on private networks](../../../solutions/observability/apps/monitor-resources-on-private-networks.md). + +Executing synthetic tests on Elastic’s global managed testing infrastructure incurs an additional charge. Tests are charged under one of two new billing dimensions depending on the monitor type. For *browser monitor* usage, there is a fee per test run. For *lightweight monitor* usage, there is a fee per region in which you run any monitors regardless of the number of test runs. For more details, refer to the [{{obs-serverless}} pricing page](https://www.elastic.co/pricing/serverless-observability). + + +## Add a lightweight monitor [synthetics-get-started-ui-add-a-lightweight-monitor] + +To use the UI to add a lightweight monitor: + +1. Find `Synthetics` in the [global search field](/explore-analyze/find-and-organize/find-apps-and-objects.md). +2. Click **Create monitor**. +3. Set the monitor type to **HTTP Ping**, **TCP Ping**, or **ICMP Ping**. +4. In *Locations*, select one or more locations. + + ::::{note} + If you don’t see any locations listed, refer to the [troubleshooting guide](../../../troubleshoot/observability/troubleshooting-synthetics.md#synthetics-troubleshooting-no-locations) for guidance. + + :::: + + + :::::{note} + If you’ve [added a {{private-location}}](../../../solutions/observability/apps/monitor-resources-on-private-networks.md), you’ll see your the {{private-location}} in the list of *Locations*. + + :::{image} ../../../images/serverless-private-locations-monitor-locations.png + :alt: Screenshot of Monitor locations options including a {private-location} + :class: screenshot + ::: + + ::::: + +5. Set the *Frequency*, and configure the monitor as needed. +6. Click **Advanced options** to see more ways to configure your monitor. +7. (Optional) Click **Run test** to verify that the test is valid. +8. Click **Create monitor**. + + :::{image} ../../../images/observability-synthetics-get-started-ui-lightweight.png + :alt: Synthetics Create monitor UI + :class: screenshot + ::: + + + +## Add a browser monitor [synthetics-get-started-ui-add-a-browser-monitor] + +You can also create a browser monitor in the UI using an **Inline script**. + +An inline script contains a single journey that you manage individually. Inline scripts can be quick to set up, but can also be more difficult to manage. Each browser monitor configured using an inline script can contain only *one* journey, which must be maintained directly in the UI. + +If you depend on external packages, have your journeys next to your code repository, or want to embed and manage more than one journey from a single monitor configuration, use a [Synthetics project](../../../solutions/observability/apps/create-monitors-with-project-monitors.md) instead. + +To use the UI to add a browser monitor: + +1. Click **Create monitor**. +2. Set the monitor type to **Multistep**. +3. In *Locations*, select one or more locations. + + ::::{note} + If you don’t see any locations listed, refer to the [troubleshooting guide](../../../troubleshoot/observability/troubleshooting-synthetics.md#synthetics-troubleshooting-no-locations) for guidance. + + :::: + +4. Set the *Frequency*. +5. Add steps to the **Script editor** code block directly. The `journey` keyword isn’t required, and variables like `page` and `params` will be part of your script’s scope. You cannot `import` any dependencies when using inline browser monitors. + + :::{image} ../../../images/observability-synthetics-ui-inline-script.png + :alt: Configure a synthetic monitor using an inline script in Elastic {{fleet}} + :class: screenshot + ::: + + ::::{note} + Alternatively, you can use the **Script recorder** option. You can use the Elastic Synthetics Recorder to interact with a web page, export journey code that reflects all the actions you took, and upload the results in the UI. For more information, refer to [Use the Synthetics Recorder](../../../solutions/observability/apps/use-synthetics-recorder.md). + + :::: + +6. Click **Advanced options** to see more ways to configure your monitor. + + * Use **Data options** to add context to the data coming from your monitors. + * Use the **Synthetics agent options** to provide fine-tuned configuration for the synthetics agent. Read more about available options in [Use the Synthetics CLI](../../../solutions/observability/apps/use-synthetics-cli.md). + +7. (Optional) Click **Run test** to verify that the test is valid. +8. Click **Create monitor**. + + +## View in your Observability project [synthetics-get-started-ui-view-in-your-observability-project] + +Navigate to **Synthetics**, where you can see screenshots of each run, set up alerts in case of test failures, and more. -% What needs to be done: Align serverless/stateful +If a test does fail (shown as `down` in the Synthetics UI), you’ll be able to view the step script that failed, any errors, and a stack trace. For more information, refer to [Analyze data from synthetic monitors](../../../solutions/observability/apps/analyze-data-from-synthetic-monitors.md#synthetics-analyze-journeys). -% Use migrated content from existing pages that map to this page: +::::{note} +When a monitor is created or updated, the first run might not occur immediately, but the time it takes for the first run to occur will be less than the monitor’s configured frequency. For example, if you create a monitor and configure it to run every 10 minutes, the first run will occur within 10 minutes of being created. After the first run, the monitor will begin running regularly based on the configured frequency. You can run a manual test if you want to see the results more quickly. -% - [ ] ./raw-migrated-files/observability-docs/observability/synthetics-get-started-ui.md -% - [ ] ./raw-migrated-files/docs-content/serverless/observability-synthetics-get-started-ui.md +:::: -% Internal links rely on the following IDs being on this page (e.g. as a heading ID, paragraph ID, etc): -$$$observability-synthetics-get-started-ui-add-a-browser-monitor$$$ +## Next steps [synthetics-get-started-ui-next-steps] -$$$private-locations$$$ +Learn more about: -$$$synthetics-get-started-ui-browser$$$ \ No newline at end of file +* [Writing user journeys](../../../solutions/observability/apps/write-synthetic-test.md) to use as inline scripts +* Using the [Synthetics Recorder](../../../solutions/observability/apps/use-synthetics-recorder.md) +* [Configuring lightweight monitors](../../../solutions/observability/apps/configure-lightweight-monitors.md) \ No newline at end of file diff --git a/solutions/observability/apps/create-monitors-with-project-monitors.md b/solutions/observability/apps/create-monitors-with-project-monitors.md index 94ad45d56a..0492676f5b 100644 --- a/solutions/observability/apps/create-monitors-with-project-monitors.md +++ b/solutions/observability/apps/create-monitors-with-project-monitors.md @@ -2,23 +2,281 @@ mapped_urls: - https://www.elastic.co/guide/en/observability/current/synthetics-get-started-project.html - https://www.elastic.co/guide/en/serverless/current/observability-synthetics-get-started-project.html + +navigation_title: "Use a Synthetics project" --- -# Create monitors with Project monitors +# Create monitors with a Synthetics project [observability-synthetics-get-started-project] + + +A Synthetics project is the most powerful and sophisticated way to configure synthetic monitors. A Synthetics project lets you define your infrastructure as code, more commonly known as IaaC or Git-ops. With monitors created and managed in Synthetics projects, you organize your YAML configuration and JavaScript- or TypeScript-defined monitors on the filesystem, use Git for version control, and deploy via a CLI tool, usually executed on a CI/CD platform. + +:::{image} ../../../images/observability-synthetics-get-started-projects.png +:alt: Diagram showing which pieces of software are used to configure monitors +::: + +This is one of [two approaches](../../../solutions/observability/apps/get-started.md) you can use to set up a synthetic monitor. + + +## Prerequisites [synthetics-get-started-project-prerequisites] + +For **serverless Observability projects**, you must be signed in as a user with [Editor](../../../solutions/observability/apps/grant-users-access-to-secured-resources.md) access. + +For **Elastic Stack deployments**, you must be signed into {{kib}} as a user with at least [synthetics write permissions](../../../solutions/observability/apps/writer-role.md), and Monitor Management must be enabled by an administrator as described in [Setup role](../../../solutions/observability/apps/setup-role.md). + +Working with a Synthetics project requires working with the Elastic Synthetics CLI tool, which can be invoked via the `npx @elastic/synthetics` command. Before getting started you’ll need to: + +1. Install [Node.js](https://nodejs.dev/en/) +2. Install the package: + + ```sh + npm install -g @elastic/synthetics + ``` + +3. Confirm your system is setup correctly: + + ```sh + npx @elastic/synthetics -h + ``` + + +You should also decide where you want to run the monitors before getting started. You can run monitors in Synthetics projects on one or both of the following: + +* **Elastic’s global managed testing infrastructure**: With Elastic’s global managed testing infrastructure, you can create and run monitors in multiple locations without having to manage your own infrastructure. Elastic takes care of software updates and capacity planning for you. +* **{{private-location}}s**: {{private-location}}s allow you to run monitors from your own premises. To use {{private-location}}s you must create a {{private-location}} before continuing. For step-by-step instructions, refer to [Monitor resources on private networks](../../../solutions/observability/apps/monitor-resources-on-private-networks.md). + +% Stateful only for following note? + +::::{note} +If you are setting up Synthetics for a deployment configured with [traffic filters](../../../deploy-manage/security/traffic-filtering.md), connections into {{es}} are restricted and results will not be able to be written back into {{es}} unless granted. For more details, refer to [Use Synthetics with traffic filters](../../../solutions/observability/apps/use-synthetics-with-traffic-filters.md). + +:::: + + +## Create a Synthetics project [synthetics-get-started-project-create-a-synthetics-project] + +Start by creating your first Synthetics project. Run the command below to create a new Synthetics project named `synthetic-project-test` in the current directory. + +```sh +npx @elastic/synthetics init synthetic-project-test +``` + +Then, follow the prompts on screen to set up the correct default variables for your Synthetics project. When complete, set the `SYNTHETICS_API_KEY` environment variable in your terminal, which is used to connect to your Observability project: + +::::{tab-set} +:group: stack-serverless + +:::{tab-item} Elastic Stack v9 +:sync: stack + +1. To generate an API key: + + 1. Go to **Synthetics** in your Observability project. + 2. Click **Settings**. + 3. Switch to the **Project API Keys** tab. + 4. Click **Generate Project API key**. + + ::::{important} + To generate a Project API key, you must be logged in as a user with [Editor](../../../solutions/observability/apps/grant-users-access-to-secured-resources.md) access. + + :::: + + + :::{image} ../../../images/serverless-synthetics-monitor-management-api-key.png + :alt: Project API Keys tab in Synthetics settings + :class: screenshot + ::: + + ::::{note} + To use an API key to push to Elastic’s global managed testing infrastructure, the *Elastic managed locations enabled* toggle must be on when generating the API key. If the *Elastic managed locations enabled* toggle is disabled, an administrator has restricted access to Elastic’s global managed testing infrastructure. + + :::: + +2. Set the `SYNTHETICS_API_KEY` environment variable in your terminal. You will most likely want to set this permanently. This is done differently in [Powershell](https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_environment_variables?view=powershell-7.2#saving-changes-to-environment-variables) and [Bash](https://unix.stackexchange.com/a/117470). + +Then, take a look at key files and directories inside your Synthetics project: + +* `journeys` is where you’ll add `.ts` and `.js` files defining your browser monitors. When you create a new Synthetics project, this directory will contain files defining sample monitors. +* `lightweight` is where you’ll add `.yaml` files defining your lightweight monitors. When you create a new Synthetics project, this directory will contain a file defining sample monitors. +* `synthetics.config.ts` contains settings for your Synthetics project. When you create a new Synthetics project, it will contain some basic configuration options that you can customize later. + + ::::{note} + The `synthetics.config.ts` in the sample Synthetics project uses a location on Elastic’s global managed testing infrastructure. Administrators can restrict access to Elastic’s global managed testing infrastructure. When you attempt to [`push` the sample monitors](../../../solutions/observability/apps/create-monitors-with-project-monitors.md#synthetics-get-started-project-test-and-connect-to-your-observability-project), if you see an error stating that you don’t have permission to use Elastic managed global locations, refer to the [troubleshooting guide](../../../troubleshoot/observability/troubleshooting-synthetics.md#synthetics-troubleshooting-no-locations) for guidance. + + :::: + +* `package.json` contains NPM settings for your Synthetics project. Learn more in the [NPM documentation](https://docs.npmjs.com/about-packages-and-modules). +* `.github` contains sample workflow files to use with GitHub Actions. +::: + +:::{tab-item} Serverless +:sync: serverless + +1. To generate an API key: + + 1. Go to **Synthetics** in your Observability project. + 2. Click **Settings**. + 3. Switch to the **Project API Keys** tab. + 4. Click **Generate Project API key**. + + ::::{important} + To generate a Project API key, you must be logged in as a user with [Editor](../../../solutions/observability/apps/grant-users-access-to-secured-resources.md) access. + + :::: + + + :::{image} ../../../images/serverless-synthetics-monitor-management-api-key.png + :alt: Project API Keys tab in Synthetics settings + :class: screenshot + ::: + + ::::{note} + To use an API key to push to Elastic’s global managed testing infrastructure, the *Elastic managed locations enabled* toggle must be on when generating the API key. If the *Elastic managed locations enabled* toggle is disabled, an administrator has restricted access to Elastic’s global managed testing infrastructure. + + :::: + +2. Set the `SYNTHETICS_API_KEY` environment variable in your terminal. You will most likely want to set this permanently. This is done differently in [Powershell](https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_environment_variables?view=powershell-7.2#saving-changes-to-environment-variables) and [Bash](https://unix.stackexchange.com/a/117470). + +Then, take a look at key files and directories inside your Synthetics project: + +* `journeys` is where you’ll add `.ts` and `.js` files defining your browser monitors. When you create a new Synthetics project, this directory will contain files defining sample monitors. +* `lightweight` is where you’ll add `.yaml` files defining your lightweight monitors. When you create a new Synthetics project, this directory will contain a file defining sample monitors. +* `synthetics.config.ts` contains settings for your Synthetics project. When you create a new Synthetics project, it will contain some basic configuration options that you can customize later. + + ::::{note} + The `synthetics.config.ts` in the sample Synthetics project uses a location on Elastic’s global managed testing infrastructure. Administrators can restrict access to Elastic’s global managed testing infrastructure. When you attempt to [`push` the sample monitors](../../../solutions/observability/apps/create-monitors-with-project-monitors.md#synthetics-get-started-project-test-and-connect-to-your-observability-project), if you see an error stating that you don’t have permission to use Elastic managed global locations, refer to the [troubleshooting guide](../../../troubleshoot/observability/troubleshooting-synthetics.md#synthetics-troubleshooting-no-locations) for guidance. + + :::: + +* `package.json` contains NPM settings for your Synthetics project. Learn more in the [NPM documentation](https://docs.npmjs.com/about-packages-and-modules). +* `.github` contains sample workflow files to use with GitHub Actions. + +::: + +:::: + +## Examine sample monitors [synthetics-get-started-project-examine-sample-monitors] + +Inside the `lightweight` directory you’ll find sample lightweight monitors. Here’s an example of a YAML file defining a lightweight monitor: + +```yaml +# lightweight.yml +heartbeat.monitors: +- type: http + name: Todos Lightweight + id: todos-lightweight + urls: "https://elastic.github.io/synthetics-demo/" + schedule: '@every 1m' +``` + +For more details on lightweight monitor configuration options, refer to [Configure lightweight monitors](../../../solutions/observability/apps/configure-lightweight-monitors.md). + +Inside the `journeys` directory you’ll find sample browser monitors. Here’s an example of a TypeScript file defining a browser monitor: + +```ts +// example.journey.ts +import { journey, step, monitor, expect } from '@elastic/synthetics'; +journey('My Example Journey', ({ page, params }) => { + // Only relevant for the push command to create + // monitors in Kibana or your serverless Observability project + monitor.use({ + id: 'example-monitor', + schedule: 10, + }); + step('launch application', async () => { + await page.goto(params.url); + }); + step('assert title', async () => { + const header = await page.locator('h1'); + expect(await header.textContent()).toBe('todos'); + }); +}); +``` + +For more details on writing journeys and configuring browser monitors, refer to [Scripting browser monitors](../../../solutions/observability/apps/scripting-browser-monitors.md). + + +## Test and connect to your Observability project or Elastic Stack deployment[synthetics-get-started-project-test-and-connect-to-your-observability-project] + +::::{tab-set} +:group: stack-serverless + +:::{tab-item} Elastic Stack v9 +:sync: stack + +While inside the project directory you can do two things with the `npx @elastic/synthetics` command: + +* Test browser-based monitors locally. To run all journeys defined in `.ts` and `.js` files: + + ```sh + npx @elastic/synthetics journeys + ``` + +* Push all monitor configurations to an Elastic deployment. Run the following command from inside your project: + + ```sh + npx @elastic/synthetics push --auth $SYNTHETICS_API_KEY --url <kibana-url> + ``` + + +One monitor will appear in the {{synthetics-app}} for each journey or lightweight monitor, and you’ll manage all monitors from your local environment. For more details on using the `push` command, refer to [`@elastic/synthetics push`](../../../solutions/observability/apps/use-synthetics-cli.md#elastic-synthetics-push-command). + +::::{note} +If you’ve [added a {{private-location}}](../../../solutions/observability/apps/monitor-resources-on-private-networks.md), you can `push` to that {{private-location}}. + +To list available {{private-location}}s, run the [`elastic-synthetics locations` command](../../../solutions/observability/apps/use-synthetics-cli.md#elastic-synthetics-locations-command) with the {{kib}} URL for the deployment from which to fetch available locations. + +:::: + +::: + +:::{tab-item} Serverless +:sync: serverless + +While inside the Synthetics project directory you can do two things with the `npx @elastic/synthetics` command: + +* Test browser-based monitors locally. To run all journeys defined in `.ts` and `.js` files: + + ```sh + npx @elastic/synthetics journeys + ``` + +* Push all monitor configurations to an Observability project. Run the following command from inside your Synthetics project directory: + + ```sh + npx @elastic/synthetics push --auth $SYNTHETICS_API_KEY --url <observability-project-url> + ``` + + +One monitor will appear in the Synthetics UI for each journey or lightweight monitor, and you’ll manage all monitors from your local environment. For more details on using the `push` command, refer to [`@elastic/synthetics push`](../../../solutions/observability/apps/use-synthetics-cli.md#elastic-synthetics-push-command). + +::::{note} +If you’ve [added a {{private-location}}](../../../solutions/observability/apps/monitor-resources-on-private-networks.md), you can `push` to that {{private-location}}. + +To list available {{private-location}}s, run the [`elastic-synthetics locations` command](../../../solutions/observability/apps/use-synthetics-cli.md#elastic-synthetics-locations-command) with the URL for the Observability project from which to fetch available locations. + +:::: + +::: + +:::: + + +## View in the Synthetics UI [synthetics-get-started-project-view-in-your-observability-project] -% What needs to be done: Align serverless/stateful +Then, go to **Synthetics** in your serverless Observability project or in {{kib}}. You should see your newly pushed monitors running. You can also go to the **Management** tab to see the monitors' configuration settings. -% Use migrated content from existing pages that map to this page: +::::{note} +When a monitor is created or updated, the first run might not occur immediately, but the time it takes for the first run to occur will be less than the monitor’s configured frequency. For example, if you create a monitor and configure it to run every 10 minutes, the first run will occur within 10 minutes of being created. After the first run, the monitor will begin running regularly based on the configured frequency. You can run a manual test if you want to see the results more quickly. -% - [ ] ./raw-migrated-files/observability-docs/observability/synthetics-get-started-project.md -% - [ ] ./raw-migrated-files/docs-content/serverless/observability-synthetics-get-started-project.md +:::: -% Internal links rely on the following IDs being on this page (e.g. as a heading ID, paragraph ID, etc): -$$$observability-synthetics-get-started-project-create-a-synthetics-project$$$ -$$$observability-synthetics-get-started-project-test-and-connect-to-your-observability-project$$$ +## Next steps [observability-synthetics-get-started-project-next-steps] -$$$synthetics-get-started-project-init$$$ +Learn more about: -$$$synthetics-get-started-project-push$$$ \ No newline at end of file +* [Configuring lightweight monitors](../../../solutions/observability/apps/configure-lightweight-monitors.md) +* [Configuring browser monitors](../../../solutions/observability/apps/write-synthetic-test.md) +* [Implementing best practices for working with Synthetics projects](../../../solutions/observability/apps/manage-monitors.md#synthetics-projects-best-practices) \ No newline at end of file diff --git a/solutions/observability/apps/dependencies.md b/solutions/observability/apps/dependencies.md index ec72996214..7f96b12fc6 100644 --- a/solutions/observability/apps/dependencies.md +++ b/solutions/observability/apps/dependencies.md @@ -4,11 +4,48 @@ mapped_urls: - https://www.elastic.co/guide/en/serverless/current/observability-apm-dependencies.html --- -# Dependencies +# Dependencies [apm-dependencies] -% What needs to be done: Align serverless/stateful +APM agents collect details about external calls made from instrumented services. Sometimes, these external calls resolve into a downstream service that’s instrumented — in these cases, you can utilize [distributed tracing](../../../solutions/observability/apps/trace-sample-timeline.md#distributed-tracing) to drill down into problematic downstream services. Other times, though, it’s not possible to instrument a downstream dependency — like with a database or third-party service. **Dependencies** gives you a window into these uninstrumented, downstream dependencies. -% Use migrated content from existing pages that map to this page: +:::{image} ../../../images/observability-dependencies.png +:alt: Dependencies view in the Applications UI +:class: screenshot +::: -% - [ ] ./raw-migrated-files/observability-docs/observability/apm-dependencies.md -% - [ ] ./raw-migrated-files/docs-content/serverless/observability-apm-dependencies.md \ No newline at end of file +Many application issues are caused by slow or unresponsive downstream dependencies. And because a single, slow dependency can significantly impact the end-user experience, it’s important to be able to quickly identify these problems and determine the root cause. + +Select a dependency to see detailed latency, throughput, and failed transaction rate metrics. + +:::{image} ../../../images/observability-dependencies-drilldown.png +:alt: Dependencies drilldown view in the Applications UI +:class: screenshot +::: + +When viewing a dependency, consider your pattern of usage with that dependency. If your usage pattern *hasn’t* increased or decreased, but the experience has been negatively affected—either with an increase in latency or errors—there’s likely a problem with the dependency that needs to be addressed. + +If your usage pattern *has* changed, the dependency view can quickly show you whether that pattern change exists in all upstream services, or just a subset of your services. You might then start digging into traces coming from impacted services to determine why that pattern change has occurred. + + +## Operations [dependencies-operations] + +::::{admonition} Dependency operations is in beta +:class: important + +The Dependency operations functionality is in beta and is subject to change. The design and code is less mature than official generally available features and is being provided as-is with no warranties. + +:::: + +**Dependency operations** provides a granular breakdown of the operations/queries a dependency is executing. + +:::{image} ../../../images/observability-operations.png +:alt: operations view in the Applications UI +:class: screenshot +::: + +Selecting an operation displays the operation’s impact and performance trends over time, via key metrics like latency, throughput, and failed transaction rate. In addition, the [**Trace sample timeline**](../../../solutions/observability/apps/trace-sample-timeline.md) provides a visual drill-down into an end-to-end trace sample. + +:::{image} ../../../images/observability-operations-detail.png +:alt: operations detail view in the Applications UI +:class: screenshot +::: diff --git a/solutions/observability/apps/drill-down-into-data.md b/solutions/observability/apps/drill-down-into-data.md index d29c41acea..0384aaf2e9 100644 --- a/solutions/observability/apps/drill-down-into-data.md +++ b/solutions/observability/apps/drill-down-into-data.md @@ -2,13 +2,18 @@ mapped_urls: - https://www.elastic.co/guide/en/observability/current/apm-ui-drill-down.html - https://www.elastic.co/guide/en/serverless/current/observability-apm-ui-drill-down.html + +navigation_title: "Drill down into data" --- -# Drill down into data +# Drill down into application data [apm-ui-drill-down] -% What needs to be done: Align serverless/stateful -% Use migrated content from existing pages that map to this page: +Notice something awry? Select a service or trace and dive deeper with: -% - [ ] ./raw-migrated-files/observability-docs/observability/apm-ui-drill-down.md -% - [ ] ./raw-migrated-files/docs-content/serverless/observability-apm-ui-drill-down.md \ No newline at end of file +* [Transactions](../../../solutions/observability/apps/transactions-2.md) +* [Trace sample timeline](../../../solutions/observability/apps/trace-sample-timeline.md) +* [Errors](../../../solutions/observability/apps/errors-2.md) +* [Metrics](../../../solutions/observability/apps/metrics-2.md) +* [Infrastructure](../../../solutions/observability/apps/infrastructure.md) +* [Logs](../../../solutions/observability/apps/logs.md) diff --git a/solutions/observability/apps/errors-2.md b/solutions/observability/apps/errors-2.md index 49f52e2144..306bf40a19 100644 --- a/solutions/observability/apps/errors-2.md +++ b/solutions/observability/apps/errors-2.md @@ -4,11 +4,31 @@ mapped_urls: - https://www.elastic.co/guide/en/serverless/current/observability-apm-errors.html --- -# Errors +# Errors [apm-errors] -% What needs to be done: Align serverless/stateful +::::{tip} +[Errors](/solutions/observability/apps/errors.md) are groups of exceptions with a similar exception or log message. +:::: -% Use migrated content from existing pages that map to this page: -% - [ ] ./raw-migrated-files/observability-docs/observability/apm-errors.md -% - [ ] ./raw-migrated-files/docs-content/serverless/observability-apm-errors.md \ No newline at end of file +*Errors* are groups of exceptions with a similar exception or log message. The **Errors** overview provides a high-level view of the exceptions that APM agents catch, or that users manually report with APM agent APIs. Like errors are grouped together to make it easy to quickly see which errors are affecting your services, and to take actions to rectify them. + +A service returning a 5xx code from a request handler, controller, etc., will not create an exception that an APM agent can catch, and will therefore not show up in this view. + +:::{image} ../../../images/observability-apm-errors-overview.png +:alt: APM Errors overview +:class: screenshot +::: + +Selecting an error group ID or error message brings you to the **Error group**. + +:::{image} ../../../images/observability-apm-error-group.png +:alt: APM Error group +:class: screenshot +::: + +The error group details page visualizes the number of error occurrences over time and compared to a recent time range. This allows you to quickly determine if the error rate is changing or remaining constant. You’ll also see the top 5 affected transactions—​enabling you to quickly narrow down which transactions are most impacted by the selected error. + +Further down, you’ll see an Error sample. The error shown is always the most recent to occur. The sample includes the exception message, culprit, stack trace where the error occurred, and additional contextual information to help debug the issue—​all of which can be copied with the click of a button. + +In some cases, you might also see a Transaction sample ID. This feature allows you to make a connection between the errors and transactions, by linking you to the specific transaction where the error occurred. This allows you to see the whole trace, including which services the request went through. \ No newline at end of file diff --git a/solutions/observability/apps/filter-application-data.md b/solutions/observability/apps/filter-application-data.md index ae15b1f810..51cd4f823b 100644 --- a/solutions/observability/apps/filter-application-data.md +++ b/solutions/observability/apps/filter-application-data.md @@ -2,17 +2,44 @@ mapped_urls: - https://www.elastic.co/guide/en/observability/current/apm-filters.html - https://www.elastic.co/guide/en/serverless/current/observability-apm-filter-your-data.html + +navigation_title: "Filters" --- -# Filter application data +# Filter application data [apm-filter-your-data] + + +Global filters are ways you can filter your APM data based on a specific time range or environment. When viewing a specific service, the filter persists as you move between tabs. + +:::{image} ../../../images/observability-global-filters.png +:alt: Global filters view +:class: screenshot +::: + +::::{note} +If you prefer to use advanced queries on your data to filter on specific pieces of information, see [Query your data](../../../solutions/observability/apps/use-advanced-queries-on-application-data.md). + +:::: + + + +## Global time range [apm-filter-your-data-global-time-range] + +The global time range filter restricts APM data to a specific time period. -% What needs to be done: Align serverless/stateful -% Use migrated content from existing pages that map to this page: +## Service environment filter [apm-filter-your-data-service-environment-filter] -% - [ ] ./raw-migrated-files/observability-docs/observability/apm-filters.md -% - [ ] ./raw-migrated-files/docs-content/serverless/observability-apm-filter-your-data.md +The environment selector is a global filter for `service.environment`. It allows you to view only relevant data and is especially useful for separating development from production environments. By default, all environments are displayed. If there are no environment options, you’ll see "not defined". -% Internal links rely on the following IDs being on this page (e.g. as a heading ID, paragraph ID, etc): +Service environments are defined when configuring your APM agents. It’s vital to be consistent when naming environments in your APM agents. To learn how to configure service environments, see the specific APM agent documentation: -$$$environment-selector$$$ \ No newline at end of file +* **Go:** [`ELASTIC_APM_ENVIRONMENT`](https://www.elastic.co/guide/en/apm/agent/go/current/configuration.html#config-environment) +* **iOS agent:** *Not yet supported* +* **Java:** [`environment`](https://www.elastic.co/guide/en/apm/agent/java/current/config-core.html#config-environment) +* **.NET:** [`Environment`](https://www.elastic.co/guide/en/apm/agent/dotnet/current/config-core.html#config-environment) +* **Node.js:** [`environment`](https://www.elastic.co/guide/en/apm/agent/nodejs/current/configuration.html#environment) +* **PHP:** [`environment`](https://www.elastic.co/guide/en/apm/agent/php/current/configuration-reference.html#config-environment) +* **Python:** [`environment`](https://www.elastic.co/guide/en/apm/agent/python/current/configuration.html#config-environment) +* **Ruby:** [`environment`](https://www.elastic.co/guide/en/apm/agent/ruby/current/configuration.html#config-environment) +* **Real User Monitoring:** [`environment`](https://www.elastic.co/guide/en/apm/agent/rum-js/current/configuration.html#environment) \ No newline at end of file diff --git a/solutions/observability/apps/filter-search-application-data.md b/solutions/observability/apps/filter-search-application-data.md index a834d19028..f92312eb8a 100644 --- a/solutions/observability/apps/filter-search-application-data.md +++ b/solutions/observability/apps/filter-search-application-data.md @@ -2,13 +2,15 @@ mapped_urls: - https://www.elastic.co/guide/en/observability/current/apm-filter-and-search-data.html - https://www.elastic.co/guide/en/serverless/current/observability-apm-filter-and-search-data.html + +navigation_title: "Filter and search data" --- -# Filter and search application data +# Filter and search application data [apm-filter-and-search-data] -% What needs to be done: Align serverless/stateful -% Use migrated content from existing pages that map to this page: +Because Elastic APM is built on top of the {{stack}}, you have the full power of Elastic’s powerful search capabilities to filter and search through your application data. Mastering how to filter and search your data can help you find bottlenecks in your code faster: -% - [ ] ./raw-migrated-files/observability-docs/observability/apm-filter-and-search-data.md -% - [ ] ./raw-migrated-files/docs-content/serverless/observability-apm-filter-and-search-data.md \ No newline at end of file +* Use global filters to [filter data](../../../solutions/observability/apps/filter-application-data.md) across the Applications UI based on a specific time range or environment. +* Use [advanced queries](../../../solutions/observability/apps/use-advanced-queries-on-application-data.md) on your data to filter on specific pieces of information. +* Use {{es}}'s [cross-cluster search](../../../solutions/observability/apps/cross-cluster-search-with-application-data.md) functionality to search APM data across multiple sources. \ No newline at end of file diff --git a/solutions/observability/apps/find-transaction-latency-failure-correlations.md b/solutions/observability/apps/find-transaction-latency-failure-correlations.md index 0a8aace8f8..ab4fbb17df 100644 --- a/solutions/observability/apps/find-transaction-latency-failure-correlations.md +++ b/solutions/observability/apps/find-transaction-latency-failure-correlations.md @@ -4,17 +4,79 @@ mapped_urls: - https://www.elastic.co/guide/en/serverless/current/observability-apm-find-transaction-latency-and-failure-correlations.html --- -# Find transaction latency and failure correlations -% What needs to be done: Align serverless/stateful +# Find transaction latency and failure correlations [observability-apm-find-transaction-latency-and-failure-correlations] -% Use migrated content from existing pages that map to this page: +Correlations surface attributes of your data that are potentially correlated with high-latency or erroneous transactions. For example, if you are a site reliability engineer who is responsible for keeping production systems up and running, you want to understand what is causing slow transactions. Identifying attributes that are responsible for higher latency transactions can potentially point you toward the root cause. You may find a correlation with a particular piece of hardware, like a host or pod. Or, perhaps a set of users, based on IP address or region, is facing increased latency due to local data center issues. -% - [ ] ./raw-migrated-files/observability-docs/observability/apm-correlations.md -% - [ ] ./raw-migrated-files/docs-content/serverless/observability-apm-find-transaction-latency-and-failure-correlations.md +To find correlations: -% Internal links rely on the following IDs being on this page (e.g. as a heading ID, paragraph ID, etc): +::::{tab-set} +:group: stack-serverless -$$$correlations-latency$$$ +:::{tab-item} Elastic Stack v9 +:sync: stack -$$$observability-apm-find-transaction-latency-and-failure-correlations-find-high-transaction-latency-correlations$$$ \ No newline at end of file +Select a service on the **Services** page in the Applications UI then select a transaction group from the **Transactions** tab. + +::::{note} +Queries within the Applications UI are also applied to the correlations. +:::: + +::: + +:::{tab-item} Serverless +:sync: serverless + +1. In your {{obs-serverless}} project, go to **Applications** → **Service Inventory**. +2. Select a service. +3. Select the **Transactions** tab. +4. Select a transaction group in the **Transactions** table. + +::::{note} +Active queries *are* applied to correlations. + +:::: + +::: + +:::: + + +## Find high transaction latency correlations [observability-apm-find-transaction-latency-and-failure-correlations-find-high-transaction-latency-correlations] + +The correlations on the **Latency correlations** tab help you discover which attributes are contributing to increased transaction latency. + +:::{image} ../../../images/observability-correlations-hover.png +:alt: APM latency correlations +:class: screenshot +::: + +The progress bar indicates the status of the asynchronous analysis, which performs statistical searches across a large number of attributes. For large time ranges and services with high transaction throughput, this might take some time. To improve performance, reduce the time range. + +The latency distribution chart visualizes the overall latency of the transactions in the transaction group. If there are attributes that have a statistically significant correlation with slow response times, they are listed in a table below the chart. The table is sorted by correlation coefficients that range from 0 to 1. Attributes with higher correlation values are more likely to contribute to high latency transactions. By default, the attribute with the highest correlation value is added to the chart. To see the latency distribution for other attributes, select their row in the table. + +If a correlated attribute seems noteworthy, use the **Filter** quick links: + +* `+` creates a new query in the Applications UI for filtering transactions containing the selected value. +* `-` creates a new query in the Applications UI to filter out transactions containing the selected value. + +You can also click the icon beside the field name to view and filter its most popular values. + +In this example screenshot, there are transactions that are skewed to the right with slower response times than the overall latency distribution. If you select the `+` filter in the appropriate row of the table, it creates a new query in the Applications UI for transactions with this attribute. With the "noise" now filtered out, you can begin viewing sample traces to continue your investigation. + + +## Find failed transaction correlations [correlations-error-rate] + +The correlations on the **Failed transaction correlations** tab help you discover which attributes are most influential in distinguishing between transaction failures and successes. In this context, the success or failure of a transaction is determined by its [event.outcome](https://www.elastic.co/guide/en/ecs/current/ecs-event.html#field-event-outcome) value. For example, APM agents set the `event.outcome` to `failure` when an HTTP transaction returns a `5xx` status code. + +The chart highlights the failed transactions in the overall latency distribution for the transaction group. If there are attributes that have a statistically significant correlation with failed transactions, they are listed in a table. The table is sorted by scores, which are mapped to high, medium, or low impact levels. Attributes with high impact levels are more likely to contribute to failed transactions. By default, the attribute with the highest score is added to the chart. To see a different attribute in the chart, select its row in the table. + +For example, in the screenshot below, there are attributes such as a specific node and pod name that have medium impact on the failed transactions. + +:::{image} ../../../images/observability-correlations-failed-transactions.png +:alt: Failed transaction correlations +:class: screenshot +::: + +Select the `+` filter to create a new query in the Applications UI for transactions with one or more of these attributes. If you are unfamiliar with a field, click the icon beside its name to view its most popular values and optionally filter on those values too. Each time that you add another attribute, it is filtering out more and more noise and bringing you closer to a diagnosis. \ No newline at end of file diff --git a/solutions/observability/apps/get-started.md b/solutions/observability/apps/get-started.md index b663f3af80..3461fa1196 100644 --- a/solutions/observability/apps/get-started.md +++ b/solutions/observability/apps/get-started.md @@ -4,17 +4,46 @@ mapped_urls: - https://www.elastic.co/guide/en/serverless/current/observability-synthetics-get-started.html --- -# Get started +# Get started [observability-synthetics-get-started] -% What needs to be done: Align serverless/stateful +To set up a synthetic monitor, you need to configure the monitor, run it, and send data back to Elastic. After setup is complete, the data will be available in your serverless Observability project or in {{kib}} to view, analyze, and alert on. -% Use migrated content from existing pages that map to this page: +There are two ways to set up a synthetic monitor: -% - [ ] ./raw-migrated-files/observability-docs/observability/synthetics-get-started.md -% - [ ] ./raw-migrated-files/docs-content/serverless/observability-synthetics-get-started.md +* Synthetics project +* The Synthetics UI -% Internal links rely on the following IDs being on this page (e.g. as a heading ID, paragraph ID, etc): +Read more about each option below, and choose the approach that works best for you. -$$$choose-projects$$$ -$$$observability-synthetics-get-started-synthetics-project$$$ +## Synthetics project [observability-synthetics-get-started-synthetics-project] + +With a Synthetics project, you write tests in an external version-controlled Node.js project using YAML for lightweight monitors and JavaScript or TypeScript for browser monitors. Then, you use the `@elastic/synthetics` NPM library’s `push` command to create monitors. + +This approach works well if you want to create both browser monitors and lightweight monitors. It also allows you to configure and update monitors using a GitOps workflow. + +Get started in [Create monitors in a Synthetics project](../../../solutions/observability/apps/create-monitors-with-project-monitors.md). + +:::{image} ../../../images/observability-synthetics-get-started-projects.png +:alt: Diagram showing which pieces of software are used to configure monitors +::: + + +## Synthetics UI [observability-synthetics-get-started-synthetics-ui] + +You can create monitors directly in the user interface. This approach works well if you want to create and manage your monitors in the browser. + +Get started in [Create monitors in the Synthetics UI](../../../solutions/observability/apps/create-monitors-in-synthetics-app.md). + +:::{image} ../../../images/observability-synthetics-get-started-ui.png +:alt: Diagram showing which pieces of software are used to configure monitors +::: + +::::{note} +The Elastic Synthetics integration is a method for creating synthetic monitors that is no longer recommended. **Do not use the Elastic Synthetics integration to set up new monitors.** + +For details on how to migrate from Elastic Synthetics integration to {{project-monitors}} or the {{synthetics-app}}, refer to [Migrate from the Elastic Synthetics integration](../../../solutions/observability/apps/migrate-from-elastic-synthetics-integration.md). + +If you’ve used the Elastic Synthetics integration to create monitors in the past and need to reference documentation about the integration, go to the [8.3 documentation](https://www.elastic.co/guide/en/observability/8.3/uptime-set-up.html#uptime-set-up-choose-agent). + +:::: \ No newline at end of file diff --git a/solutions/observability/apps/grant-users-access-to-secured-resources.md b/solutions/observability/apps/grant-users-access-to-secured-resources.md index 8b7ef2cac8..e7da241a85 100644 --- a/solutions/observability/apps/grant-users-access-to-secured-resources.md +++ b/solutions/observability/apps/grant-users-access-to-secured-resources.md @@ -4,11 +4,53 @@ mapped_urls: - https://www.elastic.co/guide/en/serverless/current/observability-synthetics-feature-roles.html --- -# Grant users access to secured resources +# Grant users access to secured resources [observability-synthetics-feature-roles] -% What needs to be done: Align serverless/stateful +You can use role-based access control to grant users access to secured resources. The roles that you set up depend on your organization’s security requirements and the minimum privileges required to use specific features. -% Use migrated content from existing pages that map to this page: +::::{tab-set} +:group: stack-serverless -% - [ ] ./raw-migrated-files/observability-docs/observability/synthetics-feature-roles.md -% - [ ] ./raw-migrated-files/docs-content/serverless/observability-synthetics-feature-roles.md \ No newline at end of file +:::{tab-item} Elastic Stack v9 +:sync: stack + +Typically you need the create the following separate roles: + +* [Setup role](../../../solutions/observability/apps/setup-role.md) for enabling Monitor Management. +* [Writer role](../../../solutions/observability/apps/writer-role.md) for creating, modifying, and deleting monitors. +* [Reader role](../../../solutions/observability/apps/reader-role.md) for {{kib}} users who need to view and create visualizations that access Synthetics data. + +{{es-security-features}} provides [built-in roles](../../../deploy-manage/users-roles/cluster-or-deployment-auth/built-in-roles.md) that grant a subset of the privileges needed by Synthetics users. When possible, assign users the built-in roles to minimize the affect of future changes on your security strategy. If no built-in role is available, you can assign users the privileges needed to accomplish a specific task. + +In general, these are types of privileges you’ll work with: + +* **{{es}} cluster privileges**: Manage the actions a user can perform against your cluster. +* **{{es}} index privileges**: Control access to the data in specific indices your cluster. +* **{{kib}} space privileges**: Grant users write or read access to features and apps within {{kib}}. + +::: + +:::{tab-item} Serverless +:sync: serverless + +* **Viewer**: + + * View and create visualizations that access Synthetics data. + +* **Editor**: + + * Create, modify, and delete monitors. + * View and create visualizations that access Synthetics data. + +* **Admin**: + + * Full access to project management, properties, and security privileges. + * Create, modify, and delete monitors. + * View and create visualizations that access Synthetics data. + + +Read more about user roles in [Assign user roles and privileges](../../../deploy-manage/users-roles/cloud-organization/user-roles.md). + +::: + +:::: \ No newline at end of file diff --git a/solutions/observability/apps/infrastructure.md b/solutions/observability/apps/infrastructure.md index de6efeddc7..d4b1c53b76 100644 --- a/solutions/observability/apps/infrastructure.md +++ b/solutions/observability/apps/infrastructure.md @@ -4,11 +4,33 @@ mapped_urls: - https://www.elastic.co/guide/en/serverless/current/observability-apm-infrastructure.html --- -# Infrastructure +# Infrastructure [observability-apm-infrastructure] -% What needs to be done: Align serverless/stateful +::::{admonition} Applications UI Infrastructure is in beta +:class: important -% Use migrated content from existing pages that map to this page: +The Applications UI Infrastructure functionality is in beta and is subject to change. The design and code is less mature than official generally available features and is being provided as-is with no warranties. -% - [ ] ./raw-migrated-files/observability-docs/observability/apm-infrastructure.md -% - [ ] ./raw-migrated-files/docs-content/serverless/observability-apm-infrastructure.md \ No newline at end of file +:::: + + +The **Infrastructure** tab provides information about the containers, pods, and hosts that the selected service is linked to. + +* **Pods**: Uses the `kubernetes.pod.name` from the [APM metrics data streams](../../../solutions/observability/apps/metrics.md). +* **Containers**: Uses the `container.id` from the [APM metrics data streams](../../../solutions/observability/apps/metrics.md). +* **Hosts**: If the application is containerized—​if the APM metrics documents include `container.id`-- the `host.name` is used from the infrastructure data streams (filtered by `container.id`). If not, `host.hostname` is used from the APM metrics data streams. + + +:::{image} ../../../images/serverless-infra.png +:alt: Example view of the Infrastructure tab in the Applications UI +:class: screenshot +::: + +IT ops and software reliability engineers (SREs) can use this tab to quickly find a service’s underlying infrastructure resources when debugging a problem. Knowing what infrastructure is related to a service allows you to remediate issues by restarting, killing hanging instances, changing configuration, rolling back deployments, scaling up, scaling out, and so on. + +::::{admonition} Why is the infrastructure tab empty? +:class: tip + +If there is no data in the Application UI’s infrastructure tab for a selected service, you can read more about why this happens and how to fix it in the [troubleshooting docs](../../../troubleshoot/observability/apm/common-problems.md#troubleshooting-apm-infra-data). + +:::: \ No newline at end of file diff --git a/solutions/observability/apps/integrate-with-machine-learning.md b/solutions/observability/apps/integrate-with-machine-learning.md index a5b5e5f291..0cae88b3ed 100644 --- a/solutions/observability/apps/integrate-with-machine-learning.md +++ b/solutions/observability/apps/integrate-with-machine-learning.md @@ -4,17 +4,51 @@ mapped_urls: - https://www.elastic.co/guide/en/serverless/current/observability-apm-integrate-with-machine-learning.html --- -# Integrate with machine learning +# Integrate with machine learning [observability-apm-integrate-with-machine-learning] -% What needs to be done: Align serverless/stateful +::::{important} +Using machine learning requires an [appropriate license](https://www.elastic.co/subscriptions). -% Use migrated content from existing pages that map to this page: +:::: -% - [ ] ./raw-migrated-files/observability-docs/observability/apm-machine-learning-integration.md -% - [ ] ./raw-migrated-files/docs-content/serverless/observability-apm-integrate-with-machine-learning.md +The Machine learning integration initiates a new job predefined to calculate anomaly scores on APM transaction durations. With this integration, you can quickly pinpoint anomalous transactions and see the health of any upstream and downstream services. -% Internal links rely on the following IDs being on this page (e.g. as a heading ID, paragraph ID, etc): +Machine learning jobs are created per environment and are based on a service’s average response time. Because jobs are created at the environment level, you can add new services to your existing environments without the need for additional machine learning jobs. -$$$create-ml-integration$$$ +Results from machine learning jobs are shown in multiple places throughout the Applications UI: -$$$observability-apm-integrate-with-machine-learning-enable-anomaly-detection$$$ \ No newline at end of file +* The **Services overview** provides a quick-glance view of the general health of all of your services. +* The transaction duration chart will show the expected bounds and add an annotation when the anomaly score is 75 or above. +* Service Maps will display a color-coded anomaly indicator based on the detected anomaly score. + + :::{image} ../../../images/observability-apm-service-map-anomaly.png + :alt: Example view of anomaly scores on service maps in the Applications UI + :class: screenshot + ::: + + +## Enable anomaly detection [observability-apm-integrate-with-machine-learning-enable-anomaly-detection] + +To enable machine learning anomaly detection: + +1. In your {{obs-serverless}} project or {{kib}}, go to any **Applications** page. +2. Click **Anomaly detection**. +3. Click **Create Job**. +4. Machine learning jobs are created at the environment level. Select all of the service environments that you want to enable anomaly detection in. Anomalies will surface for all services and transaction types within the selected environments. +5. Click **Create Jobs**. + +That’s it! After a few minutes, the job will begin calculating results; it might take additional time for results to appear on your service maps. To manage existing jobs, click **Manage jobs** (or go to **Machine learning** → **Jobs**). + + +## Anomaly detection warning [observability-apm-integrate-with-machine-learning-anomaly-detection-warning] + +To make machine learning as easy as possible to set up, Elastic will warn you when filtered to an environment without a machine learning job. + + +## Unknown service health [observability-apm-integrate-with-machine-learning-unknown-service-health] + +After enabling anomaly detection, service health may display as "Unknown". Here are some reasons why this can occur: + +1. No machine learning job exists. See [Enable anomaly detection](../../../solutions/observability/apps/integrate-with-machine-learning.md#observability-apm-integrate-with-machine-learning-enable-anomaly-detection) to enable anomaly detection and create a machine learning job. +2. There is no machine learning data for the job. If you just created the machine learning job you’ll need to wait a few minutes for data to be available. Alternatively, if the service or its environment are new, you’ll need to wait for more trace data. +3. No "request" or "page-load" transaction type exists for this service; service health is only available for these transaction types. \ No newline at end of file diff --git a/solutions/observability/apps/interpret-application-data.md b/solutions/observability/apps/interpret-application-data.md index 9a7ac22e46..66c0cc57c8 100644 --- a/solutions/observability/apps/interpret-application-data.md +++ b/solutions/observability/apps/interpret-application-data.md @@ -2,13 +2,20 @@ mapped_urls: - https://www.elastic.co/guide/en/observability/current/apm-interpret-data.html - https://www.elastic.co/guide/en/serverless/current/observability-apm-interpret-data.html + +navigation_title: "Interpret data" --- -# Interpret application data +# Interpret application data [observability-apm-interpret-data] + -% What needs to be done: Align serverless/stateful +Learn how to get the most out of your data using the Applications UI. -% Use migrated content from existing pages that map to this page: +% Stateful only for exploring mobile sessions with Discover? -% - [ ] ./raw-migrated-files/observability-docs/observability/apm-interpret-data.md -% - [ ] ./raw-migrated-files/docs-content/serverless/observability-apm-interpret-data.md \ No newline at end of file +| | | +| --- | --- | +| [Finding transaction latency and failure correlations](../../../solutions/observability/apps/find-transaction-latency-failure-correlations.md) | Surface characteristics of your data that are potentially correlated with high-latency or erroneous transactions. | +| [Tracking deployments with annotations](../../../solutions/observability/apps/track-deployments-with-annotations.md) | Annotations enable you to easily determine if your deployment has increased response times for an end-user or if the memory/CPU footprint of your application has changed. | +| [Exploring mobile sessions with Discover](../../../solutions/observability/apps/explore-mobile-sessions-with-discover.md) | Use session tracking via a globally unique identifier to explore the activities of a specific user during a specific period of time. | +| [Observing Lambda functions](../../../solutions/observability/apps/observe-lambda-functions.md) | Learn how your AWS Lambda functions relate to and depend on other services, and get insight into function execution and runtime behavior, like lambda duration, cold start rate, cold start duration, compute usage, memory usage, and more. | \ No newline at end of file diff --git a/solutions/observability/apps/learn-about-application-data-types.md b/solutions/observability/apps/learn-about-application-data-types.md index d23ec0147a..ea5a708577 100644 --- a/solutions/observability/apps/learn-about-application-data-types.md +++ b/solutions/observability/apps/learn-about-application-data-types.md @@ -4,11 +4,23 @@ mapped_urls: - https://www.elastic.co/guide/en/serverless/current/observability-apm-data-types.html --- -# Learn about application data types +# Application data types [observability-apm-data-types] -% What needs to be done: Align serverless/stateful +Elastic APM agents capture different types of information from within their instrumented applications. These are known as events, and can be spans, transactions, traces, errors, or metrics. -% Use migrated content from existing pages that map to this page: +Elastic APM helps you see what happens from start to finish when a request is made to an application: -% - [ ] ./raw-migrated-files/observability-docs/observability/apm-data-model.md -% - [ ] ./raw-migrated-files/docs-content/serverless/observability-apm-data-types.md \ No newline at end of file +* [**Spans**](../../../solutions/observability/apps/spans.md): A span contain information about the execution of a specific code path. They are the building blocks of *transactions* and *traces*. +* [**Transactions**](../../../solutions/observability/apps/transactions.md): A transaction describes an event captured by an Elastic APM agent instrumenting a service. A transaction is technically a type of span that has additional attributes associated with it and often contains multiple child *spans*. You can think of transactions as the highest level of work you’re measuring within a service. +* [**Traces**](../../../solutions/observability/apps/traces.md#apm-distributed-tracing): A trace is a group of *transactions* and *spans* with a common root. Each trace tracks the entirety of a single request. When a trace travels through multiple services, it is known as a *distributed trace*. + +:::{image} ../../../images/observability-spans-transactions-and-traces.png +:alt: Diagram illustrating the relationship between spans +::: + +In addition to the building blocks of traces, Elastic APM agents also capture: + +* [**Errors**](../../../solutions/observability/apps/errors.md): An error is created when something goes wrong with a request to an application. This event contains information to help you determine where and why an error occurred, often including in which *transaction* the error occurred. +* [**Metrics**](../../../solutions/observability/apps/metrics.md): Metrics measure the state of a system by gathering information on a regular interval. + +Events can contain additional [metadata](../../../solutions/observability/apps/metadata.md) which further enriches your data. \ No newline at end of file diff --git a/solutions/observability/apps/limitations.md b/solutions/observability/apps/limitations.md index 2b2d2a2f67..f456a3aebf 100644 --- a/solutions/observability/apps/limitations.md +++ b/solutions/observability/apps/limitations.md @@ -4,15 +4,44 @@ mapped_urls: - https://www.elastic.co/guide/en/serverless/current/observability-apm-agents-opentelemetry-limitations.html --- -# Limitations +# Limitations [apm-open-telemetry-known-limitations] -% What needs to be done: Align serverless/stateful -% Use migrated content from existing pages that map to this page: +## OpenTelemetry traces [apm-open-telemetry-traces-limitations] -% - [ ] ./raw-migrated-files/observability-docs/observability/apm-open-telemetry-known-limitations.md -% - [ ] ./raw-migrated-files/docs-content/serverless/observability-apm-agents-opentelemetry-limitations.md +* Traces of applications using `messaging` semantics might be wrongly displayed as `transactions` in the Applications UI, while they should be considered `spans` (see issue [#7001](https://github.com/elastic/apm-server/issues/7001)). +* Inability to see Stack traces in spans. +* Inability in APM views to view the "Time Spent by Span Type" (see issue [#5747](https://github.com/elastic/apm-server/issues/5747)). -% Internal links rely on the following IDs being on this page (e.g. as a heading ID, paragraph ID, etc): -$$$apm-open-telemetry-tbs$$$ \ No newline at end of file +## OpenTelemetry logs [apm-open-telemetry-logs-intake] + +* [preview] The OpenTelemetry logs intake via Elastic is in technical preview. +* The application logs data stream (`app_logs`) has dynamic mapping disabled. This means the automatic detection and mapping of new fields is disabled (see issue [#9093](https://github.com/elastic/apm-server/issues/9093)). + + +## OpenTelemetry Line Protocol (OTLP) [apm-open-telemetry-otlp-limitations] + +Elastic supports both the [OTLP/gRPC](https://opentelemetry.io/docs/specs/otlp/#otlpgrpc) and [OTLP/HTTP](https://opentelemetry.io/docs/specs/otlp/#otlphttp) protocol with ProtoBuf payload. Elastic does not yet support JSON Encoding for OTLP/HTTP. + + +## OpenTelemetry Collector exporter for Elastic [apm-open-telemetry-collector-exporter] + +The [OpenTelemetry Collector exporter for Elastic](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/v0.57.2/exporter/elasticexporter) has been deprecated and replaced by the native support of the OpenTelemetry Line Protocol in Elastic Observability (OTLP). + +The [OpenTelemetry Collector exporter for Elastic](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/exporter/elasticsearchexporter) (which is different from the legacy exporter mentioned above) is not intended to be used with Elastic APM and Elastic Observability. Use [Elastic’s native OTLP support](../../../solutions/observability/apps/upstream-opentelemetry-collectors-language-sdks.md) instead. + +% Statefull only for tail-based sampling? + +## OpenTelemetry’s tail-based sampling [apm-open-telemetry-tbs] + +Tail-based sampling allows to make sampling decisions after all spans of a trace have been completed. This allows for more powerful and informed sampling rules. + +When using OpenTelemetry with Elastic APM, there are two different implementations available for tail-based sampling: + +* Tail-based sampling using the [tailsamplingprocessor](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/processor/tailsamplingprocessor) in the OpenTelemetry Collector +* Native [tail-based sampling in the Elastic APM backend](../../../solutions/observability/apps/transaction-sampling.md#apm-tail-based-sampling) + +Using the [tailsamplingprocessor](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/processor/tailsamplingprocessor) in the OpenTelemetry Collector comes with an important limitation. Elastic’s APM backend calculates span and transaction metrics based on the incoming span events. These metrics are accurate for 100% sampling scenarios. In scenarios with probabilistic sampling, Elastic’s APM backend is being informed about the sampling rate of spans and can extrapolate throughput metrics based on the incoming, partial data. However, with tail-based sampling there’s no clear probability for sampling decisions as the rules can be more complex and the OpenTelemetry Collector does not provide sampling probability information to the Elastic backend that could be used for extrapolation of data. Therefore, there’s no way for Elastic APM to properly extrapolate throughput and count metrics that are derived from span events that have been tail-based sampled in the OpenTelemetry Collector. In these scenarios, derived throughput and count metrics are likely to be inaccurate. + +Therefore, we recommend using Elastic’s native tail-based sampling when integrating with OpenTelemetry. \ No newline at end of file diff --git a/solutions/observability/apps/logs.md b/solutions/observability/apps/logs.md index 8f706f485e..94367245fe 100644 --- a/solutions/observability/apps/logs.md +++ b/solutions/observability/apps/logs.md @@ -4,11 +4,31 @@ mapped_urls: - https://www.elastic.co/guide/en/serverless/current/observability-apm-logs.html --- -# Logs +# Logs [apm-logs] -% What needs to be done: Align serverless/stateful +The **Logs** tab shows contextual logs for the selected service. -% Use migrated content from existing pages that map to this page: +Logs provide detailed information about specific events, and are crucial to successfully debugging slow or erroneous transactions. -% - [ ] ./raw-migrated-files/observability-docs/observability/apm-logs.md -% - [ ] ./raw-migrated-files/docs-content/serverless/observability-apm-logs.md \ No newline at end of file +If you’ve correlated your application’s logs and traces, you never have to search for relevant data; it’s already available to you. Viewing log and trace data together allows you to quickly diagnose and solve problems. + +To learn how to correlate your logs with your instrumented services, refer to [Stream application logs](../../../solutions/observability/logs/stream-application-logs.md). + +:::{image} ../../../images/observability-logs.png +:alt: Example view of the Logs tab in Applications UI +:class: screenshot +::: + +::::{tip} +Logs displayed on this page are filtered on `service.name` +:::: + +## Integrate with logging frameworks [apm-logs-correlation] + +Elastic APM integrates with popular logging frameworks, making it easy to correlate your logs and traces. This enables you to: + +* View the context of a log and the parameters provided by a user +* View all logs belonging to a particular trace +* Easily move between logs and traces when debugging application issues + +See the [Stream application logs](../../../solutions/observability/logs/stream-application-logs.md) guide to get started. \ No newline at end of file diff --git a/solutions/observability/apps/manage-data-retention.md b/solutions/observability/apps/manage-data-retention.md index 35d8694d9a..2e0cbff8c6 100644 --- a/solutions/observability/apps/manage-data-retention.md +++ b/solutions/observability/apps/manage-data-retention.md @@ -4,11 +4,58 @@ mapped_urls: - https://www.elastic.co/guide/en/serverless/current/observability-synthetics-manage-retention.html --- -# Manage data retention +# Manage data retention [synthetics-manage-retention] -% What needs to be done: Align serverless/stateful +When you set up a synthetic monitor, data from the monitor is saved in [{{es}} data streams](../../../manage-data/data-store/data-streams.md), an append-only structure in {{es}}. -% Use migrated content from existing pages that map to this page: +There are six data streams recorded by synthetic monitors: `http`, `tcp`, `icmp`, `browser`, `browser.network`, `browser.screenshot`. Elastic will retain data from each data stream for some time period, and the default time period varies by data stream. If you want to reduce the amount of storage required or store data for longer, you can customize how long to retain data for each data stream. -% - [ ] ./raw-migrated-files/observability-docs/observability/synthetics-manage-retention.md -% - [ ] ./raw-migrated-files/docs-content/serverless/observability-synthetics-manage-retention.md \ No newline at end of file + +## Synthetics data streams [synthetics-manage-retention-data-streams] + +There are six data streams recorded by synthetic monitors: + +| Data stream | Data includes | Default retention period | | +| --- | --- | --- | --- | +| `http` | The URL that was checked, the status of the check, and any errors that occurred | 1 year | | +| `tcp` | The URL that was checked, the status of the check, and any errors that occurred | 1 year | | +| `icmp` | The URL that was checked, the status of the check, and any errors that occurred | 1 year | | +| `browser` | The URL that was checked, the status of the check, and any errors that occurred | 1 year | | +| `browser.screenshot` | Binary image data used to construct a screenshot and metadata with information related to de-duplicating this data | 14 days | | +| `browser.network` | Detailed metadata around requests for resources required by the pages being checked | 14 days | | + +All types of checks record core metadata. Browser-based checks store two additional types of data: network and screenshot documents. These browser-specific indices are usually many times larger than the core metadata. The relative sizes of each vary depending on the sites being checked with network data usually being the larger of the two by a significant factor. + + +## Customize data stream lifecycles [synthetics-manage-retention-customize] + +If Synthetics browser data streams are storing data longer than necessary, you can opt to retain data for a shorter period. + +To find Synthetics data streams: + +::::{tab-set} +:group: stack-serverless + +:::{tab-item} Elastic Stack v9 +:sync: stack + +1. Navigate to [{{kib}} index management](../../../manage-data/lifecycle/index-lifecycle-management/index-management-in-kibana.md). +2. Filter the list of data streams for those containing the term `synthetics`. + + 1. In the UI there will be three types of browser data streams: `synthetics-browser-*`, `synthetics-browser.network-*`, and `synthetics-browser.screenshot-*`. + +::: + +:::{tab-item} Serverless +:sync: serverless + +1. Navigate to **Project settings** → **Management*** → ***Index Management** → **Data Streams**. +2. Filter the list of data streams for those containing the term `synthetics`. + + 1. In the UI there will be three types of browser data streams: `synthetics-browser-*`, `synthetics-browser.network-*`, and `synthetics-browser.screenshot-*`. + +::: + +:::: + +Then, you can refer to [Tutorial: Customize data retention for integrations](https://www.elastic.co/guide/en/fleet/current/data-streams-ilm-tutorial.html) to learn how to apply a custom {{ilm-init}} policy to the browser data streams. \ No newline at end of file diff --git a/solutions/observability/apps/manage-monitors.md b/solutions/observability/apps/manage-monitors.md index 5dd946ebc1..d25cc26f8b 100644 --- a/solutions/observability/apps/manage-monitors.md +++ b/solutions/observability/apps/manage-monitors.md @@ -4,15 +4,113 @@ mapped_urls: - https://www.elastic.co/guide/en/serverless/current/observability-synthetics-manage-monitors.html --- -# Manage monitors +# Manage monitors [synthetics-manage-monitors] -% What needs to be done: Align serverless/stateful +After you’ve [created a synthetic monitor](../../../solutions/observability/apps/get-started.md), you’ll need to manage that monitor over time. This might include updating or permanently deleting an existing monitor. -% Use migrated content from existing pages that map to this page: +If you’re using {{project-monitors}}, you should also set up a workflow that uses [best practices for managing monitors effectively](../../../solutions/observability/apps/manage-monitors.md#synthetics-projects-best-practices) in a production environment. -% - [ ] ./raw-migrated-files/observability-docs/observability/synthetics-manage-monitors.md -% - [ ] ./raw-migrated-files/docs-content/serverless/observability-synthetics-manage-monitors.md -% Internal links rely on the following IDs being on this page (e.g. as a heading ID, paragraph ID, etc): +## Update a monitor [manage-monitors-config] -$$$synthetics-projects-best-practices$$$ \ No newline at end of file +You can update a monitor’s configuration, for example, changing the interval at which the monitor runs a test. + +You can also update the journey used in a browser monitor. For example, if you update the UI used in your application, you may want to update your journey’s selectors and assertions. + +:::::::{tab-set} + +::::::{tab-item} Project monitors +If you [set up the monitor using a Synthetic project](../../../solutions/observability/apps/create-monitors-with-project-monitors.md), you’ll update the monitor in the Synthetic project source and then `push` changes. + +For lightweight monitors, make changes to the YAML file. + +For browser monitors, you can update the configuration of one or more monitors: + +* To update the configuration of an individual monitor, edit the journey directly in the JavaScript or TypeScript files, specifically the options in `monitor.use`. +* To update the configuration of *all* monitors in a Synthetic project, edit the [global synthetics configuration file](../../../solutions/observability/apps/configure-synthetics-projects.md#synthetics-configuration-monitor). + +To update the journey that a browser monitor runs, edit the journey code directly and [test the updated journey locally](../../../solutions/observability/apps/write-synthetic-test.md#synthetics-test-locally) to make sure it’s valid. + +After making changes to the monitors, run the [`push` command](../../../solutions/observability/apps/use-synthetics-cli.md#elastic-synthetics-push-command) to replace the existing monitors with new monitors using the updated configuration or journey code. + +::::{note} +Updates are linked to a monitor’s `id`. To update a monitor you must keep its `id` the same. +:::: +:::::: + +::::::{tab-item} Synthetics UI +If you [set up the monitor using the Synthetics UI](../../../solutions/observability/apps/create-monitors-in-synthetics-app.md), you can update the monitor configuration of both lightweight and browser monitors in the {{synthetics-app}}: + +1. Go to **Management**. +2. Click the pencil icon next to the monitor you want to edit. +3. Update the *Monitor settings* as needed. + + 1. To update the journey used in a browser monitor, edit *Inline script*. + 2. Make sure to click **Run test** to validate the new journey before updating the monitor. + +4. Click **Update monitor**. +:::::: + +::::::: + +## Delete a monitor [manage-monitors-delete] + +Eventually you might want to delete a monitor altogether. For example, if the user journey you were validating no longer exists. + +:::::::{tab-set} + +::::::{tab-item} Project monitors +If you [set up the monitor using a Synthetics project](../../../solutions/observability/apps/create-monitors-with-project-monitors.md), you’ll delete the monitor from the project source and push changes. + +For lightweight monitors, delete the monitor from the YAML file. + +For browser monitors, delete the full journey from the JavaScript or TypeScript file. + +Then, run the [`push` command](../../../solutions/observability/apps/use-synthetics-cli.md#elastic-synthetics-push-command). The monitor associated with that journey that existed will be deleted. +:::::: + +::::::{tab-item} Synthetics UI +If you [set up the monitor using the Synthetics UI](../../../solutions/observability/apps/create-monitors-in-synthetics-app.md), you can delete a lightweight or browser monitor in the Synthetics UI: + +1. Go to **Management**. +2. Click the trash can icon next to the monitor you want to delete. +:::::: + +::::::: +Alternatively, you can temporarily disable a monitor by updating the monitor’s configuration in your journey’s code or in the Synthetics UI using the *Enabled* toggle. + + +## Implement best practices for Synthetics projects [synthetics-projects-best-practices] + +::::{important} +This is only relevant to monitors created using projects. +:::: + + +After you’ve [set up a project](../../../solutions/observability/apps/create-monitors-with-project-monitors.md), there are some best practices you can implement to manage the Synthetics project effectively. + + +### Use version control [synthetics-version-control] + +First, it’s recommended that you version control all files in Git. If your Synthetics project is not already in a version controlled directory add it and push it to your Git host. + + +### Set up recommended workflow [synthetics-workflow] + +While it can be convenient to run the `push` command directly from your workstation, especially when setting up a new Synthetics project, it is not recommended for production environments. + +Instead, we recommended that you: + +1. Develop and test changes locally. +2. Create a pull request for all config changes. +3. Have your CI service automatically verify the PR by running `npx @elastic/synthetics .` + + Elastic’s synthetics runner can output results in a few different formats, including JSON and JUnit (the standard format supported by most CI platforms). + + If any of your journeys fail, it will yield a non-zero exit code, which most CI systems pick up as a failure by default. + +4. Have a human approve the pull request. +5. Merge the pull request. +6. Have your CI service automatically deploy the change by running `npx @elastic/synthetics push` after changes are merged. + +The exact implementation details will depend on the CI system and Git host you use. You can reference the sample GitHub configuration file that is included in the `.github` directory when you create a new Synthetics project. \ No newline at end of file diff --git a/solutions/observability/apps/metrics-2.md b/solutions/observability/apps/metrics-2.md index e541795ef2..8dd23ead9e 100644 --- a/solutions/observability/apps/metrics-2.md +++ b/solutions/observability/apps/metrics-2.md @@ -4,11 +4,27 @@ mapped_urls: - https://www.elastic.co/guide/en/serverless/current/observability-apm-metrics.html --- -# Metrics +# Metrics [apm-metrics] -% What needs to be done: Align serverless/stateful +The **Metrics** overview provides APM agent-specific metrics, which lets you perform more in-depth root cause analysis investigations within the Applications UI. -% Use migrated content from existing pages that map to this page: +If you’re experiencing a problem with your service, you can use this page to attempt to find the underlying cause. For example, you might be able to correlate a high number of errors with a long transaction duration, high CPU usage, or a memory leak. -% - [ ] ./raw-migrated-files/observability-docs/observability/apm-metrics.md -% - [ ] ./raw-migrated-files/docs-content/serverless/observability-apm-metrics.md \ No newline at end of file +:::{image} ../../../images/observability-apm-metrics.png +:alt: Example view of the Metrics overview in Applications UI in Kibana +:class: screenshot +::: + +If you’re using the Java APM agent, you can view metrics for each JVM. + +:::{image} ../../../images/observability-jvm-metrics-overview.png +:alt: Example view of the Metrics overview for the Java Agent +:class: screenshot +::: + +Breaking down metrics by JVM makes it much easier to analyze the provided metrics: CPU usage, memory usage, heap or non-heap memory, thread count, garbage collection rate, and garbage collection time spent per minute. + +:::{image} ../../../images/observability-jvm-metrics.png +:alt: Example view of the Metrics overview for the Java Agent +:class: screenshot +::: \ No newline at end of file diff --git a/solutions/observability/apps/monitor-resources-on-private-networks.md b/solutions/observability/apps/monitor-resources-on-private-networks.md index 110653c925..983f681d3e 100644 --- a/solutions/observability/apps/monitor-resources-on-private-networks.md +++ b/solutions/observability/apps/monitor-resources-on-private-networks.md @@ -4,23 +4,121 @@ mapped_urls: - https://www.elastic.co/guide/en/serverless/current/observability-synthetics-private-location.html --- -# Monitor resources on private networks +# Monitor resources on private networks [synthetics-private-location] -% What needs to be done: Align serverless/stateful +To monitor resources on private networks you can either: -% Use migrated content from existing pages that map to this page: +* Allow Elastic’s global managed infrastructure to access your private endpoints. +* Use {{agent}} to create a {{private-location}}. -% - [ ] ./raw-migrated-files/observability-docs/observability/synthetics-private-location.md -% - [ ] ./raw-migrated-files/docs-content/serverless/observability-synthetics-private-location.md +{{private-location}}s via Elastic Agent require only outbound connections from your network, while allowing Elastic’s global managed infrastructure to access a private endpoint requires inbound access, thus posing an additional risk that users must assess. -% Internal links rely on the following IDs being on this page (e.g. as a heading ID, paragraph ID, etc): -$$$synthetics-private-location-fleet-agent$$$ +## Allow access to your private network [monitor-via-access-control] -$$$synthetics-private-location-connect$$$ +To give Elastic’s global managed infrastructure access to a private endpoint, use IP address filtering, HTTP authentication, or both. -$$$synthetics-private-location-add$$$ +To grant access via IP, use [this list of egress IPs](https://manifest.synthetics.elastic-cloud.com/v1/ip-ranges.json). The addresses and locations on this list may change, so automating updates to filtering rules is recommended. IP filtering alone will allow all users of Elastic’s global managed infrastructure access to your endpoints, if this is a concern consider adding additional protection via user/password authentication via a proxy like nginx. -$$$synthetics-private-location-scaling$$$ -$$$monitor-via-private-agent$$$ \ No newline at end of file +## Monitor via a private agent [monitor-via-private-agent] + +{{private-location}}s allow you to run monitors from your own premises. Before running a monitor on a {{private-location}}, you’ll need to: + +* [Set up {{fleet-server}} and {{agent}}](../../../solutions/observability/apps/monitor-resources-on-private-networks.md#synthetics-private-location-fleet-agent). +* [Connect {{fleet}} to the {{stack}}](../../../solutions/observability/apps/monitor-resources-on-private-networks.md#synthetics-private-location-connect) and enroll an {{agent}} in {{fleet}}. +* [Add a {{private-location}}](../../../solutions/observability/apps/monitor-resources-on-private-networks.md#synthetics-private-location-add) in the Synthetics UI. + +::::{important} +{{private-location}}s running through {{agent}} must have a direct connection to {{es}}. Do not configure any ingest pipelines, or output via Logstash as this will prevent Synthetics from working properly and is not [supported](../../../solutions/observability/apps/synthetics-support-matrix.md). + +:::: + + +## Set up {{fleet-server}} and {{agent}} [synthetics-private-location-fleet-agent] + +Start by setting up {{fleet-server}} and {{agent}}: + +* **Set up {{fleet-server}}**: If you are using {{ecloud}}, {{fleet-server}} will already be provided and you can skip this step. To learn more, refer to [Set up {{fleet-server}}](https://www.elastic.co/guide/en/fleet/current/fleet-server.html). +* **Create an agent policy**: For more information on agent policies and creating them, refer to [{{agent}} policy](https://www.elastic.co/guide/en/fleet/current/agent-policy.html#create-a-policy). + +::::{important} +A {{private-location}} should be set up against an agent policy that runs on a single {{agent}}. The {{agent}} must be **enrolled in Fleet** ({{private-location}}s cannot be set up using **standalone** {{agents}}). Do *not* run the same agent policy on multiple agents being used for {{private-location}}s, as you may end up with duplicate or missing tests. {{private-location}}s do not currently load balance tests across multiple {{agents}}. See [Scaling {{private-location}}s](../../../solutions/observability/apps/monitor-resources-on-private-networks.md#synthetics-private-location-scaling) for information on increasing the capacity within a {{private-location}}. + +By default {{private-location}}s are configured to allow two simultaneous browser tests and an unlimited number of lightweight checks. As a result, if more than two browser tests are assigned to a particular {{private-location}}, there may be a delay to run them. + +:::: + + +## Connect to the {{stack}} or your Observability serverless project [synthetics-private-location-connect] + +After setting up {{fleet}}, you’ll connect {{fleet}} to the {{stack}} or your Observability serverless project and enroll an {{agent}} in {{fleet}}. + +Elastic provides Docker images that you can use to run {{fleet}} and an {{agent}} more easily. For monitors running on {{private-location}}s, you *must* use the `elastic-agent-complete` Docker image to create a self-hosted {{agent}} node. The standard {{ecloud}} or self-hosted {{agent}} will not work. + +::::{important} +The `elastic-agent-complete` Docker image is the only way to have all available options that you see in the UI. + +:::: + + +To pull the Docker image run: + +```sh +docker pull docker.elastic.co/elastic-agent/elastic-agent-complete:8.16.1 +``` + +Then enroll and run an {{agent}}. You’ll need an enrollment token and the URL of the {{fleet-server}}. You can use the default enrollment token for your policy or create new policies and [enrollment tokens](https://www.elastic.co/guide/en/fleet/current/fleet-enrollment-tokens.html) as needed. + +For more information on running {{agent}} with Docker, refer to [Run {{agent}} in a container](https://www.elastic.co/guide/en/fleet/current/elastic-agent-container.html). + +```sh +docker run \ + --env FLEET_ENROLL=1 \ + --env FLEET_URL={fleet_server_host_url} \ + --env FLEET_ENROLLMENT_TOKEN={enrollment_token} \ + --cap-add=NET_RAW \ + --cap-add=SETUID \ + --rm docker.elastic.co/elastic-agent/elastic-agent-complete:8.16.1 +``` + +::::{important} +The `elastic-agent-complete` Docker image requires additional capabilities to operate correctly. Ensure `NET_RAW` and `SETUID` are enabled on the container. + +:::: + + +::::{note} +You may need to set other environment variables. Learn how in [{{agent}} environment variables guide](https://www.elastic.co/guide/en/fleet/current/agent-environment-variables.html). + +:::: + + +## Add a {{private-location}} [synthetics-private-location-add] + +When the {{agent}} is running you can add a new {{private-location}} in the UI: + +1. Find `Synthetics` in the [global search field](/explore-analyze/find-and-organize/find-apps-and-objects.md). +1. Go to **Settings**. +2. Go to the **{{private-location}}s** tab. +3. Click **Add location**. +4. Give your new location a unique *Location name* and select the *Agent policy* you created above. +5. Click **Save**. + +::::{important} +It is not currently possible to use custom CAs for synthetics browser tests in private locations without following a workaround. To learn more about the workaround, refer to the following GitHub issue: [elastic/synthetics#717](https://github.com/elastic/synthetics/issues/717). +:::: + + +## Scaling {{private-location}}s [synthetics-private-location-scaling] + +By default {{private-location}}s are configured to allow two simultaneous browser tests, and an unlimited number of lightweight checks. These limits can be set via the environment variables `SYNTHETICS_LIMIT_{{TYPE}}`, where `{{TYPE}}` is one of `BROWSER`, `HTTP`, `TCP`, and `ICMP` for the container running the {{agent}} docker image. + +It is critical to allocate enough memory and CPU capacity to handle configured limits. Start by allocating at least 2 GiB of memory and two cores per browser instance to ensure consistent performance and avoid out-of-memory errors. Then adjust as needed. Resource requirements will vary depending on workload. Much less memory is needed for lightweight monitors. Start by allocating at least 512MiB of memory and two cores for lightweight checks. Then increase allocated memory and CPU based on observed usage patterns. + +These limits are for simultaneous tests, not total tests. For example, if 60 browser tests were scheduled to run once per hour and each took 1 minute to run, that would fully occupy one execution slot. However, it is a good practice to set up execution slots with extra capacity. A good starting point would be to over-allocate by a factor of 5. In the previous example that would mean allocating 5 slots. + + +## Next steps [synthetics-private-location-next] + +Now you can add monitors to your {{private-location}} in [the Synthetics UI](../../../solutions/observability/apps/create-monitors-in-synthetics-app.md) or using the [Elastic Synthetics library’s `push` method](../../../solutions/observability/apps/create-monitors-with-project-monitors.md). \ No newline at end of file diff --git a/solutions/observability/apps/monitoring-aws-lambda-functions.md b/solutions/observability/apps/monitoring-aws-lambda-functions.md index 82a145b42f..f1a07fbb78 100644 --- a/solutions/observability/apps/monitoring-aws-lambda-functions.md +++ b/solutions/observability/apps/monitoring-aws-lambda-functions.md @@ -2,13 +2,39 @@ mapped_urls: - https://www.elastic.co/guide/en/observability/current/apm-monitoring-aws-lambda.html - https://www.elastic.co/guide/en/serverless/current/observability-apm-agents-aws-lambda-functions.html + +navigation_title: "AWS Lambda Functions" --- -# Monitoring AWS Lambda Functions +# Monitoring AWS Lambda Functions [apm-monitoring-aws-lambda] + + +Elastic APM lets you monitor your AWS Lambda functions. The natural integration of [distributed tracing](../../../solutions/observability/apps/traces.md#apm-distributed-tracing) into your AWS Lambda functions provides insights into the function’s execution and runtime behavior as well as its relationships and dependencies to other services. + +## AWS Lambda architecture [aws-lambda-arch] + +AWS Lambda uses a special execution model to provide a scalable, on-demand compute service for code execution. In particular, AWS freezes the execution environment of a lambda function when no active requests are being processed. This execution model poses additional requirements on APM in the context of AWS Lambda functions: + +1. To avoid data loss, APM data collected by APM agents needs to be flushed before the execution environment of a lambda function is frozen. +2. Flushing APM data must be fast so as not to impact the response times of lambda function requests. + +To accomplish the above, Elastic APM agents instrument AWS Lambda functions and dispatch APM data via an [AWS Lambda extension](https://docs.aws.amazon.com/lambda/latest/dg/using-extensions.md). + +Normally, during the execution of a Lambda function, there’s only a single language process running in the AWS Lambda execution environment. With an AWS Lambda extension, Lambda users run a *second* process alongside their main service/application process. + +:::{image} ../../../images/serverless-apm-agents-aws-lambda-functions-architecture.png +:alt: image showing data flow from lambda function +:class: screenshot +::: + +By using an AWS Lambda extension, Elastic APM agents can send data to a local Lambda extension process, and that process will forward data on to the managed intake service asynchronously. The Lambda extension ensures that any potential latency between the Lambda function and the managed intake service instance will not cause latency in the request flow of the Lambda function itself. + +## Setup [apm-agents-aws-lambda-functions-setup] -% What needs to be done: Align serverless/stateful +To get started with the setup of Elastic APM for your Lambda functions, checkout the language-specific guides: -% Use migrated content from existing pages that map to this page: +* [Quick Start with APM on AWS Lambda - Node.js](https://www.elastic.co/guide/en/apm/agent/nodejs/current/lambda.html) +* [Quick Start with APM on AWS Lambda - Python](https://www.elastic.co/guide/en/apm/agent/python/current/lambda-support.html) +* [Quick Start with APM on AWS Lambda - Java](https://www.elastic.co/guide/en/apm/agent/java/current/aws-lambda.html) -% - [ ] ./raw-migrated-files/observability-docs/observability/apm-monitoring-aws-lambda.md -% - [ ] ./raw-migrated-files/docs-content/serverless/observability-apm-agents-aws-lambda-functions.md \ No newline at end of file +Or, see the [architecture guide](https://www.elastic.co/guide/en/apm/lambda/current/aws-lambda-arch.html) to learn more about how the extension works, performance impacts, and more. \ No newline at end of file diff --git a/solutions/observability/apps/multi-factor-authentication-mfa-for-browser-monitors.md b/solutions/observability/apps/multi-factor-authentication-mfa-for-browser-monitors.md index 5c4dd504d0..7bf9cf229e 100644 --- a/solutions/observability/apps/multi-factor-authentication-mfa-for-browser-monitors.md +++ b/solutions/observability/apps/multi-factor-authentication-mfa-for-browser-monitors.md @@ -2,13 +2,56 @@ mapped_urls: - https://www.elastic.co/guide/en/observability/current/synthetics-mfa.html - https://www.elastic.co/guide/en/serverless/current/observability-synthetics-mfa.html + +navigation_title: "Multi-factor Authentication" --- -# Multi-factor Authentication (MFA) for browser monitors +# Multi-factor Authentication (MFA) for browser monitors [synthetics-mfa] + + +Multi-factor Authentication (MFA) adds an essential layer of security to applications login processes, protecting against unauthorized access. A very common use case in Synthetics is testing user journeys involving websites protected by MFA. + +Synthetics supports testing websites secured by Time-based One-Time Password (TOTP), a common MFA method that provides short-lived one-time tokens to enhance security. + + +## Configuring TOTP for MFA [configuring_totp_for_mfa] + +To test a browser journey that uses TOTP for MFA, first configure the Synthetics authenticator token in the target application. To do this, generate a One-Time Password (OTP) using the Synthetics CLI; refer to [`@elastic/synthetics totp <secret>`](../../../solutions/observability/apps/use-synthetics-cli.md). + +```sh +npx @elastic/synthetics totp <secret> + +// prints +OTP Token: 123456 +``` + + +## Applying the TOTP Token in Browser Journeys [applying_the_totp_token_in_browser_journeys] + +Once the Synthetics TOTP Authentication is configured in your application, you can now use the OTP token in the synthetics browser journeys using the `mfa` object imported from `@elastic/synthetics`. + +```ts +import { journey, step, mfa} from '@elastic/synthetics'; + +journey('MFA Test', ({ page, params }) => { + step('Login using TOTP token', async () => { + // login using username and pass and go to 2FA in next page + const token = mfa.totp(params.MFA_SECRET); + await page.getByPlaceholder("token-input").fill(token) + }); +}); +``` + +For monitors created in the Synthetics UI using the Script editor, the `mfa` object can be accessed as shown below: -% What needs to be done: Align serverless/stateful +```ts +step('Login using 2FA', async () => { + const token = mfa.totp(params.MFA_SECRET); + await page.getByPlaceholder("token-input").fill(token) +}); +``` -% Use migrated content from existing pages that map to this page: +::::{note} +`params.MFA_SECRET` would be the encoded secret that was used for registering the Synthetics Authentication in your web application. -% - [ ] ./raw-migrated-files/observability-docs/observability/synthetics-mfa.md -% - [ ] ./raw-migrated-files/docs-content/serverless/observability-synthetics-mfa.md \ No newline at end of file +:::: \ No newline at end of file diff --git a/solutions/observability/apps/observe-lambda-functions.md b/solutions/observability/apps/observe-lambda-functions.md index 57df9bf6c5..428b140d81 100644 --- a/solutions/observability/apps/observe-lambda-functions.md +++ b/solutions/observability/apps/observe-lambda-functions.md @@ -4,17 +4,44 @@ mapped_urls: - https://www.elastic.co/guide/en/serverless/current/observability-apm-observe-lambda-functions.html --- -# Observe Lambda functions +# Observe Lambda functions [apm-lambda] -% What needs to be done: Align serverless/stateful +Elastic APM provides performance and error monitoring for AWS Lambda functions. See how your Lambda functions relate to and depend on other services, and get insight into function execution and runtime behavior, like lambda duration, cold start rate, cold start duration, compute usage, memory usage, and more. -% Use migrated content from existing pages that map to this page: +To set up Lambda monitoring, refer to [AWS Lambda functions](/solutions/observability/apps/monitoring-aws-lambda-functions.md). -% - [ ] ./raw-migrated-files/observability-docs/observability/apm-lambda.md -% - [ ] ./raw-migrated-files/docs-content/serverless/observability-apm-observe-lambda-functions.md +:::{image} ../../../images/observability-lambda-overview.png +:alt: lambda overview +:class: screenshot +::: -% Internal links rely on the following IDs being on this page (e.g. as a heading ID, paragraph ID, etc): -$$$apm-lambda-cold-start-info$$$ +## Cold starts [apm-lambda-cold-start-info] -$$$observability-apm-observe-lambda-functions-cold-starts$$$ \ No newline at end of file +A cold start occurs when a Lambda function has not been used for a certain period of time. A lambda worker receives a request to run the function and prepares an execution environment. + +Cold starts are an unavoidable byproduct of the serverless world, but visibility into how they impact your services can help you make better decisions about factors like how much memory to allocate to a function, whether to enable provisioned concurrency, or if it’s time to consider removing a large dependency. + + +### Cold start rate [apm-lambda-cold-start-rate] + +The cold start rate (i.e. proportion of requests that experience a cold start) is displayed per service and per transaction. + +Cold start is also displayed in the trace waterfall, where you can drill-down into individual traces and see trace metadata like AWS request ID, trigger type, and trigger request ID. + + +### Latency distribution correlation [apm-lambda-cold-start-latency] + +The [latency correlations](../../../solutions/observability/apps/find-transaction-latency-failure-correlations.md) feature can be used to visualize the impact of Lambda cold starts on latency—​just select the `faas.coldstart` field. + +:::{image} ../../../images/observability-lambda-correlations.png +:alt: lambda correlations example +:class: screenshot +::: + + +## AWS Lambda function grouping [apm-lambda-service-config] + +The default APM agent configuration results in one APM service per AWS Lambda function, where the Lambda function name is the service name. + +In some use cases, it makes more sense to logically group multiple lambda functions under a single APM service. You can achieve this by setting the `ELASTIC_APM_SERVICE_NAME` environment variable on related Lambda functions to the same value. \ No newline at end of file diff --git a/solutions/observability/apps/overviews.md b/solutions/observability/apps/overviews.md index 6c2c6ac9c7..f7d97dd042 100644 --- a/solutions/observability/apps/overviews.md +++ b/solutions/observability/apps/overviews.md @@ -2,13 +2,23 @@ mapped_urls: - https://www.elastic.co/guide/en/observability/current/apm-ui.html - https://www.elastic.co/guide/en/serverless/current/observability-apm-ui-overview.html + +navigation_title: "Overviews" --- -# Overviews +# High-level overviews of application data [apm-ui] + + +For a quick, high-level overview of the health and performance of your application, start with: + +* [Services](../../../solutions/observability/apps/services.md) +* [Traces](../../../solutions/observability/apps/traces-2.md) +* [Dependencies](../../../solutions/observability/apps/dependencies.md) +* [Service Map](../../../solutions/observability/apps/service-map.md) -% What needs to be done: Align serverless/stateful +% Stateful for mobile service? -% Use migrated content from existing pages that map to this page: +View an individual service: -% - [ ] ./raw-migrated-files/observability-docs/observability/apm-ui.md -% - [ ] ./raw-migrated-files/docs-content/serverless/observability-apm-ui-overview.md \ No newline at end of file +* [Service overview](../../../solutions/observability/apps/service-overview.md) +* [Mobile service overview](../../../solutions/observability/apps/mobile-service-overview.md) diff --git a/solutions/observability/apps/reduce-storage.md b/solutions/observability/apps/reduce-storage.md index ec24b679cb..8075a39282 100644 --- a/solutions/observability/apps/reduce-storage.md +++ b/solutions/observability/apps/reduce-storage.md @@ -4,25 +4,99 @@ mapped_urls: - https://www.elastic.co/guide/en/serverless/current/observability-apm-reduce-your-data-usage.html --- -# Reduce storage +# Reduce storage [apm-reduce-apm-storage] -% What needs to be done: Align serverless/stateful +The richness and volume of APM data provides unique insights into your applications, but it can also mean higher costs and more noise when analyzing data. There are a couple strategies you can use to reduce your data usage while continuing to get the full value of APM data. -% Use migrated content from existing pages that map to this page: -% - [ ] ./raw-migrated-files/observability-docs/observability/apm-reduce-apm-storage.md -% - [ ] ./raw-migrated-files/docs-content/serverless/observability-apm-reduce-your-data-usage.md +## Reduce the sample rate [apm-reduce-sample-rate] -% Internal links rely on the following IDs being on this page (e.g. as a heading ID, paragraph ID, etc): +Distributed tracing can generate a substantial amount of data. More data can mean higher costs and more noise. Sampling aims to lower the amount of data ingested and the effort required to analyze that data. -$$$observability-apm-reduce-stacktrace$$$ +See [Transaction sampling](../../../solutions/observability/apps/transaction-sampling.md) to learn more. -$$$apm-reduce-sample-rate$$$ -$$$apm-reduce-stacktrace$$$ +## Enable span compression [enable_span_compression] -$$$apm-delete-data-in-kibana$$$ +In some cases, APM agents may collect large amounts of very similar or identical spans in a transaction. These repeated, similar spans often don’t provide added benefit, especially if they are of very short duration. Span compression takes these similar spans and compresses them into a single span-- retaining important information but reducing processing and storage overhead. -$$$apm-delete-data-query$$$ +See [Span compression](/solutions/observability/apps/spans.md#apm-spans-span-compression) to learn more. -$$$apm-delete-data-with-ilm$$$ \ No newline at end of file + +## Reduce collected stack trace information [observability-apm-reduce-stacktrace] + +Elastic APM agents collect `stacktrace` information under certain circumstances. This can be very helpful in identifying issues in your code, but it also comes with an overhead at collection time and increases your storage usage. + +Stack trace collection settings are managed in each APM agent. You can enable and disable this feature, or set specific configuration limits, like the maximum number of stacktrace frames to collect, or the minimum duration of a stacktrace to collect. + +% Stateful only after this? + +## Delete data [delete_data] + +You might want to only keep data for a defined time period. This might mean deleting old documents periodically, deleting data collected for specific services or customers, or deleting specific indices. + +Depending on your use case, you can delete data: + +* periodically with [{{ilm}}](../../../solutions/observability/apps/reduce-storage.md#apm-delete-data-with-ilm) +* [matching a query](../../../solutions/observability/apps/reduce-storage.md#apm-delete-data-query) +* with the [{{kib}} Index Management UI](../../../solutions/observability/apps/reduce-storage.md#apm-delete-data-in-kibana) + +If you want to delete data for security or privacy reasons, see [Secure data](../../../solutions/observability/apps/application-data-security.md). + + +### Delete data with {{ilm}} ({{ilm-init}}) [apm-delete-data-with-ilm] + +Index lifecycle management enables you to automate how you want to manage your indices over time. You can base actions on factors such as shard size and performance requirements. See [{{ilm-cap}}](../../../solutions/observability/apps/index-lifecycle-management.md) to learn more. + + +### Delete data matching a query [apm-delete-data-query] + +You can delete all APM documents matching a specific query with the [Delete By Query API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-delete-by-query). For example, to delete all documents with a given `service.name`, use the following request: + +```console +POST /.ds-*-apm*/_delete_by_query +{ + "query": { + "term": { + "service.name": { + "value": "old-service-name" + } + } + } +} +``` + + +### Delete data with {{kib}} Index Management [apm-delete-data-in-kibana] + +{{kib}}'s [Index Management](../../../manage-data/lifecycle/index-lifecycle-management/index-management-in-kibana.md) allows you to manage your cluster’s indices, data streams, index templates, and much more. + +To open **Index Management**, find **Stack Management*** in the main menu or use the [global search field](/explore-analyze/find-and-organize/find-apps-and-objects.md). Select ***Data Streams**. Select the data streams you want to delete, and click **Delete data streams**. + + +## Update existing data [apm-update-data] + +You might want to update documents that are already indexed. For example, if you your service name was set incorrectly. + +To do this, you can use the [Update By Query API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-update-by-query). To rename a service, send the following request: + +```console +POST /.ds-*-apm*/_update_by_query?expand_wildcards=all +{ + "query": { + "term": { + "service.name": { + "value": "current-service-name" + } + } + }, + "script": { + "source": "ctx._source.service.name = 'new-service-name'", + "lang": "painless" + } +} +``` + +::::{tip} +Remember to also change the service name in the [{{apm-agent}} configuration](https://www.elastic.co/guide/en/apm/agent/index.html). +:::: \ No newline at end of file diff --git a/solutions/observability/apps/resource-atrributes.md b/solutions/observability/apps/resource-atrributes.md index f1bec27a25..c8032670a9 100644 --- a/solutions/observability/apps/resource-atrributes.md +++ b/solutions/observability/apps/resource-atrributes.md @@ -4,11 +4,42 @@ mapped_urls: - https://www.elastic.co/guide/en/serverless/current/observability-apm-agents-opentelemetry-resource-attributes.html --- -# Resource Atrributes +# Resource attributes [apm-open-telemetry-resource-attributes] -% What needs to be done: Align serverless/stateful +A resource attribute is a key/value pair containing information about the entity producing telemetry. Resource attributes are mapped to Elastic Common Schema (ECS) fields like `service.*`, `cloud.*`, `process.*`, etc. These fields describe the service and the environment that the service runs in. -% Use migrated content from existing pages that map to this page: +The examples shown here set the Elastic (ECS) `service.environment` field for the resource, i.e. service, that is producing trace events. Note that Elastic maps the OpenTelemetry `deployment.environment` field to the ECS `service.environment` field on ingestion. -% - [ ] ./raw-migrated-files/observability-docs/observability/apm-open-telemetry-resource-attributes.md -% - [ ] ./raw-migrated-files/docs-content/serverless/observability-apm-agents-opentelemetry-resource-attributes.md \ No newline at end of file +**OpenTelemetry agent** + +Use the `OTEL_RESOURCE_ATTRIBUTES` environment variable to pass resource attributes at process invocation. + +```bash +export OTEL_RESOURCE_ATTRIBUTES=deployment.environment=production +``` + +**OpenTelemetry collector** + +Use the [resource processor](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/processor/resourceprocessor) to set or apply changes to resource attributes. + +```yaml +... +processors: + resource: + attributes: + - key: deployment.environment + action: insert + value: production +... +``` + +::::{tip} +Need to add event attributes instead? Use attributes—​not to be confused with resource attributes—​to add data to span, log, or metric events. Attributes can be added as a part of the OpenTelemetry instrumentation process or with the [attributes processor](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/processor/attributesprocessor). + +:::: + +% Stateful only after this? + +Elastic integrates with OpenTelemetry, allowing you to reuse your existing instrumentation to easily send observability data to the {{stack}}. + +For more information on how to combine Elastic and OpenTelemetry, see [OpenTelemetry integration](../../../solutions/observability/apps/use-opentelemetry-with-apm.md). \ No newline at end of file diff --git a/solutions/observability/apps/scale-architect-synthetics-deployment.md b/solutions/observability/apps/scale-architect-synthetics-deployment.md index 67a82124d7..afef68d550 100644 --- a/solutions/observability/apps/scale-architect-synthetics-deployment.md +++ b/solutions/observability/apps/scale-architect-synthetics-deployment.md @@ -2,13 +2,30 @@ mapped_urls: - https://www.elastic.co/guide/en/observability/current/synthetics-scale-and-architect.html - https://www.elastic.co/guide/en/serverless/current/observability-synthetics-scale-and-architect.html + +navigation_title: "Scale and architect a deployment" --- -# Scale and architect a Synthetics deployment +# Scale and architect a Synthetics deployment [synthetics-scale-and-architect] + +Use these advanced considerations when using the {{synthetics-app}} for large and complex use cases. + +% Stateful only for do not use... section + +## Do not use the Synthetics UI with CCS/CCR [synthetics-no-ccs-ccr] + +In complex environments it’s common to have multiple task-specific {{stack}} deployments with one centralized overview cluster using CCS or CCR to centralize {{kib}} dashboards and apps. **Do not use this pattern with the Synthetics UI**. Instead, configure your synthetic monitors directly on the {{kib}} instance where you want to view and manage them. + +You may, however, use Dashboards and the Discover feature with CCS to view `synthetics-*` indices. + +The Synthetics UI does *not* work with CCS/CCR because it would limit the rich user experience that the Synthetics UI provides. Unlike the majority of {{kib}} apps, the Synthetics UI relies heavily on state stored in {{kib}} shared objects in order to provide a rich user experience. Because {{kib}} saved objects cannot be shared via CCS/CCR, the Synthetics UI will not show any user data even if CCS/CCR is configured. + + +## Manage large numbers of Synthetic monitors with tags [synthetics-tagging] + +When managing larger numbers of synthetic monitors, use tags to keep them organized. Many of the views in the Synthetics UI are tag-aware and can group data by tag. -% What needs to be done: Align serverless/stateful -% Use migrated content from existing pages that map to this page: +## Create custom dashboards [synthetics-custom-dashboards] -% - [ ] ./raw-migrated-files/observability-docs/observability/synthetics-scale-and-architect.md -% - [ ] ./raw-migrated-files/docs-content/serverless/observability-synthetics-scale-and-architect.md \ No newline at end of file +If we don't provide a UI for your exact needs, you can use [dashboards](../../../explore-analyze/dashboards.md) to build custom visualizations. For a complete accounting of fields used by the Synthetics UI, refer to [{{heartbeat}}'s exported fields](https://www.elastic.co/guide/en/beats/heartbeat/current/exported-fields.html). \ No newline at end of file diff --git a/solutions/observability/apps/scripting-browser-monitors.md b/solutions/observability/apps/scripting-browser-monitors.md index 0486bf8698..1d0c5aa8b4 100644 --- a/solutions/observability/apps/scripting-browser-monitors.md +++ b/solutions/observability/apps/scripting-browser-monitors.md @@ -4,11 +4,21 @@ mapped_urls: - https://www.elastic.co/guide/en/serverless/current/observability-synthetics-journeys.html --- -# Scripting browser monitors +# Scripting browser monitors [synthetics-journeys] -% What needs to be done: Align serverless/stateful +Browser monitors are a type of synthetic monitor. Synthetic monitoring extends traditional end-to-end testing techniques because it allows your tests to run continuously on the cloud. With synthetic monitoring, you can assert that your application continues to work after a deployment by reusing the same journeys that you used to validate the software on your machine. -% Use migrated content from existing pages that map to this page: +You can use synthetic monitors to detect bugs caused by invalid states you couldn’t predict and didn’t write tests for. Synthetic monitors can also help you catch bugs in features that don’t get much traffic by allowing you to periodically simulate users' actions. -% - [ ] ./raw-migrated-files/observability-docs/observability/synthetics-journeys.md -% - [ ] ./raw-migrated-files/docs-content/serverless/observability-synthetics-journeys.md \ No newline at end of file +Start by learning the basics of synthetic monitoring, including how to: + +* [Write a synthetic test](../../../solutions/observability/apps/write-synthetic-test.md) +* [Test locally](../../../solutions/observability/apps/write-synthetic-test.md#synthetics-test-locally) +* [Configure individual browser monitors](../../../solutions/observability/apps/configure-individual-browser-monitors.md) +* [Work with params and secrets](../../../solutions/observability/apps/work-with-params-secrets.md) +* [Use the Synthetics Recorder](../../../solutions/observability/apps/use-synthetics-recorder.md) + +:::{image} ../../../images/observability-synthetic-monitor-lifecycle.png +:alt: Diagram of the lifecycle of a synthetic monitor: write a test +:class: screenshot +::: \ No newline at end of file diff --git a/solutions/observability/apps/service-map.md b/solutions/observability/apps/service-map.md index b6a5900cba..baf3604456 100644 --- a/solutions/observability/apps/service-map.md +++ b/solutions/observability/apps/service-map.md @@ -4,11 +4,74 @@ mapped_urls: - https://www.elastic.co/guide/en/serverless/current/observability-apm-service-map.html --- -# Service map +# Service Map [apm-service-maps] -% What needs to be done: Align serverless/stateful +A service map is a real-time visual representation of the instrumented services in your application’s architecture. It shows you how these services are connected, along with high-level metrics like average transaction duration, requests per minute, and errors per minute. If enabled, service maps also integrate with machine learning—​for real time health indicators based on anomaly detection scores. All of these features can help you quickly and visually assess your services' status and health. -% Use migrated content from existing pages that map to this page: +We currently surface two types of service maps: -% - [ ] ./raw-migrated-files/observability-docs/observability/apm-service-maps.md -% - [ ] ./raw-migrated-files/docs-content/serverless/observability-apm-service-map.md \ No newline at end of file +* **Global**: All services instrumented with APM agents and the connections between them are shown. +* **Service-specific**: Highlight connections for a selected service. + + +## How do service maps work? [service-maps-how] + +Service Maps rely on distributed traces to draw connections between services. As [distributed tracing](/solutions/observability/apps/traces.md) is enabled out-of-the-box for supported technologies, so are service maps. However, if a service isn’t instrumented, or a `traceparent` header isn’t being propagated to it, distributed tracing will not work, and the connection will not be drawn on the map. + + +## Visualize your architecture [visualize-your-architecture] + +Select the **Service Map** tab to get started. By default, all instrumented services and connections are shown. Whether you’re onboarding a new engineer, or just trying to grasp the big picture, drag things around, zoom in and out, and begin to visualize how your services are connected. + +Customize what the service map displays using either the query bar or the environment selector. The query bar enables you to use [advanced queries](../../../solutions/observability/apps/use-advanced-queries-on-application-data.md) to customize the service map based on your needs. The environment selector allows you to narrow displayed results to a specific environment. This can be useful if you have two or more services, in separate environments, but with the same name. Use the environment drop-down to only see the data you’re interested in, like `dev` or `production`. + +If there’s a specific service that interests you, select that service to highlight its connections. Click **Focus map** to refocus the map on the selected service and lock the connection highlighting. Click the **Transactions** tab to jump to the Transaction overview for the selected service. You can also use the tabs at the top of the page to easily jump to the ***Errors** or **Metrics** overview. + +:::{image} ../../../images/observability-service-maps-java.png +:alt: Example view of service maps in the Applications UI in Kibana +:class: screenshot +::: + + +## Anomaly detection with machine learning [service-map-anomaly-detection] + +You can create machine learning jobs to calculate anomaly scores on APM transaction durations within the selected service. When these jobs are active, service maps will display a color-coded anomaly indicator based on the detected anomaly score: + +| | | +| --- | --- | +| ![APM green service](../../../images/observability-green-service.png "") | Max anomaly score **≤25**. Service is healthy. | +| ![APM yellow service](../../../images/observability-yellow-service.png "") | Max anomaly score **26-74**. Anomalous activity detected. Service may be degraded. | +| ![APM red service](../../../images/observability-red-service.png "") | Max anomaly score **≥75**. Anomalous activity detected. Service is unhealthy. | + +:::{image} ../../../images/observability-apm-service-map-anomaly.png +:alt: Example view of anomaly scores on service maps in the Applications UI +:class: screenshot +::: + +If an anomaly has been detected, click **View anomalies** to view the anomaly detection metric viewer. This time series analysis will display additional details on the severity and time of the detected anomalies. + +To learn how to create a machine learning job, refer to [Integrate with machine learning](../../../solutions/observability/apps/integrate-with-machine-learning.md). + + +## Legend [service-maps-legend] + +Nodes appear on the map in one of two shapes: + +* **Circle**: Instrumented services. Interior icons are based on the language of the APM agent used. +* **Diamond**: Databases, external, and messaging. Interior icons represent the generic type, with specific icons for known entities, like Elasticsearch. Type and subtype are based on `span.type`, and `span.subtype`. + + +## Supported APM agents [service-maps-supported] + +Service Maps are supported for the following APM agent versions: + +| | | +| --- | --- | +| Go agent | ≥ v1.7.0 | +| Java agent | ≥ v1.13.0 | +| .NET agent | ≥ v1.3.0 | +| Node.js agent | ≥ v3.6.0 | +| PHP agent | ≥ v1.2.0 | +| Python agent | ≥ v5.5.0 | +| Ruby agent | ≥ v3.6.0 | +| Real User Monitoring (RUM) agent | ≥ v4.7.0 | \ No newline at end of file diff --git a/solutions/observability/apps/service-overview.md b/solutions/observability/apps/service-overview.md index 1356d45336..bc1725f6ec 100644 --- a/solutions/observability/apps/service-overview.md +++ b/solutions/observability/apps/service-overview.md @@ -4,13 +4,145 @@ mapped_urls: - https://www.elastic.co/guide/en/serverless/current/observability-apm-service-overview.html --- -# Service overview +# Service overview [apm-service-overview] -% What needs to be done: Align serverless/stateful +Selecting a non-mobile [**service**](../../../solutions/observability/apps/services.md) brings you to the **Service overview**. The **Service overview** contains a wide variety of charts and tables that provide high-level visibility into how a service is performing across your infrastructure: -% Use migrated content from existing pages that map to this page: +* Service details like service version, runtime version, framework, and APM agent name and version +* Container and orchestration information +* Cloud provider, machine type, service name, region, and availability zone +* Serverless function names and event trigger type +* Latency, throughput, and errors over time +* Service dependencies -% - [ ] ./raw-migrated-files/observability-docs/observability/apm-service-overview.md -% - [ ] ./raw-migrated-files/docs-content/serverless/observability-apm-service-overview.md +## Time series and expected bounds comparison [service-time-comparison] -$$$service-span-duration$$$ \ No newline at end of file +For insight into the health of your services, you can compare how a service performs relative to a previous time frame or to the expected bounds from the corresponding {{anomaly-job}}. For example, has latency been slowly increasing over time, did the service experience a sudden spike, is the throughput similar to what the {{ml}} job expects — enabling a comparison can provide the answer. + +:::{image} ../../../images/observability-time-series-expected-bounds-comparison.png +:alt: Time series and expected bounds comparison +:class: screenshot +::: + +Select the **Comparison** box to apply a time-based or expected bounds comparison. The time-based comparison options are based on the selected time filter range: + +| Time filter | Time comparison options | +| --- | --- | +| ≤ 24 hours | One day or one week | +| > 24 hours and ≤ 7 days | One week | +| > 7 days | An identical amount of time immediately before the selected time range | + +The expected bounds comparison is powered by [machine learning](../../../solutions/observability/apps/integrate-with-machine-learning.md) and requires anomaly detection to be enabled. + + +## Latency [service-latency] + +Response times for the service. You can filter the **Latency** chart to display the average, 95th, or 99th percentile latency times for the service. + +:::{image} ../../../images/observability-latency.png +:alt: Service latency +:class: screenshot +::: + + +## Throughput and transactions [service-throughput-transactions] + +The **Throughput** chart visualizes the average number of transactions per minute for the selected service. + +The **Transactions** table displays a list of *transaction groups* for the selected service and includes the latency, traffic, error rate, and the impact for each transaction. Transactions that share the same name are grouped, and only one entry is displayed for each group. + +By default, transaction groups are sorted by *Impact* to show the most used and slowest endpoints in your service. If there is a particular endpoint you are interested in, click **View transactions** to view a list of similar transactions on the [transactions overview](../../../solutions/observability/apps/transactions-2.md) page. + + +## Failed transaction rate and errors [service-error-rates] + +The failed transaction rate represents the percentage of failed transactions from the perspective of the selected service. It’s useful for visualizing unexpected increases, decreases, or irregular patterns in a service’s transactions. + +::::{tip} +HTTP **transactions** from the HTTP server perspective do not consider a `4xx` status code (client error) as a failure because the failure was caused by the caller, not the HTTP server. Thus, `event.outcome=success` and there will be no increase in failed transaction rate. + +HTTP **spans** from the client perspective however, are considered failures if the HTTP status code is ≥ 400. These spans will set `event.outcome=failure` and increase the failed transaction rate. + +If there is no HTTP status, both transactions and spans are considered successful unless an error is reported. + +:::: + + +The **Errors** table provides a high-level view of each error message when it first and last occurred, along with the total number of occurrences. This makes it very easy to quickly see which errors affect your services and take actions to rectify them. To do so, click **View errors**. + +:::{image} ../../../images/observability-error-rate.png +:alt: failed transaction rate and errors +:class: screenshot +::: + + +## Span types average duration and dependencies [service-span-duration] + +The **Time spent by span type** chart visualizes each span type’s average duration and helps you determine which spans could be slowing down transactions. The "app" label displayed under the chart indicates that something was happening within the application. This could signal that the APM agent does not have auto-instrumentation for whatever was happening during that time or that the time was spent in the application code and not in database or external requests. + +The **Dependencies** table displays a list of downstream services or external connections relevant to the service at the selected time range. The table displays latency, throughput, failed transaction rate, and the impact of each dependency. By default, dependencies are sorted by *Impact* to show the most used and the slowest dependency. If there is a particular dependency you are interested in, click **[View dependencies](../../../solutions/observability/apps/dependencies.md)** to learn more about it. + +% Stateful only for following note? + +::::{note} +Displaying dependencies for services instrumented with the Real User Monitoring (RUM) agent requires an agent version ≥ v5.6.3. +:::: + + +## Cold start rate [service-cold-start] + +The cold start rate chart is specific to serverless services, and displays the percentage of requests that trigger a cold start of a serverless function. A cold start occurs when a serverless function has not been used for a certain period of time. Analyzing the cold start rate can be useful for deciding how much memory to allocate to a function, or when to remove a large dependency. + +The cold start rate chart is currently supported for [AWS Lambda](../../../solutions/observability/apps/observe-lambda-functions.md#apm-lambda-cold-start-info) functions and Azure functions. + + +## Instances [service-instances] + +The **Instances** table displays a list of all the available service instances within the selected time range. Depending on how the service runs, the instance could be a host or a container. The table displays latency, throughput, failed transaction, CPU usage, and memory usage for each instance. By default, instances are sorted by *Throughput*. + +:::{image} ../../../images/observability-all-instances.png +:alt: All instances +:class: screenshot +::: + + +## Service metadata [service-metadata] + +To view metadata relating to the service agent, and if relevant, the container and cloud provider, click on each icon located at the top of the page beside the service name. + +:::{image} ../../../images/observability-metadata-icons.png +:alt: Service metadata +:class: screenshot +::: + +**Service information** + +* Service version +* Runtime name and version +* Framework name +* APM agent name and version + +**Container information** + +* Operating system +* Containerized (yes or no) +* Total number of instances +* Orchestration + +**Cloud provider information** + +* Cloud provider +* Cloud service name +* Availability zones +* Machine types +* Project ID +* Region + +**Serverless information** + +* Function name(s) +* Event trigger type + +**Alerts** + +* Recently fired alerts \ No newline at end of file diff --git a/solutions/observability/apps/services.md b/solutions/observability/apps/services.md index 2f58877bce..ebe1fbcb35 100644 --- a/solutions/observability/apps/services.md +++ b/solutions/observability/apps/services.md @@ -4,15 +4,86 @@ mapped_urls: - https://www.elastic.co/guide/en/serverless/current/observability-apm-services.html --- -# Services +# Services [apm-services] -% What needs to be done: Align serverless/stateful +The **Services** inventory provides a quick, high-level overview of the health and general performance of all instrumented services. -% Use migrated content from existing pages that map to this page: +To help surface potential issues, services are sorted by their health status: **critical** → **warning** → ***healthy** → **unknown**. Health status is powered by [machine learning](../../../solutions/observability/apps/integrate-with-machine-learning.md) and requires anomaly detection to be enabled. -% - [ ] ./raw-migrated-files/observability-docs/observability/apm-services.md -% - [ ] ./raw-migrated-files/docs-content/serverless/observability-apm-services.md +In addition to health status, active alerts for each service are prominently displayed in the service inventory table. Selecting an active alert badge brings you to the [**Alerts**](../../../solutions/observability/apps/create-apm-rules-alerts.md) tab where you can learn more about the active alert and take action. -% Internal links rely on the following IDs being on this page (e.g. as a heading ID, paragraph ID, etc): +:::{image} ../../../images/observability-apm-services-overview.png +:alt: Example view of services table the Applications UI in Kibana +:class: screenshot +::: -$$$service-groups$$$ \ No newline at end of file +% Stateful only for the following tip? + +::::{tip} +Want to monitor service logs without instrumenting all your services? Learn about our [Inventory](../../../solutions/observability/apps/inventory.md). +:::: + + +## Service groups [service-groups] + +::::{admonition} Required role +:class: note + +For Observability serverless projects, the **Editor** role or higher is required to create and manage service groups. To learn more, refer to [Assign user roles and privileges](../../../deploy-manage/users-roles/cloud-organization/user-roles.md#general-assign-user-roles). + +:::: + + +::::{admonition} Service grouping is in beta +:class: important + +The Service grouping functionality is in beta and is subject to change. The design and code is less mature than official generally available features and is being provided as-is with no warranties. + +:::: + + +Group services together to build meaningful views that remove noise, simplify investigations across services, and combine related alerts. + +:::{image} ../../../images/observability-apm-service-group.png +:alt: Example view of service group in the Applications UI in Kibana +:class: screenshot +::: + +To create a service group: + +::::{tab-set} +:group: stack-serverless + +:::{tab-item} Elastic Stack v9 +:sync: stack + +1. To open **Service inventory**, find **Applications** in the main menu or use the [global search field](/explore-analyze/find-and-organize/find-apps-and-objects.md). +2. Switch to **Service groups**. +3. Click **Create group**. +4. Specify a name, color, and description. +5. Click **Select services**. +6. Specify a [{{kib}} Query Language (KQL)](../../../explore-analyze/query-filter/languages/kql.md) query to filter services by one or more of the following dimensions: `agent.name`, `service.name`, `service.language.name`, `service.environment`, `labels.<xyz>`. Services that match the query within the last 24 hours will be assigned to the group. + +::: + +:::{tab-item} Serverless +:sync: serverless + +1. In your {{obs-serverless}} project, go to **Applications** → **Service Inventory**. +2. Switch to **Service groups**. +3. Click **Create group**. +4. Specify a name, color, and description. +5. Click **Select services**. +6. Specify a [Kibana Query Language (KQL)](../../../explore-analyze/query-filter/languages/kql.md) query to filter services by one or more of the following dimensions: `agent.name`, `service.name`, `service.language.name`, `service.environment`, `labels.<xyz>`. Services that match the query within the last 24 hours will be assigned to the group. + +::: + +:::: + + +### Examples [apm-services-examples] + +Not sure where to get started? Here are some sample queries you can build from: + +* **Group services by environment**: To group "production" services, use `service.environment : "production"`. +* **Group services by name**: To group all services that end in "beat", use `service.name : *beat`. This will match services named "Auditbeat", "Heartbeat", "Filebeat", and so on. \ No newline at end of file diff --git a/solutions/observability/apps/spans.md b/solutions/observability/apps/spans.md index 03f3113c04..6327dca49d 100644 --- a/solutions/observability/apps/spans.md +++ b/solutions/observability/apps/spans.md @@ -4,17 +4,479 @@ mapped_urls: - https://www.elastic.co/guide/en/serverless/current/observability-apm-compress-spans.html --- -# Spans +# Spans [apm-data-model-spans] -% What needs to be done: Align serverless/stateful +% Stateful only until Span Compressions -% Scope notes: Serverless only has span compression. +**Spans** contain information about the execution of a specific code path. They measure from the start to the end of an activity, and they can have a parent/child relationship with other spans. -% Use migrated content from existing pages that map to this page: +Agents automatically instrument a variety of libraries to capture these spans from within your application, but you can also use the Agent API for custom instrumentation of specific code paths. -% - [ ] ./raw-migrated-files/observability-docs/observability/apm-data-model-spans.md -% - [ ] ./raw-migrated-files/docs-content/serverless/observability-apm-compress-spans.md +Among other things, spans can contain: -% Internal links rely on the following IDs being on this page (e.g. as a heading ID, paragraph ID, etc): +* A `transaction.id` attribute that refers to its parent [transaction](../../../solutions/observability/apps/transactions.md). +* A `parent.id` attribute that refers to its parent span or transaction. +* Its start time and duration. +* A `name`, `type`, `subtype`, and `action`—see the [span name/type alignment](https://docs.google.com/spreadsheets/d/1SmWeX5AeqUcayrArUauS_CxGgsjwRgMYH4ZY8yQsMhQ/edit#gid=644582948) sheet for span name patterns and examples by {{apm-agent}}. In addition, some APM agents test against a public [span type/subtype spec](https://github.com/elastic/apm/blob/main/tests/agents/json-specs/span_types.json). +* An optional `stack trace`. Stack traces consist of stack frames, which represent a function call on the call stack. They include attributes like function name, file name and path, line number, etc. -$$$apm-spans-span-compression$$$ \ No newline at end of file +::::{tip} +Most agents limit keyword fields, like `span.id`, to 1024 characters, and non-keyword fields, like `span.start.us`, to 10,000 characters. +:::: + + + +## Dropped spans [apm-data-model-dropped-spans] + +For performance reasons, APM agents can choose to sample or omit spans purposefully. This can be useful in preventing edge cases, like long-running transactions with over 100 spans, that would otherwise overload both the Agent and the APM Server. When this occurs, the Applications UI will display the number of spans dropped. + +To configure the number of spans recorded per transaction, see the relevant Agent documentation: + +* Android: *Not yet supported* +* Go: [`ELASTIC_APM_TRANSACTION_MAX_SPANS`](https://www.elastic.co/guide/en/apm/agent/go/current/configuration.html#config-transaction-max-spans) +* iOS: *Not yet supported* +* Java: [`transaction_max_spans`](https://www.elastic.co/guide/en/apm/agent/java/current/config-core.html#config-transaction-max-spans) +* .NET: [`TransactionMaxSpans`](https://www.elastic.co/guide/en/apm/agent/dotnet/current/config-core.html#config-transaction-max-spans) +* Node.js: [`transactionMaxSpans`](https://www.elastic.co/guide/en/apm/agent/nodejs/current/configuration.html#transaction-max-spans) +* PHP: [`transaction_max_spans`](https://www.elastic.co/guide/en/apm/agent/php/current/configuration-reference.html#config-transaction-max-spans) +* Python: [`transaction_max_spans`](https://www.elastic.co/guide/en/apm/agent/python/current/configuration.html#config-transaction-max-spans) +* Ruby: [`transaction_max_spans`](https://www.elastic.co/guide/en/apm/agent/ruby/current/configuration.html#config-transaction-max-spans) + + +## Missing spans [apm-data-model-missing-spans] + +Agents stream spans to the APM Server separately from their transactions. Because of this, unforeseen errors may cause spans to go missing. Agents know how many spans a transaction should have; if the number of expected spans does not equal the number of spans received by the APM Server, the Applications UI will calculate the difference and display a message. + + +## Data streams [_data_streams] + +Spans are stored with transactions in the following data streams: + +* Application traces: `traces-apm-<namespace>` +* RUM and iOS agent application traces: `traces-apm.rum-<namespace>` + +See [Data streams](../../../solutions/observability/apps/data-streams.md) to learn more. + + +## Example span document [_example_span_document] + +This example shows what span documents can look like when indexed in {{es}}. + +::::{dropdown} Expand {{es}} document +```json +[ + { + "@timestamp": "2017-05-30T18:53:27.154Z", + "agent": { + "name": "elastic-node", + "version": "3.14.0" + }, + "ecs": { + "version": "1.12.0" + }, + "event": { + "outcome": "unknown" + }, + "http": { + "request": { + "method": "GET" + }, + "response": { + "status_code": 200 + } + }, + "labels": { + "span_tag": "something" + }, + "observer": { + "hostname": "ix.lan", + "type": "apm-server", + "version": "8.0.0" + }, + "parent": { + "id": "945254c567a5417e" + }, + "processor": { + "event": "span", + "name": "transaction" + }, + "service": { + "environment": "staging", + "name": "1234_service-12a3" + }, + "span": { + "action": "query", + "db": { + "instance": "customers", + "statement": "SELECT * FROM product_types WHERE user_id=?", + "type": "sql", + "user": { + "name": "readonly_user" + } + }, + "duration": { + "us": 3781 + }, + "http": { + "method": "GET", + "response": { + "status_code": 200 + } + }, + "http.url.original": "http://localhost:8000", + "id": "0aaaaaaaaaaaaaaa", + "name": "SELECT FROM product_types", + "stacktrace": [ + { + "abs_path": "net.js", + "context": { + "post": [ + " ins.currentTransaction = prev", + " return result", + "}" + ], + "pre": [ + " var trans = this.currentTransaction", + "" + ] + }, + "exclude_from_grouping": false, + "filename": "net.js", + "function": "onread", + "library_frame": true, + "line": { + "column": 4, + "context": "line3", + "number": 547 + }, + "module": "some module", + "vars": { + "key": "value" + } + }, + { + "exclude_from_grouping": false, + "filename": "my2file.js", + "line": { + "number": 10 + } + } + ], + "start": { + "us": 2830 + }, + "subtype": "postgresql", + "sync": false, + "type": "db" + }, + "timestamp": { + "us": 1496170407154000 + }, + "trace": { + "id": "945254c567a5417eaaaaaaaaaaaaaaaa" + }, + "transaction": { + "id": "945254c567a5417e" + }, + "url": { + "original": "http://localhost:8000" + } + }, + { + "@timestamp": "2017-05-30T18:53:42.281Z", + "agent": { + "name": "js-base", + "version": "1.3" + }, + "destination": { + "address": "0:0::0:1", + "ip": "0:0::0:1", + "port": 5432 + }, + "ecs": { + "version": "1.12.0" + }, + "event": { + "outcome": "unknown" + }, + "observer": { + "ephemeral_id": "2f13d8fa-83cd-4356-8123-aabfb47a1808", + "hostname": "goat", + "id": "17ad47dd-5671-4c89-979f-ef4533565ba2", + "type": "apm-server", + "version": "8.0.0" + }, + "parent": { + "id": "85925e55b43f4342" + }, + "processor": { + "event": "span", + "name": "transaction" + }, + "service": { + "environment": "staging", + "name": "serviceabc" + }, + "span": { + "action": "query.custom", + "db": { + "instance": "customers", + "statement": "SELECT * FROM product_types WHERE user_id=?", + "type": "sql", + "user": { + "name": "readonly_user" + } + }, + "destination": { + "service": { + "name": "postgresql", + "resource": "postgresql", + "type": "db" + } + }, + "duration": { + "us": 3781 + }, + "id": "15aaaaaaaaaaaaaa", + "name": "SELECT FROM product_types", + "start": { + "us": 2830 + }, + "subtype": "postgresql", + "type": "db.postgresql.query" + }, + "timestamp": { + "us": 1496170422281000 + }, + "trace": { + "id": "85925e55b43f4342aaaaaaaaaaaaaaaa" + }, + "transaction": { + "id": "85925e55b43f4342" + } + }, + { + "@timestamp": "2017-05-30T18:53:27.154Z", + "agent": { + "name": "elastic-node", + "version": "3.14.0" + }, + "ecs": { + "version": "1.12.0" + }, + "event": { + "outcome": "unknown" + }, + "observer": { + "ephemeral_id": "2f13d8fa-83cd-4356-8123-aabfb47a1808", + "hostname": "goat", + "id": "17ad47dd-5671-4c89-979f-ef4533565ba2", + "type": "apm-server", + "version": "8.0.0" + }, + "parent": { + "id": "945254c567a5417e" + }, + "processor": { + "event": "span", + "name": "transaction" + }, + "service": { + "environment": "staging", + "name": "1234_service-12a3" + }, + "span": { + "duration": { + "us": 32592 + }, + "id": "1aaaaaaaaaaaaaaa", + "name": "GET /api/types", + "start": { + "us": 0 + }, + "subtype": "external", + "type": "request" + }, + "timestamp": { + "us": 1496170407154000 + }, + "trace": { + "id": "945254c567a5417eaaaaaaaaaaaaaaaa" + }, + "transaction": { + "id": "945254c567a5417e" + } + }, + { + "@timestamp": "2017-05-30T18:53:27.154Z", + "agent": { + "name": "elastic-node", + "version": "3.14.0" + }, + "ecs": { + "version": "1.12.0" + }, + "event": { + "outcome": "unknown" + }, + "observer": { + "ephemeral_id": "2f13d8fa-83cd-4356-8123-aabfb47a1808", + "hostname": "goat", + "id": "17ad47dd-5671-4c89-979f-ef4533565ba2", + "type": "apm-server", + "version": "8.0.0" + }, + "parent": { + "id": "945254c567a5417e" + }, + "processor": { + "event": "span", + "name": "transaction" + }, + "service": { + "environment": "staging", + "name": "1234_service-12a3" + }, + "span": { + "action": "post", + "duration": { + "us": 3564 + }, + "id": "2aaaaaaaaaaaaaaa", + "name": "GET /api/types", + "start": { + "us": 1845 + }, + "subtype": "http", + "type": "request" + }, + "timestamp": { + "us": 1496170407154000 + }, + "trace": { + "id": "945254c567a5417eaaaaaaaaaaaaaaaa" + }, + "transaction": { + "id": "945254c567a5417e" + } + }, + { + "@timestamp": "2017-05-30T18:53:27.154Z", + "agent": { + "name": "elastic-node", + "version": "3.14.0" + }, + "child": { + "id": [ + "4aaaaaaaaaaaaaaa" + ] + }, + "ecs": { + "version": "1.12.0" + }, + "event": { + "outcome": "unknown" + }, + "observer": { + "ephemeral_id": "2f13d8fa-83cd-4356-8123-aabfb47a1808", + "hostname": "goat", + "id": "17ad47dd-5671-4c89-979f-ef4533565ba2", + "type": "apm-server", + "version": "8.0.0" + }, + "parent": { + "id": "945254c567a5417e" + }, + "processor": { + "event": "span", + "name": "transaction" + }, + "service": { + "environment": "staging", + "name": "1234_service-12a3" + }, + "span": { + "duration": { + "us": 13980 + }, + "id": "3aaaaaaaaaaaaaaa", + "name": "GET /api/types", + "start": { + "us": 0 + }, + "type": "request" + }, + "timestamp": { + "us": 1496170407154000 + }, + "trace": { + "id": "945254c567a5417eaaaaaaaaaaaaaaaa" + }, + "transaction": { + "id": "945254c567a5417e" + } + } +] +``` + +:::: + + +## Span compression [apm-spans-span-compression] + +In some cases, APM agents may collect large amounts of very similar or identical spans in a transaction. For example, this can happen if spans are captured inside a loop or in unoptimized SQL queries that use multiple queries instead of joins to fetch related data. + +In such cases, the upper limit of spans per transaction (by default, 500 spans) can be reached quickly, causing the agent to stop capturing potentially more relevant spans for a given transaction. + +Capturing similar or identical spans often isn’t helpful, especially if they are of very short duration. They can also clutter the UI, and cause processing and storage overhead. + +To address this problem, APM agents can compress similar spans into a single span. The compressed span retains most of the original span information, including the overall duration and number of spans it represents. + +Regardless of the compression strategy, a span is eligible for compression if: + +* It has not propagated its trace context. +* It is an *exit* span (such as database query spans). +* Its outcome is not `"failure"`. + + +### Compression strategies [apm-span-compression-strategy] + +The {{apm-agent}} selects between two strategies to decide if adjacent spans can be compressed. In both strategies, only one previous span needs to be kept in memory. This ensures that the agent doesn’t require large amounts of memory to enable span compression. + + +#### Same-Kind strategy [apm-span-compression-same] + +The agent uses the same-kind strategy if two adjacent spans have the same: + +* span type +* span subtype +* `destination.service.resource` (e.g. database name) + +#### Exact-Match strategy [apm-span-compression-exact] + +The agent uses the exact-match strategy if two adjacent spans have the same: + +* span name +* span type +* span subtype +* `destination.service.resource` (e.g. database name) + + +### Settings [apm-span-compression-settings] + +You can specify the maximum span duration in the agent’s configuration settings. Spans with a duration longer than the specified value will not be compressed. + +For the "Same-Kind" strategy, the default maximum span duration is 0 milliseconds, which means that the "Same-Kind" strategy is disabled by default. For the "Exact-Match" strategy, the default limit is 50 milliseconds. + + +### Agent support [apm-span-compression-support] + +Support for span compression is available in the following agents and can be configured using the options listed below: + +| Agent | Same-kind config | Exact-match config | +| --- | --- | --- | +| **Go agent** | [`ELASTIC_APM_SPAN_COMPRESSION_SAME_KIND_MAX_DURATION`](https://www.elastic.co/guide/en/apm/agent/go/current/configuration.html#config-span-compression-exact-match-duration) | +| **Java agent** | [`span_compression_same_kind_max_duration`](https://www.elastic.co/guide/en/apm/agent/java/current/config-huge-traces.html#config-span-compression-same-kind-max-duration) | [`span_compression_exact_match_max_duration`](https://www.elastic.co/guide/en/apm/agent/java/current/config-huge-traces.html#config-span-compression-exact-match-max-duration) | +| **.NET agent** | [`SpanCompressionSameKindMaxDuration`](https://www.elastic.co/guide/en/apm/agent/dotnet/current/config-core.html#config-span-compression-exact-match-max-duration) | +| **Node.js agent** | [`spanCompressionSameKindMaxDuration`](https://www.elastic.co/guide/en/apm/agent/nodejs/current/configuration.html#span-compression-exact-match-max-duration) | +| **Python agent** | [`span_compression_same_kind_max_duration`](https://www.elastic.co/guide/en/apm/agent/python/current/configuration.html#config-span-compression-exact-match-max_duration) | \ No newline at end of file diff --git a/solutions/observability/apps/synthetic-monitoring.md b/solutions/observability/apps/synthetic-monitoring.md index 635b0b43b9..61a853ee81 100644 --- a/solutions/observability/apps/synthetic-monitoring.md +++ b/solutions/observability/apps/synthetic-monitoring.md @@ -4,21 +4,46 @@ mapped_urls: - https://www.elastic.co/guide/en/serverless/current/observability-monitor-synthetics.html --- -# Synthetic monitoring +# Synthetic monitoring [monitor-uptime-synthetics] -% What needs to be done: Align serverless/stateful +::::{note} +The Synthetics UI is for viewing result data from monitors created and managed directly in the [Synthetics UI](../../../solutions/observability/apps/create-monitors-in-synthetics-app.md) or managed externally using a [Synthetics project](../../../solutions/observability/apps/create-monitors-with-project-monitors.md). This can include both lightweight and browser-based monitors, and can include monitors running from either Elastic’s global managed testing infrastructure or from [{{private-location}}s](../../../solutions/observability/apps/monitor-resources-on-private-networks.md). -% Use migrated content from existing pages that map to this page: +:::: -% - [ ] ./raw-migrated-files/observability-docs/observability/monitor-uptime-synthetics.md -% - [ ] ./raw-migrated-files/docs-content/serverless/observability-monitor-synthetics.md -% Internal links rely on the following IDs being on this page (e.g. as a heading ID, paragraph ID, etc): +Synthetics periodically checks the status of your services and applications. Monitor the availability of network endpoints and services using the following types of monitors: -$$$monitoring-uptime$$$ +* [Lightweight HTTP/S, TCP, and ICMP monitors](../../../solutions/observability/apps/synthetic-monitoring.md#monitoring-uptime) +* [Browser monitors](../../../solutions/observability/apps/synthetic-monitoring.md#monitoring-synthetics) -$$$monitoring-synthetics$$$ +:::{image} ../../../images/observability-synthetics-monitor-page.png +:alt: {{synthetics-app}} in {{kib}} +:class: screenshot +::: -$$$observability-monitor-synthetics-browser-monitors$$$ -$$$observability-monitor-synthetics-lightweight-https-tcp-and-icmp-monitors$$$ \ No newline at end of file +## Lightweight HTTP/S, TCP, and ICMP monitors [monitoring-uptime] + +You can monitor the status of network endpoints using the following lightweight checks: + +| | | +| --- | --- | +| **HTTP monitor** | Monitor your website. The HTTP monitor checks to make sure specific endpoints return the correctstatus code and display the correct text. | +| **ICMP monitor** | Check the availability of your hosts. The ICMP monitor uses ICMP (v4 and v6) EchoRequests to check the network reachability of the hosts you are pinging. This will tell you whether thehost is available and connected to the network, but doesn’t tell you if a service on the host is running ornot. | +| **TCP monitor** | Monitor the services running on your hosts. The TCP monitor checks individual portsto make sure the service is accessible and running. | + +To set up your first monitor, refer to [Get started](../../../solutions/observability/apps/get-started.md). + + +## Browser monitors [monitoring-synthetics] + +Real browser synthetic monitoring enables you to test critical actions and requests that an end-user would make on your site at predefined intervals and in a controlled environment. Synthetic monitoring extends traditional end-to-end testing techniques because it allows your tests to run continuously on the cloud. The result is rich, consistent, and repeatable data that you can trend and alert on. + +For example, you can test popular user journeys, like logging in, adding items to a cart, and checking out — actions that need to work for your users consistently. + +You can run an automated Synthetics project on a real Chromium browser and view each synthetic monitoring journey in your Observability project side-by-side with your other monitors. + +Alerting helps you detect degraded performance or broken actions before your users do. By receiving alerts early, you can fix issues before they impact your bottom line or customer experience. + +To set up your first monitor, refer to [Get started](../../../solutions/observability/apps/get-started.md). \ No newline at end of file diff --git a/solutions/observability/apps/synthetics-encryption-security.md b/solutions/observability/apps/synthetics-encryption-security.md index 634f6db6ee..bf3c1e064b 100644 --- a/solutions/observability/apps/synthetics-encryption-security.md +++ b/solutions/observability/apps/synthetics-encryption-security.md @@ -4,11 +4,23 @@ mapped_urls: - https://www.elastic.co/guide/en/serverless/current/observability-synthetics-security-encryption.html --- -# Synthetics Encryption and Security +# Synthetics Encryption and Security [synthetics-security-encryption] -% What needs to be done: Align serverless/stateful +Elastic Synthetics was designed with security in mind encrypting both persisted and transmitted data. This page catalogs the points within Elastic Synthetics where data is either stored or transmitted in an encrypted fashion. -% Use migrated content from existing pages that map to this page: -% - [ ] ./raw-migrated-files/observability-docs/observability/synthetics-security-encryption.md -% - [ ] ./raw-migrated-files/docs-content/serverless/observability-synthetics-security-encryption.md \ No newline at end of file +## Synthetics UI (Kibana App) [_synthetics_ui_kibana_app] + +Data is stored in [Kibana Secure Saved Objects](../../../deploy-manage/security/secure-saved-objects.md), with sensitive fields encrypted. These fields include your script source, params, and global params. + + +## Synthetics Service [_synthetics_service] + +The Global Elastic Synthetics Service performs all communication of sensitive data (both internally, and with Kibana) over encrypted connections and encrypts all data persisted to disk as well. + + +## Synthetics Private Locations [_synthetics_private_locations] + +In Kibana configuration for private locations is stored in two places, Synthetics saved objects which always encrypt sensitive fields using [Kibana Secure Saved Objects](../../../deploy-manage/security/secure-saved-objects.md) and also in Fleet, which uses unencrypted saved objects restricted by user permissions. For Elastic Cloud customers all data is secured on disk regardless of whether additional saved object encryption is present. See our [Cloud Security Statement](https://www.elastic.co/cloud/security) for more information. We recommend that self-managed customers encrypt disks for their Elasticsearch instances if this is a concern. + +All data is encrypted in transit. See [Elastic Agent configuration encryption](https://www.elastic.co/guide/en/fleet/current/_elastic_agent_configuration_encryption.html) for more details. \ No newline at end of file diff --git a/solutions/observability/apps/trace-sample-timeline.md b/solutions/observability/apps/trace-sample-timeline.md index 500606e5a6..9d570c9ad4 100644 --- a/solutions/observability/apps/trace-sample-timeline.md +++ b/solutions/observability/apps/trace-sample-timeline.md @@ -4,17 +4,62 @@ mapped_urls: - https://www.elastic.co/guide/en/serverless/current/observability-apm-trace-sample-timeline.html --- -# Trace sample timeline +# Trace sample timeline [apm-spans] -% What needs to be done: Align serverless/stateful +The trace sample timeline visualization is a high-level view of what your application was doing while it was trying to respond to a request. This makes it useful for visualizing where a selected transaction spent most of its time. -% Use migrated content from existing pages that map to this page: +:::{image} ../../../images/observability-apm-transaction-sample.png +:alt: Example of distributed trace colors in the Applications UI +:class: screenshot +::: -% - [ ] ./raw-migrated-files/observability-docs/observability/apm-spans.md -% - [ ] ./raw-migrated-files/docs-content/serverless/observability-apm-trace-sample-timeline.md +View a span in detail by clicking on it in the timeline waterfall. For example, when you click on an SQL Select database query, the information displayed includes the actual SQL that was executed, how long it took, and the percentage of the trace’s total time. You also get a stack trace, which shows the SQL query in your code. Finally, APM knows which files are your code and which are just modules or libraries that you’ve installed. These library frames will be minimized by default in order to show you the most relevant stack trace. -% Internal links rely on the following IDs being on this page (e.g. as a heading ID, paragraph ID, etc): +::::{tip} +A [span](/solutions/observability/apps/spans.md) is the duration of a single event. Spans are automatically captured by APM agents, and you can also define custom spans. Each span has a type and is defined by a different color in the timeline/waterfall visualization. +:::: -$$$distributed-tracing$$$ -$$$observability-apm-trace-sample-timeline-distributed-tracing$$$ \ No newline at end of file +:::{image} ../../../images/observability-apm-span-detail.png +:alt: Example view of a span detail in the Applications UI +:class: screenshot +::: + + +## Investigate [trace-sample-investigate] + +The trace sample timeline features an **Investigate** button which provides a quick way to jump to other areas of the Elastic Observability UI while maintaining the context of the currently selected trace sample. For example, quickly view: + +* logs and metrics for the selected pod +* logs and metrics for the selected host +* trace logs for the selected `trace.id` +* uptime status of the selected domain +* the [service map](../../../solutions/observability/apps/service-map.md) filtered by the selected trace +* the selected transaction in **Discover** +* your [custom links](../../../solutions/observability/apps/create-custom-links.md) + + +## Distributed tracing [distributed-tracing] + +When a trace travels through multiple services it is known as a *distributed trace*. In APM, the colors in a distributed trace represent different services and are listed in the order they occur. + +:::{image} ../../../images/observability-apm-services-trace.png +:alt: Example of distributed trace colors in the Applications UI +:class: screenshot +::: + +As application architectures are shifting from monolithic to more distributed, service-based architectures, distributed tracing has become a crucial feature of modern application performance monitoring. It allows you to trace requests through your service architecture automatically, and visualize those traces in one single view in the Applications UI. From initial web requests to your front-end service, to queries made to your back-end services, this makes finding possible bottlenecks throughout your application much easier and faster. + +:::{image} ../../../images/observability-apm-distributed-tracing.png +:alt: Example view of the distributed tracing in the Applications UI +:class: screenshot +::: + +Don’t forget; by definition, a distributed trace includes more than one transaction. When viewing distributed traces in the timeline waterfall, you’ll see this icon: ![APM icon](../../../images/observability-transaction-icon.png ""), which indicates the next transaction in the trace. For easier problem isolation, transactions can be collapsed in the waterfall by clicking the icon to the left of the transactions. Transactions can also be expanded and viewed in detail by clicking on them. + +After exploring these traces, you can return to the full trace by clicking **View full trace**. + +::::{tip} +Distributed tracing is supported by all APM agents, and there’s no additional configuration needed. + +:::: diff --git a/solutions/observability/apps/traces.md b/solutions/observability/apps/traces.md index 3b15e0055c..b85aae6f86 100644 --- a/solutions/observability/apps/traces.md +++ b/solutions/observability/apps/traces.md @@ -4,17 +4,429 @@ mapped_urls: - https://www.elastic.co/guide/en/observability/current/apm-data-model-traces.html --- -# Traces +# Traces [apm-data-model-traces] -% What needs to be done: Lift-and-shift +A trace is a group of [transactions](../../../solutions/observability/apps/transactions.md) and [spans](../../../solutions/observability/apps/spans.md) with a common root. Each trace tracks the entirety of a single request. It describes the individual operations and their causality that ensue from a single logical operation. -% Scope notes: Serverless only has distributed tracing. -% Use migrated content from existing pages that map to this page: +## Distributed tracing [apm-distributed-tracing] -% - [ ] ./raw-migrated-files/docs-content/serverless/observability-apm-distributed-tracing.md -% - [ ] ./raw-migrated-files/observability-docs/observability/apm-data-model-traces.md +When a trace travels through multiple services, as is common in a microservice architecture, it is known as a **distributed trace**. A distributed trace comprises operations across multiple distributed components, crossing process, network, and security boundaries. -% Internal links rely on the following IDs being on this page (e.g. as a heading ID, paragraph ID, etc): +### Why is distributed tracing important? [apm-why-distributed-tracing] -$$$apm-distributed-tracing$$$ \ No newline at end of file +Distributed tracing enables you to analyze performance throughout your microservice architecture by tracing the entirety of a request — from the initial web request on your front-end service all the way to database queries made on your back-end services. + +Tracking requests as they propagate through your services provides an end-to-end picture of where your application is spending time, where errors are occurring, and where bottlenecks are forming. Distributed tracing eliminates individual service’s data silos and reveals what’s happening outside of service borders. + +For supported technologies, distributed tracing works out-of-the-box, with no additional configuration required. + + +### How distributed tracing works [apm-how-distributed-tracing] + +Distributed tracing works by injecting a custom `traceparent` HTTP header into outgoing requests. This header includes information, like `trace-id`, which is used to identify the current trace, and `parent-id`, which is used to identify the parent of the current span on incoming requests or the current span on an outgoing request. + +When a service is working on a request, it checks for the existence of this HTTP header. If it’s missing, the service starts a new trace. If it exists, the service ensures the current action is added as a child of the existing trace, and continues to propagate the trace. + + +### Trace propagation examples [apm-trace-propagation] + +In this example, Elastic’s Ruby agent communicates with Elastic’s Java agent. Both support the `traceparent` header, and trace data is successfully propagated. + +:::{image} ../../../images/observability-dt-trace-ex1.png +:alt: How traceparent propagation works +::: + +In this example, Elastic’s Ruby agent communicates with OpenTelemetry’s Java agent. Both support the `traceparent` header, and trace data is successfully propagated. + +:::{image} ../../../images/observability-dt-trace-ex2.png +:alt: How traceparent propagation works +::: + +In this example, the trace meets a piece of middleware that doesn’t propagate the `traceparent` header. The distributed trace ends and any further communication will result in a new trace. + +:::{image} ../../../images/observability-dt-trace-ex3.png +:alt: How traceparent propagation works +::: + + +### W3C Trace Context specification [apm-w3c-tracecontext-spec] + +All Elastic agents now support the official W3C Trace Context specification and `traceparent` header. See the table below for the minimum required agent version: + +% Stateful only for RUM agent in following table? + +| Agent name | Agent Version | +| --- | --- | +| **Go Agent** | ≥`1.6` | +| **Java Agent** | ≥`1.14` | +| **.NET Agent** | ≥`1.3` | +| **Node.js Agent** | ≥`3.4` | +| **PHP Agent** | ≥`1.0` | +| **Python Agent** | ≥`5.4` | +| **Ruby Agent** | ≥`3.5` | +| **RUM Agent** | ≥`5.0` | + +::::{note} +Older Elastic agents use a unique `elastic-apm-traceparent` header. For backward-compatibility purposes, new versions of Elastic agents still support this header. +:::: + + + +### Visualize distributed tracing [apm-visualize-distributed-tracing] + +APM's timeline visualization provides a visual deep-dive into each of your application’s traces: + +:::{image} ../../../images/observability-apm-distributed-tracing.png +:alt: Distributed tracing in the Applications UI +:class: screenshot +::: + + +### Manual distributed tracing [apm-manual-distributed-tracing] + +Elastic agents automatically propagate distributed tracing context for supported technologies. If your service communicates over a different, unsupported protocol, you can manually propagate distributed tracing context from a sending service to a receiving service with each agent’s API. + + +#### Add the `traceparent` header to outgoing requests [apm-distributed-tracing-outgoing] + +Sending services must add the `traceparent` header to outgoing requests. + +:::::::{tab-set} + +::::::{tab-item} Android +*Not applicable.* +:::::: + +::::::{tab-item} Go +1. Start a transaction with [`StartTransaction`](https://www.elastic.co/guide/en/apm/agent/go/current/api.html#tracer-api-start-transaction) or a span with [`StartSpan`](https://www.elastic.co/guide/en/apm/agent/go/current/api.html#transaction-start-span). +2. Get the active `TraceContext`. +3. Send the `TraceContext` to the receiving service. + +Example: + +```go +transaction := apm.DefaultTracer().StartTransaction("GET /", "request") <1> +traceContext := transaction.TraceContext() <2> + +// Send TraceContext to receiving service +traceparent := apmhttp.FormatTraceparentHeader(traceContext) <3> +tracestate := traceContext.State.String() +``` + +1. Start a transaction +2. Get `TraceContext` from current Transaction +3. Format the `TraceContext` or `tracestate` as a `traceparent` header. +:::::: + +::::::{tab-item} iOS +The agent will automatically inject trace headers into network requests using `URLSessions`, but if you’re using a non-standard network library you may need to manually inject them. It will be done using the OpenTelemetry APIs: + +1. Create a `Setter` +2. Create a `Span` per [Open Telemetry standards](https://github.com/open-telemetry/opentelemetry-swift/blob/main/Examples/Simple%20Exporter/main.swift#L35) +3. Inject trace context to header dictionary +4. Follow the procedure of your network library to complete the network request. Make sure to call `span.end()` when the request succeeds or fails. + +```swift +import OpenTelemetryApi +import OpenTelemetrySdk + +struct BasicSetter: Setter { + func set(carrier: inout [String: String], key: String, value: String) { + carrier[key] = value + } +} + +let span : Span = ... +let setter = BasicSetter() +let propagator = W3CTraceContextPropagator() +var headers = [String:String]() + +propagator.inject(spanContext: span.context, carrier: &headers, setter:setter) + +let request = URLRequest(...) +request.allHTTPHeaderFields = headers +... +span.end() +``` +:::::: + +::::::{tab-item} Java +1. Start a transaction with [`startTransaction`](https://www.elastic.co/guide/en/apm/agent/java/current/public-api.html#api-start-transaction), or a span with [`startSpan`](https://www.elastic.co/guide/en/apm/agent/java/current/public-api.html#api-span-start-span). +2. Inject the `traceparent` header into the request object with [`injectTraceHeaders`](https://www.elastic.co/guide/en/apm/agent/java/current/public-api.html#api-transaction-inject-trace-headers) +3. make network request + +Example of manually instrumenting an RPC framework: + +```java +// Hook into a callback provided by the RPC framework that is called on outgoing requests +public Response onOutgoingRequest(Request request) throws Exception { + Span span = ElasticApm.currentSpan() <1> + .startSpan("external", "http", null) + .setName(request.getMethod() + " " + request.getHost()); + try (final Scope scope = transaction.activate()) { + span.injectTraceHeaders((name, value) -> request.addHeader(name, value)); <2> + return request.execute(); + } catch (Exception e) { + span.captureException(e); + throw e; + } finally { + span.end(); <3> + } +} +``` + +1. Create a span representing an external call +2. Inject the `traceparent` header into the request object +3. End the span +:::::: + +::::::{tab-item} .NET +1. Serialize the distributed tracing context of the active transaction or span with [`CurrentTransaction`](https://www.elastic.co/guide/en/apm/agent/dotnet/current/public-api.html#api-current-transaction) or [`CurrentSpan`](https://www.elastic.co/guide/en/apm/agent/dotnet/current/public-api.html#api-current-span). +2. Send the serialized context the receiving service. + +Example: + +```csharp +string outgoingDistributedTracingData = + (Agent.Tracer.CurrentSpan?.OutgoingDistributedTracingData + ?? Agent.Tracer.CurrentTransaction?.OutgoingDistributedTracingData)?.SerializeToString(); +// Now send `outgoingDistributedTracingData` to the receiving service +``` +:::::: + +::::::{tab-item} Node.js +1. Start a transaction with [`apm.startTransaction()`](https://www.elastic.co/guide/en/apm/agent/nodejs/current/agent-api.html#apm-start-transaction), or a span with [`apm.startSpan()`](https://www.elastic.co/guide/en/apm/agent/nodejs/current/agent-api.html#apm-start-span). +2. Get the serialized `traceparent` string of the started transaction/span with [`currentTraceparent`](https://www.elastic.co/guide/en/apm/agent/nodejs/current/agent-api.html#apm-current-traceparent). +3. Encode the `traceparent` and send it to the receiving service inside your regular request. + +Example using raw UDP to communicate between two services, A and B: + +```js +agent.startTransaction('my-service-a-transaction'); <1> +const traceparent = agent.currentTraceparent; <2> +sendMetadata(`traceparent: ${traceparent}\n`); <3> +``` + +1. Start a transaction +2. Get the current `traceparent` +3. Send the `traceparent` as a header to service B. +:::::: + +::::::{tab-item} PHP +1. On the client side (i.e., the side sending the request) get the current distributed tracing context. +2. Serialize the current distributed tracing context to a format supported by the request’s transport and send it to the server side (i.e., the side receiving the request). + +Example: + +```php +$distDataAsString = ElasticApm::getSerializedCurrentDistributedTracingData(); <1> +``` + +1. Get the current distributed tracing data serialized as string +:::::: + +::::::{tab-item} Python +1. Start a transaction with [`begin_transaction()`](https://www.elastic.co/guide/en/apm/agent/python/current/api.html#client-api-begin-transaction). +2. Get the `trace_parent` of the active transaction. +3. Send the `trace_parent` to the receiving service. + +Example: + +```python +client.begin_transaction('new-transaction')<1> + +elasticapm.get_trace_parent_header('new-transaction') <2> + +# Send `trace_parent_str` to another service +``` + +1. Start a new transaction +2. Return the string representation of the current transaction’s `TraceParent` object +:::::: + +::::::{tab-item} Ruby +1. Start a span with [`with_span`](https://www.elastic.co/guide/en/apm/agent/ruby/current/api.html#api-agent-with_span). +2. Get the active `TraceContext`. +3. Send the `TraceContext` to the receiving service. + +```ruby +ElasticAPM.with_span "Name" do |span| <1> + header = span.trace_context.traceparent.to_header <2> + # send the TraceContext Header to a receiving service... +end +``` + +1. Start a span +2. Get the `TraceContext` +:::::: + +::::::: + +#### Parse the `traceparent` header on incoming requests [apm-distributed-tracing-incoming] + +Receiving services must parse the incoming `traceparent` header, and start a new transaction or span as a child of the received context. + +:::::::{tab-set} + +::::::{tab-item} Android +*Not applicable.* +:::::: + +::::::{tab-item} Go +1. Parse the incoming `TraceContext` with [`ParseTraceparentHeader`](https://pkg.go.dev/go.elastic.co/apm/module/apmhttp/v2#ParseTraceparentHeader) or [`ParseTracestateHeader`](https://pkg.go.dev/go.elastic.co/apm/module/apmhttp/v2#ParseTracestateHeader). +2. Start a new transaction or span as a child of the incoming transaction with [`StartTransactionOptions`](https://www.elastic.co/guide/en/apm/agent/go/current/api.html#tracer-api-start-transaction-options) or [`StartSpanOptions`](https://www.elastic.co/guide/en/apm/agent/go/current/api.html#transaction-start-span-options). + +Example: + +```go +// Receive incoming TraceContext +traceContext, _ := apmhttp.ParseTraceparentHeader(r.Header.Get("Traceparent")) <1> +traceContext.State, _ = apmhttp.ParseTracestateHeader(r.Header["Tracestate"]...) <2> + +opts := apm.TransactionOptions{ + TraceContext: traceContext, <3> +} +transaction := apm.DefaultTracer().StartTransactionOptions("GET /", "request", opts) <4> +``` + +1. Parse the `TraceParent` header +2. Parse the `Tracestate` header +3. Set the parent trace context +4. Start a new transaction as a child of the received `TraceContext` +:::::: + +::::::{tab-item} iOS +*Not applicable.* +:::::: + +::::::{tab-item} Java +1. Create a transaction as a child of the incoming transaction with [`startTransactionWithRemoteParent()`](https://www.elastic.co/guide/en/apm/agent/java/current/public-api.html#api-transaction-inject-trace-headers). +2. Start and name the transaction with [`activate()`](https://www.elastic.co/guide/en/apm/agent/java/current/public-api.html#api-transaction-activate) and [`setName()`](https://www.elastic.co/guide/en/apm/agent/java/current/public-api.html#api-set-name). + +Example: + +```java +// Hook into a callback provided by the framework that is called on incoming requests +public Response onIncomingRequest(Request request) throws Exception { + // creates a transaction representing the server-side handling of the request + Transaction transaction = ElasticApm.startTransactionWithRemoteParent(request::getHeader, request::getHeaders); <1> + try (final Scope scope = transaction.activate()) { <2> + String name = "a useful name like ClassName#methodName where the request is handled"; + transaction.setName(name); <3> + transaction.setType(Transaction.TYPE_REQUEST); <4> + return request.handle(); + } catch (Exception e) { + transaction.captureException(e); + throw e; + } finally { + transaction.end(); <5> + } +} +``` + +1. Create a transaction as the child of a remote parent +2. Activate the transaction +3. Name the transaction +4. Add a transaction type +5. Eventually, end the transaction +:::::: + +::::::{tab-item} .NET +Deserialize the incoming distributed tracing context, and pass it to any of the [`StartTransaction`](https://www.elastic.co/guide/en/apm/agent/dotnet/current/public-api.html#api-start-transaction) or [`CaptureTransaction`](https://www.elastic.co/guide/en/apm/agent/dotnet/current/public-api.html#convenient-capture-transaction) APIs — all of which have an optional `DistributedTracingData` parameter. This will create a new transaction or span as a child of the incoming trace context. + +Example starting a new transaction: + +```csharp +var transaction2 = Agent.Tracer.StartTransaction("Transaction2", "TestTransaction", + DistributedTracingData.TryDeserializeFromString(serializedDistributedTracingData)); +``` +:::::: + +::::::{tab-item} Node.js +1. Decode and store the `traceparent` in the receiving service. +2. Pass in the `traceparent` as the `childOf` option to manually start a new transaction as a child of the received `traceparent` with [`apm.startTransaction()`](https://www.elastic.co/guide/en/apm/agent/nodejs/current/agent-api.html#apm-start-transaction). + +Example receiving a `traceparent` over raw UDP: + +```js +const traceparent = readTraceparentFromUDPPacket() <1> +agent.startTransaction('my-service-b-transaction', { childOf: traceparent }) <2> +``` + +1. Read the `traceparent` from the incoming request. +2. Use the `traceparent` to initialize a new transaction that is a child of the original `traceparent`. +:::::: + +::::::{tab-item} PHP +1. Receive the distributed tracing data on the server side. +2. Begin a new transaction using the agent’s public API. For example, use [`ElasticApm::beginCurrentTransaction`](https://www.elastic.co/guide/en/apm/agent/php/current/public-api.html#api-elasticapm-class-begin-current-transaction) and pass the received distributed tracing data (serialized as string) as a parameter. This will create a new transaction as a child of the incoming trace context. +3. Don’t forget to eventually end the transaction on the server side. + +Example: + +```php +$receiverTransaction = ElasticApm::beginCurrentTransaction( <1> + 'GET /data-api', + 'data-layer', + /* timestamp */ null, + $distDataAsString <2> +); +``` + +1. Start a new transaction +2. Pass in the received distributed tracing data (serialized as string) + + +Once this new transaction has been created in the receiving service, you can create child spans, or use any other agent API methods as you typically would. +:::::: + +::::::{tab-item} Python +1. Create a `TraceParent` object from a string or HTTP header. +2. Start a new transaction as a child of the `TraceParent` by passing in a `TraceParent` object. + +Example using HTTP headers: + +```python +parent = elasticapm.trace_parent_from_headers(headers_dict) <1> +client.begin_transaction('processors', trace_parent=parent) <2> +``` + +1. Create a `TraceParent` object from HTTP headers formed as a dictionary +2. Begin a new transaction as a child of the received `TraceParent` + + +::::{tip} +See the [`TraceParent` API](https://www.elastic.co/guide/en/apm/agent/python/current/api.html#traceparent-api) for additional examples. +:::: +:::::: + +::::::{tab-item} Ruby +Start a new transaction or span as a child of the incoming transaction or span with [`with_transaction`](https://www.elastic.co/guide/en/apm/agent/ruby/current/api.html#api-agent-with_transaction) or [`with_span`](https://www.elastic.co/guide/en/apm/agent/ruby/current/api.html#api-agent-with_span). + +Example: + +```ruby +# env being a Rack env +context = ElasticAPM::TraceContext.parse(env: env) <1> + +ElasticAPM.with_transaction("Do things", trace_context: context) do <2> + ElasticAPM.with_span("Do nested thing", trace_context: context) do <3> + end +end +``` + +1. Parse the incoming `TraceContext` +2. Create a transaction as a child of the incoming `TraceContext` +3. Create a span as a child of the newly created transaction. `trace_context` is optional here, as spans are automatically created as a child of their parent’s transaction’s `TraceContext` when none is passed. +:::::: + +::::::: + +% Stateful only for RUM + +### Distributed tracing with RUM [apm-distributed-tracing-rum] + +Some additional setup may be required to correlate requests correctly with the Real User Monitoring (RUM) agent. + +See the [RUM distributed tracing guide](https://www.elastic.co/guide/en/apm/agent/rum-js/current/distributed-tracing-guide.html) for information on enabling cross-origin requests, setting up server configuration, and working with dynamically-generated HTML. \ No newline at end of file diff --git a/solutions/observability/apps/track-deployments-with-annotations.md b/solutions/observability/apps/track-deployments-with-annotations.md index 515f3618ff..bb11c90511 100644 --- a/solutions/observability/apps/track-deployments-with-annotations.md +++ b/solutions/observability/apps/track-deployments-with-annotations.md @@ -2,13 +2,33 @@ mapped_urls: - https://www.elastic.co/guide/en/observability/current/apm-transactions-annotations.html - https://www.elastic.co/guide/en/serverless/current/observability-apm-track-deployments-with-annotations.html + +navigation_title: "Track deployments with annotations" --- -# Track deployments with annotations +# Track deployments with annotations [apm-transactions-annotations] + +::::{admonition} Required role +:class: note + +For Observability serverless projects, the **Admin** role or higher is required to create and manage annotations. To learn more, refer to [Assign user roles and privileges](../../../deploy-manage/users-roles/cloud-organization/user-roles.md#general-assign-user-roles). + +:::: + +:::{image} ../../../images/observability-apm-transaction-annotation.png +:alt: Example view of transactions annotation in the Applications UI +:class: screenshot +::: + +For enhanced visibility into your deployments, we offer deployment annotations on all transaction charts. This feature enables you to easily determine if your deployment has increased response times for an end-user, or if the memory/CPU footprint of your application has changed. Being able to quickly identify bad deployments enables you to rollback and fix issues without causing costly outages. + +By default, automatic deployment annotations are enabled. This means APM will create an annotation on your data when the `service.version` of your application changes. + +Alternatively, you can explicitly create deployment annotations with our annotation API. The API can integrate into your CI/CD pipeline, so that each time you deploy, a POST request is sent to the annotation API endpoint. -% What needs to be done: Align serverless/stateful +Refer to the [annotation API](../../../solutions/observability/apps/annotation-api.md) reference for more information. -% Use migrated content from existing pages that map to this page: +::::{note} +If custom annotations have been created for the selected time period, any derived annotations, i.e., those created automatically when `service.version` changes, will not be shown. -% - [ ] ./raw-migrated-files/observability-docs/observability/apm-transactions-annotations.md -% - [ ] ./raw-migrated-files/docs-content/serverless/observability-apm-track-deployments-with-annotations.md \ No newline at end of file +:::: \ No newline at end of file diff --git a/solutions/observability/apps/transaction-sampling.md b/solutions/observability/apps/transaction-sampling.md index 5208243c91..bb1536d505 100644 --- a/solutions/observability/apps/transaction-sampling.md +++ b/solutions/observability/apps/transaction-sampling.md @@ -4,25 +4,301 @@ mapped_urls: - https://www.elastic.co/guide/en/serverless/current/observability-apm-transaction-sampling.html --- -# Transaction sampling +# Transaction sampling [apm-sampling] -% What needs to be done: Align serverless/stateful +[Distributed tracing](../../../solutions/observability/apps/traces.md) can generate a substantial amount of data. More data can mean higher costs and more noise. Sampling aims to lower the amount of data ingested and the effort required to analyze that data — all while still making it easy to find anomalous patterns in your applications, detect outages, track errors, and lower mean time to recovery (MTTR). -% Use migrated content from existing pages that map to this page: +Elastic APM supports two types of sampling: -% - [ ] ./raw-migrated-files/observability-docs/observability/apm-sampling.md -% - [ ] ./raw-migrated-files/docs-content/serverless/observability-apm-transaction-sampling.md +% Stateful only for Tail-based sampling -% Internal links rely on the following IDs being on this page (e.g. as a heading ID, paragraph ID, etc): +* [Head-based sampling](../../../solutions/observability/apps/transaction-sampling.md#apm-head-based-sampling) +* [Tail-based sampling](../../../solutions/observability/apps/transaction-sampling.md#apm-tail-based-sampling) -$$$apm-tail-based-sampling$$$ -$$$apm-configure-head-based-sampling$$$ +## Head-based sampling [apm-head-based-sampling] -$$$apm-configure-tail-based-sampling$$$ +In head-based sampling, the sampling decision for each trace is made when the trace is initiated. Each trace has a defined and equal probability of being sampled. -$$$apm-head-based-sampling$$$ +For example, a sampling value of `.2` indicates a transaction sample rate of `20%`. This means that only `20%` of traces will send and retain all of their associated information. The remaining traces will drop contextual information to reduce the transfer and storage size of the trace. -$$$distributed-tracing-examples$$$ +Head-based sampling is quick and easy to set up. Its downside is that it’s entirely random — interesting data might be discarded purely due to chance. -$$$observability-apm-transaction-sampling-distributed-tracing$$$ \ No newline at end of file +See [Configure head-based sampling](../../../solutions/observability/apps/transaction-sampling.md#apm-configure-head-based-sampling) to get started. + + +### Distributed tracing [distributed-tracing-examples] + +In a distributed trace, the sampling decision is still made when the trace is initiated. Each subsequent service respects the initial service’s sampling decision, regardless of its configured sample rate; the result is a sampling percentage that matches the initiating service. + +In the example in *Figure 1*, `Service A` initiates four transactions and has sample rate of `.5` (`50%`). The upstream sampling decision is respected, so even if the sample rate is defined and is a different value in `Service B` and `Service C`, the sample rate will be `.5` (`50%`) for all services. + +**Figure 1. Upstream sampling decision is respected** + +:::{image} ../../../images/observability-dt-sampling-example-1.png +:alt: Distributed tracing and head based sampling example one +:class: screenshot +::: + +In the example in *Figure 2*, `Service A` initiates four transactions and has a sample rate of `1` (`100%`). Again, the upstream sampling decision is respected, so the sample rate for all services will be `1` (`100%`). + +**Figure 2. Upstream sampling decision is respected** + +:::{image} ../../../images/observability-dt-sampling-example-2.png +:alt: Distributed tracing and head based sampling example two +:class: screenshot +::: + + +#### Trace continuation strategies with distributed tracing [_trace_continuation_strategies_with_distributed_tracing] + +In addition to setting the sample rate, you can also specify which *trace continuation strategy* to use. There are three trace continuation strategies: `continue`, `restart`, and `restart_external`. + +The **`continue`** trace continuation strategy is the default and will behave similar to the examples in the [Distributed tracing section](../../../solutions/observability/apps/transaction-sampling.md#distributed-tracing-examples). + +Use the **`restart_external`** trace continuation strategy on an Elastic-monitored service to start a new trace if the previous service did not have a `traceparent` header with `es` vendor data. This can be helpful if a transaction includes an Elastic-monitored service that is receiving requests from an unmonitored service. + +In the example in *Figure 3*, `Service A` is an Elastic-monitored service that initiates four transactions with a sample rate of `.25` (`25%`). Because `Service B` is unmonitored, the traces started in `Service A` will end there. `Service C` is an Elastic-monitored service that initiates four transactions that start new traces with a new sample rate of `.5` (`50%`). Because `Service D` is also Elastic-monitored service, the upstream sampling decision defined in `Service C` is respected. The end result will be three sampled traces. + +**Figure 3. Using the `restart_external` trace continuation strategy** + +:::{image} ../../../images/observability-dt-sampling-continuation-strategy-restart_external.png +:alt: Distributed tracing and head based sampling with restart_external continuation strategy +:class: screenshot +::: + +Use the **`restart`** trace continuation strategy on an Elastic-monitored service to start a new trace regardless of whether the previous service had a `traceparent` header. This can be helpful if an Elastic-monitored service is publicly exposed, and you do not want tracing data to possibly be spoofed by user requests. + +In the example in *Figure 4*, `Service A` and `Service B` are Elastic-monitored services that use the default trace continuation strategy. `Service A` has a sample rate of `.25` (`25%`), and that sampling decision is respected in `Service B`. `Service C` is an Elastic-monitored service that uses the `restart` trace continuation strategy and has a sample rate of `1` (`100%`). Because it uses `restart`, the upstream sample rate is *not* respected in `Service C` and all four traces will be sampled as new traces in `Service C`. The end result will be five sampled traces. + +:::{image} ../../../images/observability-dt-sampling-continuation-strategy-restart.png +:alt: Distributed tracing and head based sampling with restart continuation strategy +:title: Using the `restart` trace continuation strategy +::: + + +### OpenTelemetry [_opentelemetry] + +Head-based sampling is implemented directly in the APM agents and SDKs. The sample rate must be propagated between services and the managed intake service in order to produce accurate metrics. + +OpenTelemetry offers multiple samplers. However, most samplers do not propagate the sample rate. This results in inaccurate span-based metrics, like APM throughput, latency, and error metrics. + +For accurate span-based metrics when using head-based sampling with OpenTelemetry, you must use a [consistent probability sampler](https://opentelemetry.io/docs/specs/otel/trace/tracestate-probability-sampling/). These samplers propagate the sample rate between services and the managed intake service, resulting in accurate metrics. + +::::{note} +OpenTelemetry does not offer consistent probability samplers in all languages. OpenTelemetry users should consider using tail-based sampling instead. + +Refer to the documentation of your favorite OpenTelemetry agent or SDK for more information on the availability of consistent probability samplers. + +:::: + +% Stateful only for tail-based sampling + +## Tail-based sampling [apm-tail-based-sampling] + +::::{admonition} Support for tail-based sampling +:class: note + +Tail-based sampling is only supported when writing to {{es}}. If you are using a different [output](../../../solutions/observability/apps/configure-output.md), tail-based sampling is *not* supported. + +Tail-based sampling is *not* compatible with [{{serverless-full}}](https://docs.elastic.co/serverless). + +:::: + + +In tail-based sampling, the sampling decision for each trace is made after the trace has completed. This means all traces will be analyzed against a set of rules, or policies, which will determine the rate at which they are sampled. + +Unlike head-based sampling, each trace does not have an equal probability of being sampled. Because slower traces are more interesting than faster ones, tail-based sampling uses weighted random sampling — so traces with a longer root transaction duration are more likely to be sampled than traces with a fast root transaction duration. + +A downside of tail-based sampling is that it results in more data being sent from APM agents to the APM Server. The APM Server will therefore use more CPU, memory, and disk than with head-based sampling. However, because the tail-based sampling decision happens in APM Server, there is less data to transfer from APM Server to {{es}}. So running APM Server close to your instrumented services can reduce any increase in transfer costs that tail-based sampling brings. + +See [Configure tail-based sampling](../../../solutions/observability/apps/transaction-sampling.md#apm-configure-tail-based-sampling) to get started. + + +### Distributed tracing with tail-based sampling [_distributed_tracing_with_tail_based_sampling] + +With tail-based sampling, all traces are observed and a sampling decision is only made once a trace completes. + +In this example, `Service A` initiates four transactions. If our sample rate is `.5` (`50%`) for traces with a `success` outcome, and `1` (`100%`) for traces with a `failure` outcome, the sampled traces would look something like this: + +:::{image} ../../../images/observability-dt-sampling-example-3.png +:alt: Distributed tracing and tail based sampling example one +::: + + +### OpenTelemetry with tail-based sampling [_opentelemetry_with_tail_based_sampling] + +Tail-based sampling is implemented entirely in APM Server, and will work with traces sent by either Elastic APM agents or OpenTelemetry SDKs. + +Due to [OpenTelemetry tail-based sampling limitations](../../../solutions/observability/apps/limitations.md#apm-open-telemetry-tbs) when using [tailsamplingprocessor](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/processor/tailsamplingprocessor), we recommend using APM Server tail-based sampling instead. + + +## Sampled data and visualizations [_sampled_data_and_visualizations] + +A sampled trace retains all data associated with it. A non-sampled trace drops all [span](../../../solutions/observability/apps/spans.md) and [transaction](../../../solutions/observability/apps/transactions.md) data1. Regardless of the sampling decision, all traces retain [error](../../../solutions/observability/apps/errors.md) data. + +Some visualizations in the {{apm-app}}, like latency, are powered by aggregated transaction and span [metrics](../../../solutions/observability/apps/metrics.md). The way these metrics are calculated depends on the sampling method used: + +* **Head-based sampling**: Metrics are calculated based on all sampled events. +* **Tail-based sampling**: Metrics are calculated based on all events, regardless of whether they are ultimately sampled or not. +* **Both head and tail-based sampling**: When both methods are used together, metrics are calculated based on all events that were sampled by the head-based sampling policy. + +For all sampling methods, metrics are weighted by the inverse sampling rate of the head-based sampling policy to provide an estimate of the total population. For example, if your head-based sampling rate is 5%, each sampled trace is counted as 20. As the variance of latency increases or the head-based sampling rate decreases, the level of error in these calculations may increase. + +These calculation methods ensure that the APM app provides the most accurate metrics possible given the sampling strategy in use, while also accounting for the head-based sampling rate to estimate the full population of traces. + +1 Real User Monitoring (RUM) traces are an exception to this rule. The {{kib}} apps that utilize RUM data depend on transaction events, so non-sampled RUM traces retain transaction data — only span data is dropped. + + +## Sample rates [_sample_rates] + +What’s the best sampling rate? Unfortunately, there isn’t one. Sampling is dependent on your data, the throughput of your application, data retention policies, and other factors. A sampling rate from `.1%` to `100%` would all be considered normal. You’ll likely decide on a unique sample rate for different scenarios. Here are some examples: + +* Services with considerably more traffic than others might be safe to sample at lower rates +* Routes that are more important than others might be sampled at higher rates +* A production service environment might warrant a higher sampling rate than a development environment +* Failed trace outcomes might be more interesting than successful traces — thus requiring a higher sample rate + +Regardless of the above, cost conscious customers are likely to be fine with a lower sample rate. + + +## Configure head-based sampling [apm-configure-head-based-sampling] + +There are three ways to adjust the head-based sampling rate of your APM agents: + + +### Dynamic configuration [_dynamic_configuration] + +The transaction sample rate can be changed dynamically (no redeployment necessary) on a per-service and per-environment basis with [{{apm-agent}} Configuration](../../../solutions/observability/apps/apm-agent-central-configuration.md) in {{kib}}. + + +### {{kib}} API configuration [_kib_api_configuration] + +{{apm-agent}} configuration exposes an API that can be used to programmatically change your agents' sampling rate. An example is provided in the [Agent configuration API reference](../../../solutions/observability/apps/agent-configuration-api.md). + + +### {{apm-agent}} configuration [_apm_agent_configuration] + +Each agent provides a configuration value used to set the transaction sample rate. See the relevant agent’s documentation for more details: + +* Go: [`ELASTIC_APM_TRANSACTION_SAMPLE_RATE`](https://www.elastic.co/guide/en/apm/agent/go/current/configuration.html#config-transaction-sample-rate) +* Java: [`transaction_sample_rate`](https://www.elastic.co/guide/en/apm/agent/java/current/config-core.html#config-transaction-sample-rate) +* .NET: [`TransactionSampleRate`](https://www.elastic.co/guide/en/apm/agent/dotnet/current/config-core.html#config-transaction-sample-rate) +* Node.js: [`transactionSampleRate`](https://www.elastic.co/guide/en/apm/agent/nodejs/current/configuration.html#transaction-sample-rate) +* PHP: [`transaction_sample_rate`](https://www.elastic.co/guide/en/apm/agent/php/current/configuration-reference.html#config-transaction-sample-rate) +* Python: [`transaction_sample_rate`](https://www.elastic.co/guide/en/apm/agent/python/current/configuration.html#config-transaction-sample-rate) +* Ruby: [`transaction_sample_rate`](https://www.elastic.co/guide/en/apm/agent/ruby/current/configuration.html#config-transaction-sample-rate) + + +## Configure tail-based sampling [apm-configure-tail-based-sampling] + +Enable tail-based sampling with [Enable tail-based sampling](../../../solutions/observability/apps/tail-based-sampling.md#sampling-tail-enabled-ref). When enabled, trace events are mapped to sampling policies. Each sampling policy must specify a sample rate, and can optionally specify other conditions. All of the policy conditions must be true for a trace event to match it. + +Trace events are matched to policies in the order specified. Each policy list must conclude with a default policy — one that only specifies a sample rate. This default policy is used to catch remaining trace events that don’t match a stricter policy. Requiring this default policy ensures that traces are only dropped intentionally. If you enable tail-based sampling and send a transaction that does not match any of the policies, APM Server will reject the transaction with the error `no matching policy`. + +::::{important} +Please note that from version `8.3.1` APM Server implements a default storage limit of 3GB, but, due to how the limit is calculated and enforced the actual disk space may still grow slightly over the limit. +:::: + + + +### Example configuration [_example_configuration] + +This example defines three tail-based sampling polices: + +```yaml +- sample_rate: 1 <1> + service.environment: production + trace.name: "GET /very_important_route" +- sample_rate: .01 <2> + service.environment: production + trace.name: "GET /not_important_route" +- sample_rate: .1 <3> +``` + +1. Samples 100% of traces in `production` with the trace name `"GET /very_important_route"` +2. Samples 1% of traces in `production` with the trace name `"GET /not_important_route"` +3. Default policy to sample all remaining traces at 10%, e.g. traces in a different environment, like `dev`, or traces with any other name + + + +### Configuration reference [_configuration_reference] + + +#### Top-level tail-based sampling settings [_top_level_tail_based_sampling_settings] + + +##### Enable tail-based sampling [sampling-tail-enabled-{{input-type}}] + +Set to `true` to enable tail based sampling. Disabled by default. (bool) + +| | | +| --- | --- | +| APM Server binary | `sampling.tail.enabled` | +| Fleet-managed | `Enable tail-based sampling` | + + +##### Interval [sampling-tail-interval-{{input-type}}] + +Synchronization interval for multiple APM Servers. Should be in the order of tens of seconds or low minutes. Default: `1m` (1 minute). (duration) + +| | | +| --- | --- | +| APM Server binary | `sampling.tail.interval` | +| Fleet-managed | `Interval` | + + +##### Policies [sampling-tail-policies-{{input-type}}] + +Criteria used to match a root transaction to a sample rate. + +Policies map trace events to a sample rate. Each policy must specify a sample rate. Trace events are matched to policies in the order specified. All policy conditions must be true for a trace event to match. Each policy list should conclude with a policy that only specifies a sample rate. This final policy is used to catch remaining trace events that don’t match a stricter policy. (`[]policy`) + +| | | +| --- | --- | +| APM Server binary | `sampling.tail.policies` | +| Fleet-managed | `Policies` | + + +##### Storage limit [sampling-tail-storage_limit-{{input-type}}] + +The amount of storage space allocated for trace events matching tail sampling policies. Caution: Setting this limit higher than the allowed space may cause APM Server to become unhealthy. + +If the configured storage limit is insufficient, it logs "configured storage limit reached". The event will bypass sampling and will always be indexed when storage limit is reached. + +Default: `3GB`. (text) + +| | | +| --- | --- | +| APM Server binary | `sampling.tail.storage_limit` | +| Fleet-managed | `Storage limit` | + + +#### Policy settings [_policy_settings] + + +##### **`sample_rate`** [sampling-tail-sample-rate-{{input-type}}] + +The sample rate to apply to trace events matching this policy. Required in each policy. + +The sample rate must be greater than or equal to `0` and less than or equal to `1`. For example, a `sample_rate` of `0.01` means that 1% of trace events matching the policy will be sampled. A `sample_rate` of `1` means that 100% of trace events matching the policy will be sampled. (int) + + +##### **`trace.name`** [sampling-tail-trace-name-{{input-type}}] + +The trace name for events to match a policy. A match occurs when the configured `trace.name` matches the `transaction.name` of the root transaction of a trace. A root transaction is any transaction without a `parent.id`. (string) + + +##### **`trace.outcome`** [sampling-tail-trace-outcome-{{input-type}}] + +The trace outcome for events to match a policy. A match occurs when the configured `trace.outcome` matches a trace’s `event.outcome` field. Trace outcome can be `success`, `failure`, or `unknown`. (string) + + +##### **`service.name`** [sampling-tail-service-name-{{input-type}}] + +The service name for events to match a policy. (string) + + +##### **`service.environment`** [sampling-tail-service-environment-{{input-type}}] + +The service environment for events to match a policy. (string) \ No newline at end of file diff --git a/solutions/observability/apps/transactions-2.md b/solutions/observability/apps/transactions-2.md index 928c72063d..828bc81e9c 100644 --- a/solutions/observability/apps/transactions-2.md +++ b/solutions/observability/apps/transactions-2.md @@ -4,17 +4,166 @@ mapped_urls: - https://www.elastic.co/guide/en/serverless/current/observability-apm-transactions.html --- -# Transactions +# Transactions [apm-transactions] -% What needs to be done: Align serverless/stateful +A *transaction* describes an event captured by an Elastic APM agent instrumenting a service. APM agents automatically collect performance metrics on HTTP requests, database queries, and much more. -% Use migrated content from existing pages that map to this page: -% - [ ] ./raw-migrated-files/observability-docs/observability/apm-transactions.md -% - [ ] ./raw-migrated-files/docs-content/serverless/observability-apm-transactions.md +:::{image} ../../../images/observability-apm-transactions-overview.png +:alt: Example view of transactions table in the Applications UI +:class: screenshot +::: -% Internal links rely on the following IDs being on this page (e.g. as a heading ID, paragraph ID, etc): +The **Latency**, **Throughput**, **Failed transaction rate**, **Time spent by span type**, and **Cold start rate** charts display information on all transactions associated with the selected service: -$$$transaction-trace-sample$$$ +**Latency** +: Response times for the service. Options include average, 95th, and 99th percentile. If there’s a weird spike that you’d like to investigate, you can simply zoom in on the graph - this will adjust the specific time range, and all of the data on the page will update accordingly. -$$$transaction-details$$$ \ No newline at end of file +**Throughput** +: Visualize response codes: `2xx`, `3xx`, `4xx`, etc. Useful for determining if more responses than usual are being served with a particular response code. Like in the latency graph, you can zoom in on anomalies to further investigate them. + +**Failed transaction rate** +: The failed transaction rate represents the percentage of failed transactions from the perspective of the selected service. It’s useful for visualizing unexpected increases, decreases, or irregular patterns in a service’s transactions. + + ::::{tip} + HTTP **transactions** from the HTTP server perspective do not consider a `4xx` status code (client error) as a failure because the failure was caused by the caller, not the HTTP server. Thus, `event.outcome=success` and there will be no increase in failed transaction rate. + + HTTP **spans** from the client perspective however, are considered failures if the HTTP status code is ≥ 400. These spans will set `event.outcome=failure` and increase the failed transaction rate. + + If there is no HTTP status, both transactions and spans are considered successful unless an error is reported. + + :::: + + +**Time spent by span type** +: Visualize where your application is spending most of its time. For example, is your app spending time in external calls, database processing, or application code execution? + + The time a transaction took to complete is also recorded and displayed on the chart under the "app" label. "app" indicates that something was happening within the application, but we’re not sure exactly what. This could be a sign that the APM agent does not have auto-instrumentation for whatever was happening during that time. + + It’s important to note that if you have asynchronous spans, the sum of all span times may exceed the duration of the transaction. + + +**Cold start rate** +: Only applicable to serverless transactions, this chart displays the percentage of requests that trigger a cold start of a serverless function. See [Cold starts](../../../solutions/observability/apps/observe-lambda-functions.md#apm-lambda-cold-start-info) for more information. + + +## Transactions table [transactions-table] + +The **Transactions** table displays a list of *transaction groups* for the selected service. In other words, this view groups all transactions of the same name together, and only displays one entry for each group. + +:::{image} ../../../images/observability-apm-transactions-table.png +:alt: Example view of the transactions table in the Applications UI in Kibana +:class: screenshot +::: + +By default, transaction groups are sorted by *Impact*. Impact helps show the most used and slowest endpoints in your service - in other words, it’s the collective amount of pain a specific endpoint is causing your users. If there’s a particular endpoint you’re worried about, you can click on it to view the [transaction details](../../../solutions/observability/apps/transactions-2.md#transaction-details). + +::::{important} +If you only see one route in the Transactions table, or if you have transactions named "unknown route", it could be a symptom that the APM agent either wasn’t installed correctly or doesn’t support your framework. + +For further details, including troubleshooting and custom implementation instructions, refer to the documentation for each [APM Agent](https://www.elastic.co/guide/en/apm/agent) you’ve implemented. + +:::: + +% Stateful only for RUM + +## RUM Transaction overview [rum-transaction-overview] + +The transaction overview page is customized for the JavaScript RUM agent. Specifically, the page highlights **page load times** for your service: + +:::{image} ../../../images/observability-apm-geo-ui.png +:alt: average page load duration distribution +:class: screenshot +::: + +Additional RUM goodies, like core vitals, and visitor breakdown by browser, location, and device, are available in the Observability User Experience tab. + + +## Transaction details [transaction-details] + +Selecting a transaction group will bring you to the **transaction** details. This page is visually similar to the transaction overview, but it shows data from all transactions within the selected transaction group. + +:::{image} ../../../images/observability-apm-transactions-overview.png +:alt: Example view of response time distribution +:class: screenshot +::: + + +### Latency distribution [transaction-duration-distribution] + +The latency distribution shows a plot of all transaction durations for the given time period. The following screenshot shows a typical distribution and indicates most of our requests were served quickly — awesome! The requests on the right are taking longer than average; we probably need to focus on them. + +:::{image} ../../../images/observability-apm-transaction-duration-dist.png +:alt: Example view of latency distribution graph +:class: screenshot +::: + +Click and drag to select a latency duration *bucket* to display up to 500 trace samples. + + +### Trace samples [transaction-trace-sample] + +Trace samples are based on the *bucket* selection in the **Latency distribution** chart; update the samples by selecting a new *bucket*. The number of requests per bucket is displayed when hovering over the graph, and the selected bucket is highlighted to stand out. + +Each bucket presents up to ten trace samples in a **timeline**, trace sample **metadata**, and any related **logs**. + +**Trace sample timeline** + +Each sample has a trace timeline waterfall that shows how a typical request in that bucket executed. This waterfall is useful for understanding the parent/child hierarchy of transactions and spans, and ultimately determining *why* a request was slow. For large waterfalls, expand problematic transactions and collapse well-performing ones for easier problem isolation and troubleshooting. + +:::{image} ../../../images/observability-apm-transaction-sample.png +:alt: Example view of transactions sample +:class: screenshot +::: + +::::{note} +More information on timeline waterfalls is available in [spans](../../../solutions/observability/apps/trace-sample-timeline.md). + +:::: + + +**Trace sample metadata** + +Learn more about a trace sample in the **Metadata** tab: + +* Labels: Custom labels added by APM agents +* HTTP request/response information +* Host information +* Container information +* Service: The service/application runtime, APM agent, name, etc.. +* Process: The process id that served up the request. +* APM agent information +* URL +* User: Requires additional configuration, but allows you to see which user experienced the current transaction. +* FaaS information, like cold start, AWS request ID, trigger type, and trigger request ID + +::::{tip} +All of this data is stored in documents in Elasticsearch. This means you can select "Actions - View transaction in Discover" to see the actual Elasticsearch document under the discover tab. + +:::: + + +**Trace sample logs** + +The **Logs** tab displays logs related to the sampled trace. + +Logs provide detailed information about specific events, and are crucial to successfully debugging slow or erroneous transactions. + +If you’ve correlated your application’s logs and traces, you never have to search for relevant data; it’s already available to you. Viewing log and trace data together allows you to quickly diagnose and solve problems. + +To learn how to correlate your logs with your instrumented services, see [Stream application logs](../../../solutions/observability/logs/stream-application-logs.md) + +:::{image} ../../../images/observability-apm-logs-tab.png +:alt: APM logs tab +:class: screenshot +::: + + +### Correlations [transaction-latency-correlations] + +Correlations surface attributes of your data that are potentially correlated with high-latency or erroneous transactions. To learn more, see [Find transaction latency and failure correlations](../../../solutions/observability/apps/find-transaction-latency-failure-correlations.md). + +:::{image} ../../../images/observability-correlations-hover.png +:alt: APM lattency correlations +:class: screenshot +::: \ No newline at end of file diff --git a/solutions/observability/apps/tune-data-ingestion.md b/solutions/observability/apps/tune-data-ingestion.md index 5adb3e9ebc..aa584c0a7d 100644 --- a/solutions/observability/apps/tune-data-ingestion.md +++ b/solutions/observability/apps/tune-data-ingestion.md @@ -8,30 +8,30 @@ mapped_pages: This section explains how to adapt data ingestion according to your needs. -## Tune APM Server [apm-tune-apm-server] +## Tune APM Server [apm-tune-apm-server] * [Add APM Server or {{agent}} instances](#apm-add-apm-server-instances) * [Reduce the payload size](#apm-reduce-payload-size) * [Adjust anonymous auth rate limit](#apm-adjust-event-rate) -### Add APM Server or {{agent}} instances [apm-add-apm-server-instances] +### Add APM Server or {{agent}} instances [apm-add-apm-server-instances] If the APM Server cannot process data quickly enough, you will see request timeouts. One way to solve this problem is to increase processing power. Increase processing power by either migrating to a more powerful machine or adding more APM Server/Elastic Agent instances. Having several instances will also increase [availability](high-availability.md). -### Reduce the payload size [apm-reduce-payload-size] +### Reduce the payload size [apm-reduce-payload-size] Large payloads may result in request timeouts. You can reduce the payload size by decreasing the flush interval in the agents. This will cause agents to send smaller and more frequent requests. -Optionally you can also [reduce the sample rate](reduce-storage.md#apm-reduce-sample-rate) or [reduce the amount of stack traces](reduce-storage.md#apm-reduce-stacktrace). +Optionally you can also [reduce the sample rate](reduce-storage.md#apm-reduce-sample-rate) or [reduce the amount of stack traces](reduce-storage.md#observability-apm-reduce-stacktrace). Read more in the [agents documentation](https://www.elastic.co/guide/en/apm/agent/index.html). -### Adjust anonymous auth rate limit [apm-adjust-event-rate] +### Adjust anonymous auth rate limit [apm-adjust-event-rate] Agents make use of long running requests and flush as many events over a single request as possible. Thus, the rate limiter for anonymous authentication is bound to the number of *events* sent per second, per IP. @@ -45,7 +45,7 @@ Increasing the default value for the following configuration variable will help | Fleet-managed | `Anonymous Event rate limit (event limit)` | -## Tune {{es}} [apm-tune-elasticsearch] +## Tune {{es}} [apm-tune-elasticsearch] The {{es}} Reference provides insight on tuning {{es}}. diff --git a/solutions/observability/apps/upstream-opentelemetry-collectors-language-sdks.md b/solutions/observability/apps/upstream-opentelemetry-collectors-language-sdks.md index 9275e3c6c2..5d86dd67e6 100644 --- a/solutions/observability/apps/upstream-opentelemetry-collectors-language-sdks.md +++ b/solutions/observability/apps/upstream-opentelemetry-collectors-language-sdks.md @@ -4,21 +4,262 @@ mapped_urls: - https://www.elastic.co/guide/en/serverless/current/observability-apm-agents-opentelemetry-opentelemetry-native-support.html --- -# Upstream OpenTelemetry Collectors and language SDKs +# Upstream OpenTelemetry Collectors and language SDKs [apm-open-telemetry-direct] -% What needs to be done: Align serverless/stateful +::::{note} +This is one of several approaches you can use to integrate Elastic with OpenTelemetry. **To compare approaches and choose the best approach for your use case, refer to [OpenTelemetry](../../../solutions/observability/apps/use-opentelemetry-with-apm.md).** -% Use migrated content from existing pages that map to this page: +:::: -% - [ ] ./raw-migrated-files/observability-docs/observability/apm-open-telemetry-direct.md -% - [ ] ./raw-migrated-files/docs-content/serverless/observability-apm-agents-opentelemetry-opentelemetry-native-support.md -% Internal links rely on the following IDs being on this page (e.g. as a heading ID, paragraph ID, etc): +The {{stack}} natively supports the OpenTelemetry protocol (OTLP). This means trace data and metrics collected from your applications and infrastructure can be sent directly to the {{stack}}. -$$$apm-instrument-apps-otel$$$ +* Send data to Elastic from an upstream [OpenTelemetry Collector](../../../solutions/observability/apps/upstream-opentelemetry-collectors-language-sdks.md#apm-connect-open-telemetry-collector) +* Send data to Elastic from an upstream [OpenTelemetry language SDK](../../../solutions/observability/apps/upstream-opentelemetry-collectors-language-sdks.md#apm-instrument-apps-otel) -$$$apm-connect-open-telemetry-collector$$$ -$$$observability-apm-agents-opentelemetry-opentelemetry-native-support-send-data-from-an-upstream-opentelemetry-collector$$$ +## Send data from an upstream OpenTelemetry Collector [apm-connect-open-telemetry-collector] -$$$observability-apm-agents-opentelemetry-opentelemetry-native-support-send-data-from-an-upstream-opentelemetry-sdk$$$ \ No newline at end of file +Connect your OpenTelemetry Collector instances to Elastic {{observability}} or {{obs-serverless}} using the OTLP exporter: + +::::{tab-set} +:group: stack-serverless + +:::{tab-item} Elastic Stack v9 +:sync: stack + + +```yaml +receivers: <1> + # ... + otlp: + protocols: + grpc: + endpoint: 0.0.0.0:4317 + http: + endpoint: 0.0.0.0:4318 +processors: <2> + # ... + memory_limiter: + check_interval: 1s + limit_mib: 2000 + batch: + +exporters: + debug: + verbosity: detailed <3> + otlp: <4> + # Elastic APM server https endpoint without the "https://" prefix + endpoint: "${env:ELASTIC_APM_SERVER_ENDPOINT}" <5> <7> + headers: + # Elastic APM Server secret token + Authorization: "Bearer ${env:ELASTIC_APM_SECRET_TOKEN}" <6> <7> + +service: + pipelines: + traces: + receivers: [otlp] + processors: [..., memory_limiter, batch] + exporters: [debug, otlp] + metrics: + receivers: [otlp] + processors: [..., memory_limiter, batch] + exporters: [debug, otlp] + logs: <8> + receivers: [otlp] + processors: [..., memory_limiter, batch] + exporters: [debug, otlp] +``` + +1. The receivers, like the [OTLP receiver](https://github.com/open-telemetry/opentelemetry-collector/tree/main/receiver/otlpreceiver), that forward data emitted by APM agents, or the [host metrics receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/hostmetricsreceiver). +2. We recommend using the [Batch processor](https://github.com/open-telemetry/opentelemetry-collector/blob/main/processor/batchprocessor/README.md) and the [memory limiter processor](https://github.com/open-telemetry/opentelemetry-collector/blob/main/processor/memorylimiterprocessor/README.md). For more information, see [recommended processors](https://github.com/open-telemetry/opentelemetry-collector/blob/main/processor/README.md#recommended-processors). +3. The [debug exporter](https://github.com/open-telemetry/opentelemetry-collector/tree/main/exporter/debugexporter) is helpful for troubleshooting, and supports configurable verbosity levels: `basic` (default), `normal`, and `detailed`. +4. Elastic {{observability}} endpoint configuration. APM Server supports a ProtoBuf payload via both the OTLP protocol over gRPC transport [(OTLP/gRPC)](https://opentelemetry.io/docs/specs/otlp/#otlpgrpc) and the OTLP protocol over HTTP transport [(OTLP/HTTP)](https://opentelemetry.io/docs/specs/otlp/#otlphttp). To learn more about these exporters, see the OpenTelemetry Collector documentation: [OTLP/HTTP Exporter](https://github.com/open-telemetry/opentelemetry-collector/tree/main/exporter/otlphttpexporter) or [OTLP/gRPC exporter](https://github.com/open-telemetry/opentelemetry-collector/tree/main/exporter/otlpexporter). When adding an endpoint to an existing configuration an optional name component can be added, like `otlp/elastic`, to distinguish endpoints as described in the [OpenTelemetry Collector Configuration Basics](https://opentelemetry.io/docs/collector/configuration/#basics). +5. Hostname and port of the APM Server endpoint. For example, `elastic-apm-server:8200`. +6. Credential for Elastic APM [secret token authorization](../../../solutions/observability/apps/secret-token.md) (`Authorization: "Bearer a_secret_token"`) or [API key authorization](../../../solutions/observability/apps/api-keys.md) (`Authorization: "ApiKey an_api_key"`). +7. Environment-specific configuration parameters can be conveniently passed in as environment variables documented [here](https://opentelemetry.io/docs/collector/configuration/#environment-variables) (e.g. `ELASTIC_APM_SERVER_ENDPOINT` and `ELASTIC_APM_SECRET_TOKEN`). +8. [preview] To send OpenTelemetry logs to {{stack}} version 8.0+, declare a `logs` pipeline. + +::: + +:::{tab-item} Serverless +:sync: serverless + +```yaml +receivers: <1> + # ... + otlp: + +processors: <2> + # ... + memory_limiter: + check_interval: 1s + limit_mib: 2000 + batch: + +exporters: + logging: + loglevel: warn <3> + otlp/elastic: <4> + # Elastic https endpoint without the "https://" prefix + endpoint: "${ELASTIC_APM_SERVER_ENDPOINT}" <5> <7> + headers: + # Elastic API key + Authorization: "ApiKey ${ELASTIC_APM_API_KEY}" <6> <7> + +service: + pipelines: + traces: + receivers: [otlp] + processors: [..., memory_limiter, batch] + exporters: [logging, otlp/elastic] + metrics: + receivers: [otlp] + processors: [..., memory_limiter, batch] + exporters: [logging, otlp/elastic] + logs: <8> + receivers: [otlp] + processors: [..., memory_limiter, batch] + exporters: [logging, otlp/elastic] +``` + +1. The receivers, like the [OTLP receiver](https://github.com/open-telemetry/opentelemetry-collector/tree/main/receiver/otlpreceiver), that forward data emitted by APM agents, or the [host metrics receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/hostmetricsreceiver). +2. We recommend using the [Batch processor](https://github.com/open-telemetry/opentelemetry-collector/blob/main/processor/batchprocessor/README.md) and the [memory limiter processor](https://github.com/open-telemetry/opentelemetry-collector/blob/main/processor/memorylimiterprocessor/README.md). For more information, see [recommended processors](https://github.com/open-telemetry/opentelemetry-collector/blob/main/processor/README.md#recommended-processors). +3. The [logging exporter](https://github.com/open-telemetry/opentelemetry-collector/tree/main/exporter/loggingexporter) is helpful for troubleshooting and supports various logging levels, like `debug`, `info`, `warn`, and `error`. +4. {{obs-serverless}} endpoint configuration. Elastic supports a ProtoBuf payload via both the OTLP protocol over gRPC transport [(OTLP/gRPC)](https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/protocol/otlp.md#otlpgrpc) and the OTLP protocol over HTTP transport [(OTLP/HTTP)](https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/protocol/otlp.md#otlphttp). To learn more about these exporters, see the OpenTelemetry Collector documentation: [OTLP/HTTP Exporter](https://github.com/open-telemetry/opentelemetry-collector/tree/main/exporter/otlphttpexporter) or [OTLP/gRPC exporter](https://github.com/open-telemetry/opentelemetry-collector/tree/main/exporter/otlpexporter). +5. Hostname and port of the Elastic endpoint. For example, `elastic-apm-server:8200`. +6. Credential for Elastic APM API key authorization (`Authorization: "ApiKey an_api_key"`). +7. Environment-specific configuration parameters can be conveniently passed in as environment variables documented [here](https://opentelemetry.io/docs/collector/configuration/#configuration-environment-variables) (e.g. `ELASTIC_APM_SERVER_ENDPOINT` and `ELASTIC_APM_API_KEY`). +8. [preview] To send OpenTelemetry logs to your project, declare a `logs` pipeline. + +::: + +:::: + + +You’re now ready to export traces and metrics from your services and applications. + +::::{tip} +When using the OpenTelemetry Collector, you should always prefer sending data via the [`OTLP` exporter](https://github.com/open-telemetry/opentelemetry-collector/tree/main/exporter/otlphttpexporter). Using other methods, like the [`elasticsearch` exporter](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/exporter/elasticsearchexporter), will bypass all of the validation and data processing that Elastic performs. In addition, your data will not be viewable in your Observability project if you use the `elasticsearch` exporter. +:::: + + +## Send data from an upstream OpenTelemetry SDK [apm-instrument-apps-otel] + +::::{note} +This document outlines how to send data directly from an upstream OpenTelemetry SDK to Elastic, which is appropriate when getting started. However, in many cases you should use the OpenTelemetry SDK to send data to an OpenTelemetry Collector that processes and exports data to Elastic. Read more about when and how to use a collector in the [OpenTelemetry documentation](https://opentelemetry.io/docs/collector/#when-to-use-a-collector). + +:::: + + +To export traces and metrics to Elastic, instrument your services and applications with the OpenTelemetry API, SDK, or both. For example, if you are a Java developer, you need to instrument your Java app with the [OpenTelemetry agent for Java](https://github.com/open-telemetry/opentelemetry-java-instrumentation). See the [OpenTelemetry Instrumentation guides](https://opentelemetry.io/docs/instrumentation/) to download the OpenTelemetry agent or SDK for your language. + +Define environment variables to configure the OpenTelemetry agent or SDK and enable communication with Elastic APM. For example, if you are instrumenting a Java app, define the following environment variables: + +::::{tab-set} +:group: stack-serverless + +:::{tab-item} Elastic Stack v9 +:sync: stack + +```bash +export OTEL_RESOURCE_ATTRIBUTES=service.name=checkoutService,service.version=1.1,deployment.environment=production +export OTEL_EXPORTER_OTLP_ENDPOINT=https://apm_server_url:8200 +export OTEL_EXPORTER_OTLP_HEADERS="Authorization=Bearer an_apm_secret_token" +export OTEL_METRICS_EXPORTER="otlp" \ +export OTEL_LOGS_EXPORTER="otlp" \ <1> +java -javaagent:/path/to/opentelemetry-javaagent-all.jar \ + -classpath lib/*:classes/ \ + com.mycompany.checkout.CheckoutServiceServer +``` + +1. [preview] The OpenTelemetry logs intake via APM Server is currently in technical preview.`OTEL_RESOURCE_ATTRIBUTES` +: Fields that describe the service and the environment that the service runs in. See [resource attributes](../../../solutions/observability/apps/resource-atrributes.md) for more information. + +`OTEL_EXPORTER_OTLP_ENDPOINT` +: APM Server URL. The host and port that APM Server listens for events on. + +`OTEL_EXPORTER_OTLP_HEADERS` +: Authorization header that includes the Elastic APM Secret token or API key: `"Authorization=Bearer an_apm_secret_token"` or `"Authorization=ApiKey an_api_key"`. + + For information on how to format an API key, see [API keys](../../../solutions/observability/apps/api-keys.md). + + Please note the required space between `Bearer` and `an_apm_secret_token`, and `ApiKey` and `an_api_key`. + + ::::{note} + If you are using a version of the Python OpenTelemetry agent *before* 1.27.0, the content of the header *must* be URL-encoded. You can use the Python standard library’s `urllib.parse.quote` function to encode the content of the header. + :::: + + +`OTEL_METRICS_EXPORTER` +: Metrics exporter to use. See [exporter selection](https://opentelemetry.io/docs/specs/otel/configuration/sdk-environment-variables/#exporter-selection) for more information. + +`OTEL_LOGS_EXPORTER` +: Logs exporter to use. See [exporter selection](https://opentelemetry.io/docs/specs/otel/configuration/sdk-environment-variables/#exporter-selection) for more information. + +::: + +:::{tab-item} Serverless +:sync: serverless + +```bash +export OTEL_RESOURCE_ATTRIBUTES=service.name=checkoutService,service.version=1.1,deployment.environment=production +export OTEL_EXPORTER_OTLP_ENDPOINT=https://apm_server_url:8200 +export OTEL_EXPORTER_OTLP_HEADERS="Authorization=ApiKey an_apm_api_key" +export OTEL_METRICS_EXPORTER="otlp" \ +export OTEL_LOGS_EXPORTER="otlp" \ <1> +java -javaagent:/path/to/opentelemetry-javaagent-all.jar \ + -classpath lib/*:classes/ \ + com.mycompany.checkout.CheckoutServiceServer +``` + +1. [preview] The OpenTelemetry logs intake via Elastic is currently in technical preview.`OTEL_RESOURCE_ATTRIBUTES` +: Fields that describe the service and the environment that the service runs in. See [resource attributes](../../../solutions/observability/apps/resource-atrributes.md) for more information. + +`OTEL_EXPORTER_OTLP_ENDPOINT` +: Elastic URL. The host and port that Elastic listens for APM events on. + +`OTEL_EXPORTER_OTLP_HEADERS` +: Authorization header that includes the Elastic APM API key: `"Authorization=ApiKey an_api_key"`. Note the required space between `ApiKey` and `an_api_key`. + + For information on how to format an API key, refer to [Secure communication with APM agents](../../../solutions/observability/apps/use-apm-securely.md). + + ::::{note} + If you are using a version of the Python OpenTelemetry agent *before* 1.27.0, the content of the header *must* be URL-encoded. You can use the Python standard library’s `urllib.parse.quote` function to encode the content of the header. + + :::: + + +`OTEL_METRICS_EXPORTER` +: Metrics exporter to use. See [exporter selection](https://opentelemetry.io/docs/specs/otel/configuration/sdk-environment-variables/#exporter-selection) for more information. + +`OTEL_LOGS_EXPORTER` +: Logs exporter to use. See [exporter selection](https://opentelemetry.io/docs/specs/otel/configuration/sdk-environment-variables/#exporter-selection) for more information. + + +::: + +:::: + + +You are now ready to collect traces and [metrics](../../../solutions/observability/apps/collect-metrics.md) before [verifying metrics](../../../solutions/observability/apps/collect-metrics.md#apm-open-telemetry-verify-metrics) and [visualizing metrics](../../../solutions/observability/apps/collect-metrics.md#apm-open-telemetry-visualize). + + +## Proxy requests to APM Server [apm-open-telemetry-proxy-apm] + +APM Server supports both the [OTLP/gRPC](https://opentelemetry.io/docs/specs/otlp/#otlpgrpc) and [OTLP/HTTP](https://opentelemetry.io/docs/specs/otlp/#otlphttp) protocol on the same port as Elastic APM agent requests. For ease of setup, we recommend using OTLP/HTTP when proxying or load balancing requests to Elastic. + +If you use the OTLP/gRPC protocol, requests to Elastic must use either HTTP/2 over TLS or HTTP/2 Cleartext (H2C). No matter which protocol is used, OTLP/gRPC requests will have the header: `"Content-Type: application/grpc"`. + +When using a layer 7 (L7) proxy like AWS ALB, requests must be proxied in a way that ensures requests to Elastic follow the rules outlined above. For example, with ALB you can create rules to select an alternative backend protocol based on the headers of requests coming into ALB. In this example, you’d select the gRPC protocol when the `"Content-Type: application/grpc"` header exists on a request. + +For more information on how to configure an AWS ALB to support gRPC, see this AWS blog post: [Application Load Balancer Support for End-to-End HTTP/2 and gRPC](https://aws.amazon.com/blogs/aws/new-application-load-balancer-support-for-end-to-end-http-2-and-grpc/). + +For more information on how APM Server services gRPC requests, see [Muxing gRPC and HTTP/1.1](https://github.com/elastic/apm-server/blob/main/dev_docs/otel.md#muxing-grpc-and-http11). + + +## Next steps [apm-open-telemetry-direct-next] + +* [Collect metrics](../../../solutions/observability/apps/collect-metrics.md) +* Add [Resource attributes](../../../solutions/observability/apps/resource-atrributes.md) +* Learn about the [limitations of this integration](../../../solutions/observability/apps/limitations.md) \ No newline at end of file diff --git a/solutions/observability/apps/use-advanced-queries-on-application-data.md b/solutions/observability/apps/use-advanced-queries-on-application-data.md index 4088506a60..69f7fa9699 100644 --- a/solutions/observability/apps/use-advanced-queries-on-application-data.md +++ b/solutions/observability/apps/use-advanced-queries-on-application-data.md @@ -2,13 +2,73 @@ mapped_urls: - https://www.elastic.co/guide/en/observability/current/apm-advanced-queries.html - https://www.elastic.co/guide/en/serverless/current/observability-apm-query-your-data.html + +navigation_title: "Advanced queries" --- -# Use advanced queries on your application data +# Use advanced queries on your application data [apm-advanced-queries] + + +Querying your APM data is an essential tool that can make finding bottlenecks in your code even more straightforward. + +Using the query bar, a powerful data query feature, you can pass advanced queries on your data to filter on specific pieces of information you’re interested in. APM queries entered into the query bar are added as parameters to the URL, so it’s easy to share a specific query or view with others. + +The query bar comes with a handy autocomplete that helps find the fields and even provides suggestions to the data they include. You can select the query bar and hit the down arrow on your keyboard to begin scanning recommendations. + +When you type, you can begin to see some of the transaction fields available for filtering: + +:::{image} ../../../images/observability-apm-query-bar.png +:alt: Example of the Kibana Query bar in Applications UI in Kibana +:class: screenshot +::: + +::::{tip} +To learn more about the {{kib}} query language capabilities, see the [Kibana Query Language Enhancements](../../../explore-analyze/query-filter/languages/kql.md) documentation. + +:::: + + +### Applications UI queries [apm-app-queries] + +APM queries can be handy for removing noise from your data in the [Services](../../../solutions/observability/apps/services.md), [Transactions](../../../solutions/observability/apps/transactions-2.md), [Errors](../../../solutions/observability/apps/errors-2.md), [Metrics](../../../solutions/observability/apps/metrics-2.md), and [Traces](../../../solutions/observability/apps/traces-2.md) views. + +For example, in the **Services** view, you can quickly view a list of all the instrumented services running on your production environment: `service.environment : production`. Or filter the list by including the APM agent’s name and the host it’s running on: `service.environment : "production" and agent.name : "java" and host.name : "prod-server1"`. + +On the **Traces** view, you might want to view failed transaction results from any of your running containers: `transaction.result :"FAILURE" and container.id : *`. + +On the **Transactions** view, you may want to list only the slower transactions than a specified time threshold: `transaction.duration.us > 2000000`. Or filter the list by including the service version and the Kubernetes pod it’s running on: `transaction.duration.us > 2000000 and service.version : "7.12.0" and kubernetes.pod.name : "pod-5468b47f57-pqk2m"`. + + +## Querying in Discover [discover-advanced-queries] + +Alternatively, you can query your APM documents in [**Discover**](../../../explore-analyze/discover.md). Querying documents in **Discover** works the same way as queries in the Applications UI, and **Discover** supports all of the example APM queries shown on this page. + + +### Discover queries [discover-queries] + +One example where you may want to make use of **Discover** is to view *all* transactions for an endpoint instead of just a sample. + +Use the Applications UI to find a transaction name and time bucket that you’re interested in learning more about. Then, switch to **Discover** and make a search: + +```shell +processor.event: "transaction" AND transaction.name: "<TRANSACTION_NAME_HERE>" and transaction.duration.us > 13000 and transaction.duration.us < 14000` +``` + +In this example, we’re interested in viewing all of the `APIRestController#customers` transactions that took between 13 and 14 milliseconds. Here’s what Discover returns: + +:::{image} ../../../images/observability-advanced-discover.png +:alt: View all transactions in bucket +:class: screenshot +::: -% What needs to be done: Align serverless/stateful +You can now explore the data until you find a specific transaction that you’re interested in. Copy that transaction’s `transaction.id` and paste it into APM to view the data in the context of the APM: -% Use migrated content from existing pages that map to this page: +:::{image} ../../../images/observability-specific-transaction-search.png +:alt: View specific transaction in Applications UI +:class: screenshot +::: -% - [ ] ./raw-migrated-files/observability-docs/observability/apm-advanced-queries.md -% - [ ] ./raw-migrated-files/docs-content/serverless/observability-apm-query-your-data.md \ No newline at end of file +:::{image} ../../../images/observability-specific-transaction.png +:alt: View specific transaction in Applications UI +:class: screenshot +::: \ No newline at end of file diff --git a/solutions/observability/apps/use-apm-securely.md b/solutions/observability/apps/use-apm-securely.md index fb599530a5..49037714c9 100644 --- a/solutions/observability/apps/use-apm-securely.md +++ b/solutions/observability/apps/use-apm-securely.md @@ -4,15 +4,13 @@ mapped_urls: - https://www.elastic.co/guide/en/serverless/current/observability-apm-keep-data-secure.html --- -# Use APM securely +# Use APM securely [apm-securing-apm-server] -% What needs to be done: Align serverless/stateful +When setting up Elastic APM, it’s critical to ensure that application data is secure from start to finish. You should approach securing your application data from different perspectives: -% Use migrated content from existing pages that map to this page: - -% - [ ] ./raw-migrated-files/observability-docs/observability/apm-securing-apm-server.md -% - [ ] ./raw-migrated-files/docs-content/serverless/observability-apm-keep-data-secure.md - -% Internal links rely on the following IDs being on this page (e.g. as a heading ID, paragraph ID, etc): - -$$$observability-apm-keep-data-secure-secure-communication-with-apm-agents$$$ \ No newline at end of file +| | | +| --- | --- | +| **What kind of data is collected?** | Ensure that data doesn’t contain sensitive information like passwords, credit card numbers, health data, or other identifiable information.<br> Read more in [Secure data](../../../solutions/observability/apps/application-data-security.md). | +| **How do APM agents and {{agent}} communicate?** | Ensure that any communication between APM agents and {{agent}} are both encrypted and authenticated.<br> Read more in [Secure communication with APM agents](../../../solutions/observability/apps/secure-communication-with-apm-agents.md). | +| **How do APM Server and the {{stack}} communicate?** | Use role-based access control to grant APM Server users access to secured resources. The roles that you set up depend on your organization’s security requirements and the minimum privileges required to use specific features.<br> Read more in [Secure communication with the {{stack}}](../../../solutions/observability/apps/secure-communication-with-elastic-stack.md). | +| **Who can use the Applications UI?** | Use role-based access control to grant users access to features of the Applications UI.<br> Read more in [Secure access to the Applications UI](../../../solutions/observability/apps/secure-access-to-applications-ui.md). | \ No newline at end of file diff --git a/solutions/observability/apps/use-opentelemetry-with-apm.md b/solutions/observability/apps/use-opentelemetry-with-apm.md index 991b3c18e1..8c06adefb0 100644 --- a/solutions/observability/apps/use-opentelemetry-with-apm.md +++ b/solutions/observability/apps/use-opentelemetry-with-apm.md @@ -2,31 +2,117 @@ mapped_urls: - https://www.elastic.co/guide/en/serverless/current/observability-apm-agents-opentelemetry.html - https://www.elastic.co/guide/en/observability/current/apm-open-telemetry.html + +navigation_title: "OpenTelemetry" --- -# Use OpenTelemetry with APM +# Use OpenTelemetry with APM [apm-open-telemetry] + + +::::{note} +For a complete overview of using OpenTelemetry with Elastic, explore** [**Elastic Distributions of OpenTelemetry**](https://github.com/elastic/opentelemetry). + +:::: + + +[OpenTelemetry](https://opentelemetry.io/docs/concepts/what-is-opentelemetry/) is a set of APIs, SDKs, tooling, and integrations that enable the capture and management of telemetry data from your services and applications. + +Elastic integrates with OpenTelemetry, allowing you to reuse your existing instrumentation to easily send observability data to the {{stack}}. There are several ways to integrate OpenTelemetry with the {{stack}}: + +* [Elastic Distributions of OpenTelemetry language SDKs](../../../solutions/observability/apps/use-opentelemetry-with-apm.md#apm-otel-elastic-distros) +* [Upstream OpenTelemetry API/SDK + Elastic APM agent](../../../solutions/observability/apps/use-opentelemetry-with-apm.md#apm-otel-api-sdk-elastic-agent) +* [Upstream OpenTelemetry Collector and language SDKs](../../../solutions/observability/apps/use-opentelemetry-with-apm.md#apm-otel-upstream) +* [AWS Lambda collector exporter](../../../solutions/observability/apps/use-opentelemetry-with-apm.md#apm-otel-lambda) + + +## Elastic Distributions of OpenTelemetry language SDKs [apm-otel-elastic-distros] + +::::{warning} +This functionality is in technical preview and may be changed or removed in a future release. Elastic will work to fix any issues, but features in technical preview are not subject to the support SLA of official GA features. +:::: + + +Elastic offers several distributions of OpenTelemetry language SDKs. A *distribution* is a customized version of an upstream OpenTelemetry repository. Each Elastic Distribution of OpenTelemetry is a customized version of an [OpenTelemetry language SDK](https://opentelemetry.io/docs/languages/). + +:::{image} ../../../images/observability-apm-otel-distro.png +:alt: apm otel distro +:class: screenshot +::: + +With an Elastic Distribution of OpenTelemetry language SDK you have access to all the features of the OpenTelemetry SDK that it customizes, plus: + +* You may get access to SDK improvements and bug fixes contributed by the Elastic team *before* the changes are available upstream in the OpenTelemetry repositories. +* The distribution preconfigures the collection of tracing and metrics signals, applying some opinionated defaults, such as which sources are collected by default. + +Get started with an Elastic Distribution of OpenTelemetry language SDK: + +* [**Elastic Distribution of OpenTelemetry Java →**](https://github.com/elastic/elastic-otel-java) +* [preview] [**Elastic Distribution of OpenTelemetry .NET →**](https://github.com/elastic/elastic-otel-dotnet) +* [preview] [**Elastic Distribution of OpenTelemetry Node.js →**](https://github.com/elastic/elastic-otel-node) +* [preview] [**Elastic Distribution of OpenTelemetry Python →**](https://github.com/elastic/elastic-otel-python) +* [preview] [**Elastic Distribution of OpenTelemetry PHP →**](https://github.com/elastic/elastic-otel-php) + +::::{note} +For more details about OpenTelemetry distributions in general, visit the [OpenTelemetry documentation](https://opentelemetry.io/docs/concepts/distributions). + +:::: + + +## Upstream OpenTelemetry API/SDK + Elastic APM agent [apm-otel-api-sdk-elastic-agent] + +Use the OpenTelemetry API/SDKs with [Elastic APM agents](../../../solutions/observability/apps/fleet-managed-apm-server.md#_step_3_install_apm_agents) to translate OpenTelemetry API calls to Elastic APM API calls. + +:::{image} ../../../images/observability-apm-otel-api-sdk-elastic-agent.png +:alt: apm otel api sdk elastic agent +:class: screenshot +::: + +This allows you to reuse your existing OpenTelemetry instrumentation to create Elastic APM transactions and spans — ​avoiding vendor lock-in and having to redo manual instrumentation. + +However, not all features of the OpenTelemetry API are supported when using this approach, and not all Elastic APM agents support this approach. + +Find more details about how to use an OpenTelemetry API or SDK with an Elastic APM agent and which OpenTelemetry API features are supported in the APM agent documentation: + +* [**APM Java agent →**](https://www.elastic.co/guide/en/apm/agent/java/current/opentelemetry-bridge.html) +* [**APM .NET agent →**](https://www.elastic.co/guide/en/apm/agent/dotnet/current/opentelemetry-bridge.html) +* [**APM Node.js agent →**](https://www.elastic.co/guide/en/apm/agent/nodejs/current/opentelemetry-bridge.html) +* [**APM Python agent →**](https://www.elastic.co/guide/en/apm/agent/python/current/opentelemetry-bridge.html) + + +## Upstream OpenTelemetry Collector and language SDKs [apm-otel-upstream] + +The {{stack}} natively supports the OpenTelemetry protocol (OTLP). This means trace data and metrics collected from your applications and infrastructure by an OpenTelemetry Collector or OpenTelemetry language SDK can be sent to the {{stack}}. + +You can set up an [OpenTelemetry Collector](https://opentelemetry.io/docs/collector/), instrument your application with an [OpenTelemetry language SDK](https://opentelemetry.io/docs/languages/) that sends data to the collector, and use the collector to process and export the data to APM Server. + +:::{image} ../../../images/observability-apm-otel-api-sdk-collector.png +:alt: apm otel api sdk collector +:class: screenshot +::: + +::::{note} +It’s also possible to send data directly to APM Server from an upstream OpenTelemetry SDK. You might do this during development or if you’re monitoring a small-scale application. Read more about when to use a collector in the [OpenTelemetry documentation](https://opentelemetry.io/docs/collector/#when-to-use-a-collector). -% What needs to be done: Align serverless/stateful +:::: -% Use migrated content from existing pages that map to this page: -% - [ ] ./raw-migrated-files/docs-content/serverless/observability-apm-agents-opentelemetry.md -% - [ ] ./raw-migrated-files/observability-docs/observability/apm-open-telemetry.md +This approach works well when you need to instrument a technology that Elastic doesn’t provide a solution for. For example, if you want to instrument C or C++ you could use the [OpenTelemetry C++ client](https://github.com/open-telemetry/opentelemetry-cpp). -% Internal links rely on the following IDs being on this page (e.g. as a heading ID, paragraph ID, etc): +However, there are some limitations when using collectors and language SDKs built and maintained by OpenTelemetry, including: -$$$apm-otel-api-sdk-elastic-agent$$$ +* Elastic can’t provide implementation support on how to use upstream OpenTelemetry tools. +* You won’t have access to Elastic enterprise APM features. +* You may experience problems with performance efficiency. -$$$apm-otel-elastic-distros$$$ +For more on the limitations associated with using upstream OpenTelemetry tools, refer to [Limitations](../../../solutions/observability/apps/limitations.md). -$$$apm-otel-lambda$$$ +[**Get started with upstream OpenTelemetry Collectors and language SDKs →**](../../../solutions/observability/apps/upstream-opentelemetry-collectors-language-sdks.md) -$$$apm-otel-upstream$$$ -$$$observability-apm-agents-opentelemetry-aws-lambda-collector-exporter$$$ +## AWS Lambda collector exporter [apm-otel-lambda] -$$$observability-apm-agents-opentelemetry-elastic-distributions-of-opentelemetry-language-sdks$$$ +AWS Lambda functions can be instrumented with OpenTelemetry and monitored with Elastic {{observability}} or {{obs-serverless}}. -$$$observability-apm-agents-opentelemetry-upstream-opentelemetry-apisdk-elastic-apm-agent$$$ +To get started, follow the official AWS Distro for OpenTelemetry Lambda documentation, and configure the OpenTelemetry Collector to output traces and metrics to your Elastic cluster: -$$$observability-apm-agents-opentelemetry-upstream-opentelemetry-collector-and-language-sdks$$$ \ No newline at end of file +[**Get started with the AWS Distro for OpenTelemetry Lambda**](https://aws-otel.github.io/docs/getting-started/lambda) \ No newline at end of file diff --git a/solutions/observability/apps/use-synthetics-cli.md b/solutions/observability/apps/use-synthetics-cli.md index b413921250..907daeb417 100644 --- a/solutions/observability/apps/use-synthetics-cli.md +++ b/solutions/observability/apps/use-synthetics-cli.md @@ -2,25 +2,315 @@ mapped_urls: - https://www.elastic.co/guide/en/observability/current/synthetics-command-reference.html - https://www.elastic.co/guide/en/serverless/current/observability-synthetics-command-reference.html + +navigation_title: "Use the CLI" --- -# Use the Synthetics CLI +# Use the Synthetics CLI [synthetics-command-reference] + + + +## `@elastic/synthetics` [elastic-synthetics-command] + +Elastic uses the [@elastic/synthetics](https://www.npmjs.com/package/@elastic/synthetics) Node.js library to run synthetic browser tests and report the test results. The library also provides a CLI to help you scaffold, develop/run tests locally, and push tests to {{kib}}. + +```sh +npx @elastic/synthetics [options] [files] [dir] +``` + +You will not need to use most command line flags. However, there are some you may find useful: + +`--match <string>` +: run tests with a name or tags that match the given glob pattern. + +`--tags Array<string>` +: run tests with the given tags that match the given glob pattern. + +`--pattern <string>` +: RegExp pattern to match journey files in the current working directory. Defaults to `/*.journey.(ts|js)$/`, which matches files ending with `.journey.ts` or `.journey.js`. + +`--params <jsonstring>` +: JSON object that defines any variables your tests require. Read more in [Work with params and secrets](../../../solutions/observability/apps/work-with-params-secrets.md). + + Params passed will be merged with params defined in your [`synthetics.config.js` file](../../../solutions/observability/apps/configure-synthetics-projects.md#synthetics-configuration-params). Params defined via the CLI take precedence. + + +`--playwright-options <jsonstring>` +: JSON object to pass in custom Playwright options for the agent. For more details on relevant Playwright options, refer to the [the configuration docs](../../../solutions/observability/apps/configure-synthetics-projects.md#synthetics-configuration-playwright-options). + + Options passed will be merged with Playwright options defined in your [`synthetics.config.js` file](../../../solutions/observability/apps/configure-synthetics-projects.md#synthetics-configuration-playwright-options). Options defined via the CLI take precedence. + + +`--screenshots <on|off|only-on-failure>` +: Control whether or not to capture screenshots at the end of each step. Options include `'on'`, `'off'`, or `'only-on-failure'`. + + This can also be set in the configuration file using [`monitor.screenshot`](../../../solutions/observability/apps/configure-synthetics-projects.md#synthetics-configuration-monitor). The value defined via the CLI will take precedence. + + +`-c, --config <string>` +: Path to the configuration file. By default, test runner looks for a `synthetics.config.(js|ts)` file in the current directory. Synthetics configuration provides options to configure how your tests are run and pushed to Elastic. Allowed options are described in the [configuration file](../../../solutions/observability/apps/configure-synthetics-projects.md). + +`--reporter <json|junit|buildkite-cli|default>` +: One of `json`, `junit`, `buildkite-cli`, or `default`. Use the JUnit or Buildkite reporter to provide easily parsed output to CI systems. + +`--inline` +: Instead of reading from a file, `cat` inline scripted journeys and pipe them through `stdin`. For example, `cat path/to/file.js | npx @elastic/synthetics --inline`. + +`--no-throttling` +: Does not apply throttling. + + Throttling can also be disabled in the configuration file using [`monitor.throttling`](../../../solutions/observability/apps/configure-synthetics-projects.md#synthetics-configuration-monitor). The value defined via the CLI will take precedence. + + +::::{note} +Network throttling for browser based monitors is disabled. See this [documention](https://github.com/elastic/synthetics/blob/main/docs/throttling.md) for more details. + +:::: + + +`--no-headless` +: Runs with the browser in headful mode. + + This is the same as setting [Playwright’s `headless` option](https://playwright.dev/docs/api/class-testoptions#test-options-headless) to `false` by running `--playwright-options '{"headless": false}'`. + + +::::{note} +Headful mode should only be used locally to see the browser and interact with DOM elements directly for testing purposes. Do not attempt to run in headful mode when running through Elastic’s global managed testing infrastructure or {{private-location}}s as this is not supported. + +:::: + + +`-h, --help` +: Shows help for the `npx @elastic/synthetics` command. + +::::{note} +The `--pattern`, `--tags`, and `--match` flags for filtering are only supported when you run synthetic tests locally or push them to Elastic. Filtering is *not* supported in any other subcommands like `init` and `locations`. + +:::: + + +::::{note} +For debugging synthetic tests locally, you can set an environment variable, `DEBUG=synthetics npx @elastic/synthetics`, to capture Synthetics agent logs. + +:::: + + + +## `@elastic/synthetics init` [elastic-synthetics-init-command] + +Scaffold a new Synthetics project using Elastic Synthetics. + +This will create a template Node.js project that includes the synthetics agent, required dependencies, a synthetics configuration file, and example browser and lightweight monitor files. These files can be edited and then pushed to Elastic to create monitors. + +```sh +npx @elastic/synthetics init <name-of-project> +``` + +Read more about what’s included in a template Synthetics project in [Create a Synthetics project](../../../solutions/observability/apps/create-monitors-with-project-monitors.md). + + +## `@elastic/synthetics push` [elastic-synthetics-push-command] + +Create monitors by using your local journeys. By default, running `push` command will use the `project` settings field from the `synthetics.config.ts` file, which is set up using the `init` command. However, you can override these settings using the CLI flags. + +```sh +SYNTHETICS_API_KEY=<api-key> npx @elastic/synthetics push --url <kibana-url> --id <id|name> +``` + +::::{note} +The `push` command includes interactive prompts to prevent you from accidentally deleting or duplicating monitors. You will see a prompt when: + +* You `push` a Synthetics project that used to contain one or more monitors but either no longer contains previously running monitors or has any monitors. Select `yes` to delete the monitors associated with the project ID being pushed. +* You `push` a Synthetics project that’s already been pushed using one Synthetics project ID and then try to `push` it using a *different* ID. Select `yes` to create duplicates of all monitors in the project. You can set `DEBUG=synthetics` environment variable to capture the deleted monitors. + +:::: + + +::::{note} +If the journey contains external NPM packages other than the `@elastic/synthetics`, those packages will be bundled along with the journey code when the `push` command is invoked. However there are some limitations when using external packages: + +* Bundled journeys after compression should not be more than 1500 Kilobytes. +* Native node modules will not work as expected due to platform inconsistency. +* Uploading files in journey scripts(via locator.setInputFiles) is not supported. + +:::: + + +`--auth <string>` +: API key used for [authentication](../../../deploy-manage/api-keys/elasticsearch-api-keys.md). You can also set the API key via the `SYNTHETICS_API_KEY` environment variable. + + If you are pushing to a [{{private-location}}](../../../solutions/observability/apps/create-monitors-in-synthetics-app.md), you must use an API key generated in 8.4 or higher. + + On {{stack}}, you must be logged into {{kib}} as a user with the privileges described in [Writer role](../../../solutions/observability/apps/writer-role.md) to create an API key. + + On {{obs-serverless}}, you must be logged in as a user with [Editor](../../../solutions/observability/apps/grant-users-access-to-secured-resources.md) access to create an API key. + + +`--id <string>` +: A unique id associated with your project. It will be used for logically grouping monitors. + + If you used [`init` to create a project](../../../solutions/observability/apps/use-synthetics-cli.md#elastic-synthetics-init-command), this is the `<name-of-project>` you specified. + + This can also be set in the configuration file using [`project.id`](../../../solutions/observability/apps/configure-synthetics-projects.md#synthetics-configuration-project). The value defined via the CLI will take precedence. + + +`--url <string>` +: The URL for the deployment or Observability serverless project to which you want to upload the monitors. + + This can also be set in the configuration file using [`project.url`](../../../solutions/observability/apps/configure-synthetics-projects.md#synthetics-configuration-project). The value defined via the CLI will take precedence. + +% Stateful only for --space + +`--space <string>` +: The identifier of the target [{{kib}} space](../../../deploy-manage/manage-spaces.md) for the pushed monitors. Spaces help you organize pushed monitors. Pushes to the "default" space if not specified. + + This can also be set in the configuration file using [`project.space`](../../../solutions/observability/apps/configure-synthetics-projects.md#synthetics-configuration-project). The value defined via the CLI will take precedence. + + +`--schedule <number>` +: The interval (in minutes) at which the monitor should run. + + This can also be set in the configuration file using [`monitor.schedule`](../../../solutions/observability/apps/configure-synthetics-projects.md#synthetics-configuration-monitor). The value defined via the CLI will take precedence. + + +[`--locations Array<SyntheticsLocationsType>`](https://github.com/elastic/synthetics/blob/v1.3.0/src/locations/public-locations.ts#L28-L37) +: Where to deploy the monitor. Monitors can be deployed in multiple locations so that you can detect differences in availability and response times across those locations. + + To list available locations, refer to [`@elastic/synthetics locations`](../../../solutions/observability/apps/use-synthetics-cli.md#elastic-synthetics-locations-command). + + This can also be set in the configuration file using [`monitor.locations` in the configuration file](../../../solutions/observability/apps/configure-synthetics-projects.md#synthetics-configuration-monitor). The value defined via the CLI will take precedence. + + +`--private-locations Array<string>` +: The [{{private-location}}s](../../../solutions/observability/apps/monitor-resources-on-private-networks.md) to which the monitors will be deployed. These {{private-location}}s refer to locations hosted and managed by you, whereas `locations` are hosted by Elastic. You can specify a {{private-location}} using the location’s name. + + To list available {{private-location}}s, refer to [`@elastic/synthetics locations`](../../../solutions/observability/apps/use-synthetics-cli.md#elastic-synthetics-locations-command). + + This can also be set in the configuration file using [`monitor.privateLocations` in the configuration file](../../../solutions/observability/apps/configure-synthetics-projects.md#synthetics-configuration-monitor). The value defined via the CLI will take precedence. + + +`--fields <string>` +: A list of key-value pairs that will be sent with each monitor event. The `fields` are appended to {{es}} documents as `labels`, and those labels are displayed in {{kib}} in the *Monitor details* panel in the [individual monitor’s *Overview* tab](../../../solutions/observability/apps/analyze-data-from-synthetic-monitors.md#synthetics-analyze-individual-monitors-overview). + + Example: `--fields '{ "foo": bar", "team": "synthetics" }'` + + This can also be set in the configuration file using [the `monitor.fields` option](../../../solutions/observability/apps/configure-synthetics-projects.md#synthetics-configuration-monitor). The value defined via the CLI will take precedence. + + +`--yes` +: The `push` command includes interactive prompts to prevent you from accidentally deleting or duplicating monitors. If running the CLI non-interactively, you can override these prompts using the `--yes` option. When the `--yes` option is passed to `push`: + + * If you `push` a Synthetics project that used to contain one or more monitors but no longer contains any monitors, all monitors associated with the Synthetics project ID being pushed will be deleted. + * If you `push` a Synthetics project that’s already been pushed using one Synthetics project ID and then try to `push` it using a *different* ID, it will create duplicates of all monitors in the Synthetics project. + + + +## Tag monitors [tagging-and-filtering] + +Synthetics journeys can be tagged with one or more tags. Use tags to filter journeys when running tests locally or pushing them to Elastic. + +To add tags to a single journey, add the `tags` parameter to the `journey` function or use the `monitor.use` method. + +```js +import {journey, monitor} from "@elastic/synthetics"; + +journey({name: "example journey", tags: ["env:qa"] }, ({ page }) => { + monitor.use({ + tags: ["env:qa"] + }) + // Add steps here +}); +``` + +For lightweight monitors, use the `tags` field in the yaml configuration file. + +```yaml +name: example monitor +tags: + - env:qa +``` + +To apply tags to all browser and lightweight monitors, configure using [the `monitor.tags`](../../../solutions/observability/apps/configure-synthetics-projects.md#synthetics-configuration-monitor) field in the `synthetics.config.ts` file. + + +### Filter monitors [_filtering_monitors] + +When running the `npx @elastic/synthetics push` command, you can filter the monitors that are pushed to {{kib}} using the following flags: + +`--tags Array<string>` +: Push monitors with the given tags that match the glob pattern. + +`--match <string>` +: Push monitors with a name or tags that match the glob pattern. + +`--pattern <string>` +: RegExp pattern to match the journey files in the current working directory. Defaults to `/*.journey.(ts|js)$/` for browser monitors and `/.(yml|yaml)$/` for lightweight monitors. + +You can combine these techniques and push the monitors to different Kibana clusters/space based on the tags by using multiple configuration files. + +```sh +npx @elastic/synthetics push --config synthetics.qa.config.ts --tags env:qa +npx @elastic/synthetics push --config synthetics.prod.config.ts --tags env:prod +``` + + +## `@elastic/synthetics locations` [elastic-synthetics-locations-command] + +List all available locations for running synthetics monitors. + +```sh +npx @elastic/synthetics locations --url <host> --auth <api-key> +``` + +Run `npx @elastic/synthetics locations` with no flags to list all the available global locations managed by Elastic for running synthetics monitors. + +To list both locations on Elastic’s global managed infrastructure and {{private-locations}}, include: + +`--url <string>` +: The URL for the {{kib}} deployment or Observability serverless project from which to fetch all available public and {{private-location}}s. + +`--auth <string>` +: API key used for [authentication](../../../deploy-manage/api-keys/elasticsearch-api-keys.md). + +::::{note} +If an administrator has disabled Elastic managed locations for the role you are assigned and you do *not* include `--url` and `--auth`, all global locations managed by Elastic will be listed. However, you will not be able to push to these locations with your API key and will see an error: *You don’t have permission to use Elastic managed global locations*. For more details, refer to the [troubleshooting docs](../../../troubleshoot/observability/troubleshooting-synthetics.md#synthetics-troubleshooting-public-locations-disabled). +:::: + + +## `@elastic/synthetics totp <secret>` [elastic-synthetics-totp-command] + +Generate a Time-based One-Time Password (TOTP) for multifactor authentication (MFA) in Synthetics. + +::::{tab-set} +:group: stack-serverless + +:::{tab-item} Elastic Stack v9 +:sync: stack + +```sh +npx @elastic/synthetics totp <secret> +npx @elastic/synthetics totp <secret> --issuer <string> --label <string> +``` -% What needs to be done: Align serverless/stateful +::: -% Use migrated content from existing pages that map to this page: +:::{tab-item} Serverless +:sync: serverless -% - [ ] ./raw-migrated-files/observability-docs/observability/synthetics-command-reference.md -% - [ ] ./raw-migrated-files/docs-content/serverless/observability-synthetics-command-reference.md +```sh +npx @elastic/synthetics totp <secret> --issuer <issuer> --label <label> +``` -% Internal links rely on the following IDs being on this page (e.g. as a heading ID, paragraph ID, etc): +::: -$$$elastic-synthetics-locations-command$$$ +:::: -$$$elastic-synthetics-push-command$$$ -$$$elastic-synthetics-command$$$ +`secret` +: The encoded secret key used to generate the TOTP. -$$$elastic-synthetics-init-command$$$ +`--issuer <string>` +: Name of the provider or service that is assocaited with the account. -$$$elastic-synthetics-totp-command$$$ \ No newline at end of file +`--label <string>` +: Identifier for the account. Defaults to `SyntheticsTOTP` \ No newline at end of file diff --git a/solutions/observability/apps/use-synthetics-recorder.md b/solutions/observability/apps/use-synthetics-recorder.md index 0d03efd77c..e616f498df 100644 --- a/solutions/observability/apps/use-synthetics-recorder.md +++ b/solutions/observability/apps/use-synthetics-recorder.md @@ -4,15 +4,126 @@ mapped_urls: - https://www.elastic.co/guide/en/serverless/current/observability-synthetics-recorder.html --- -# Use the Synthetics Recorder +# Use the Synthetics Recorder [synthetics-recorder] -% What needs to be done: Align serverless/stateful +::::{important} +As with any script recording technology, the Elastic Synthetics Recorder should be used as a tool to help create the main structure of the script. For simpler sites, you may be able to use the Synthetics Recorder’s output directly to create a synthetic monitor, but for more complex and dynamic sites, or to limit flakiness, you’ll likely need to edit its output before using it to create a monitor. -% Use migrated content from existing pages that map to this page: +:::: -% - [ ] ./raw-migrated-files/observability-docs/observability/synthetics-recorder.md -% - [ ] ./raw-migrated-files/docs-content/serverless/observability-synthetics-recorder.md -% Internal links rely on the following IDs being on this page (e.g. as a heading ID, paragraph ID, etc): +You can use the Synthetics Recorder to [write a synthetic test](../../../solutions/observability/apps/write-synthetic-test.md) by interacting with a web page and exporting journey code that reflects all the actions you took. -$$$synthetics-recorder-test-the-journey$$$ \ No newline at end of file +:::{image} ../../../images/observability-synthetics-create-test-script-recorder.png +:alt: Elastic Synthetics Recorder after recording a journey and clicking Export +:class: screenshot +::: + + +### Set up [synthetics-recorder-set-up] + +For information on how to download the Elastic Synthetics Recorder, go to the [download page](https://github.com/elastic/synthetics-recorder/blob/main/docs/DOWNLOAD.md). + + +## Record a journey [synthetics-recorder-record-a-journey] + +To record a journey: + +1. Enter a starting URL in the search box. This URL will be the starting point of the journey script the recorder will create. +2. Click **Start** or press Enter on your keyboard. This will launch a Chromium window open to the page you specified and start recording. +3. Start interacting with the browser. This can include clicking on text, navigation, focusing on inputs like buttons and text fields, and more. +4. (Optional) You can click **Pause** to temporarily stop recording actions while you continue to interact with the browser. Click again to start recording actions again. Note: It’s especially important to [test the journey](../../../solutions/observability/apps/use-synthetics-recorder.md#synthetics-recorder-test-the-journey) if you paused recording at any point. +5. When you’re done interacting with the browser window, click **Stop** or close the browser to stop recording. + + +## Edit a journey [synthetics-recorder-edit-a-journey] + +Once you’ve started recording, you can use the Synthetics Recorder UI to edit steps and individual actions before generating the journey code. You can also edit the journey after you’ve stopped recording. + + +### Name steps [synthetics-recorder-name-steps] + +Naming steps can help make the resulting journey code easier to understand. If you provide a step name, the name will be used in both the UI and the resulting code. If you don’t name steps, the UI will show "Step 1", "Step 2", and so on, and the resulting code will use the first action in the step as the step text. + +To edit a step name: + +1. Hover over the current step name and click the pencil icon that appears. +2. Edit the text in the text box. +3. Click Return or Enter on your keyboard to save the updated name. + + +### Split into multiple steps [synthetics-recorder-split-into-multiple-steps] + +Steps represent groups of actions that should be completed in a specific order. Breaking a journey into steps can make it easier to read the resulting code. It can also make it easier to interpret results in the Synthetics UI since each step is displayed individually in the UI along with screenshots for convenient debugging and error tracking. + +By default, the Synthetics Recorder will group all actions in a single step, but you can break actions into any number of steps. + +To add a step: + +1. Click the plus icon between two actions to create a new step. +2. (Optional) Consider naming the step. + +Use the trash can icon to delete the step divider, adding the actions from the deleted step into the previous step. + + +### Edit or delete recorded actions [synthetics-recorder-edit-or-delete-recorded-actions] + +You can fine-tune a journey by editing actions that were generated by the recorder. You can’t change the type of command (for example, "click" or "navigate"), but you can change the value that is passed to the command. + +To edit an action: + +1. Hover over an action and click the pencil icon that appears. +2. Edit the value as needed. +3. Click **Save**. + +To delete an action: + +1. Hover over the action you want to delete and click the three dots for more options. +2. Click **Delete action**. + +::::{important} +If you changed or deleted any actions to ensure the journey still works, it’s especially important to [test the journey](../../../solutions/observability/apps/use-synthetics-recorder.md#synthetics-recorder-test-the-journey). +:::: + + + +### Add assertions [synthetics-recorder-add-assertions] + +Assertions can play an important role in effective synthetic journeys by making determinations about the state of the page you are testing. This can include checking if an element is visible or checking the contents of a text field. You can’t generate an assertion just from interacting with the browser window. Instead, you can add assertions between generated actions. + +To add an assertion: + +1. Find the generated action that should be done right before you want to assert a condition. +2. Hover over that action and click the three dots for more options. +3. Click **Add assertion**. This will add a new "assert" action in the UI. +4. Provide the type of assertion, selector, and value. +5. Click **Save**. + +::::{important} +If you added any assertions after you’ve finished recording to ensure the journey still works, it’s especially important to [test the journey](../../../solutions/observability/apps/use-synthetics-recorder.md#synthetics-recorder-test-the-journey). +:::: + + + +## Test the journey [synthetics-recorder-test-the-journey] + +At any point during or after the recording process concludes, you can test your script. + +When you click the **Test** button, Elastic Synthetics will run the journey. As the test runs, the recorder will display results on a per-step basis. If there are any errors that prevent the journey from running, the recorder will display the relevant error message to help you debug. + +::::{important} +If you paused recording, updated actions, or added assertions manually in the recorder it is especially important that you test the journey to verify that the actions work in sequence. +:::: + + + +## Export [synthetics-recorder-export] + +When you are satisfied with journey you’ve created, you can export it from the recorder. + +Click **Export** to view the final journey code. From there you can use the code by: + +* Copy and pasting code containing all steps into a new or existing [Synthetics project](../../../solutions/observability/apps/create-monitors-with-project-monitors.md) or an [inline monitor](../../../solutions/observability/apps/create-monitors-in-synthetics-app.md). +* Click **Export** to save a JavaScript file containing all steps. + +You can also check **Export as project** and either copy and paste or **Export** to get the full journey code including `journey` and imports for all dependencies. \ No newline at end of file diff --git a/solutions/observability/apps/view-analyze-data.md b/solutions/observability/apps/view-analyze-data.md index c42a03d6fd..86bc07f282 100644 --- a/solutions/observability/apps/view-analyze-data.md +++ b/solutions/observability/apps/view-analyze-data.md @@ -4,11 +4,18 @@ mapped_urls: - https://www.elastic.co/guide/en/serverless/current/observability-apm-view-and-analyze-traces.html --- -# View and analyze data +# View and analyze data [apm-view-and-analyze-data] -% What needs to be done: Align serverless/stateful +After you’ve started [sending application data to Elastic](../../../solutions/observability/apps/collect-application-data.md), you can open the Applications UI to view your data in a variety of visualizations and start analyzing data. -% Use migrated content from existing pages that map to this page: +The Applications UI allows you to monitor your software services and applications in real-time. You can visualize detailed performance information on your services, identify and analyze errors, and monitor host-level and APM agent-specific metrics like JVM and Go runtime metrics. -% - [ ] ./raw-migrated-files/observability-docs/observability/apm-view-and-analyze-data.md -% - [ ] ./raw-migrated-files/docs-content/serverless/observability-apm-view-and-analyze-traces.md \ No newline at end of file +Having access to application-level insights with just a few clicks can drastically decrease the time you spend debugging errors, slow response times, and crashes. + +For example, you can see information about response times, requests per minute, and status codes per endpoint. You can even dive into a specific request sample and get a complete waterfall view of what your application is spending its time on. You might see that your bottlenecks are in database queries, cache calls, or external requests. For each incoming request and each application error, you can also see contextual information such as the request header, user information, system values, or custom data that you manually attached to the request. + +To get started with the Applications UI: + +* Start with quick, high-level [overviews](../../../solutions/observability/apps/overviews.md) that show you the overall health and performance of your application. +* [Drill down into data](../../../solutions/observability/apps/drill-down-into-data.md) for specific services or traces to get additional insight into your application. +* Learn how to get the most out of your data by mastering how to [search and filter data](../../../solutions/observability/apps/filter-search-application-data.md), getting tips on [how to interpret data](../../../solutions/observability/apps/interpret-application-data.md), and taking advantage of [machine learning](../../../solutions/observability/apps/integrate-with-machine-learning.md). diff --git a/solutions/observability/apps/work-with-params-secrets.md b/solutions/observability/apps/work-with-params-secrets.md index 7fd81fa67a..ccf01153ca 100644 --- a/solutions/observability/apps/work-with-params-secrets.md +++ b/solutions/observability/apps/work-with-params-secrets.md @@ -4,19 +4,152 @@ mapped_urls: - https://www.elastic.co/guide/en/serverless/current/observability-synthetics-params-secrets.html --- -# Work with params and secrets +# Work with params and secrets [observability-synthetics-params-secrets] -% What needs to be done: Align serverless/stateful +Params allow you to use dynamically defined values in your synthetic monitors. For example, you may want to test a production website with a particular demo account whose password is only known to the team managing the synthetic monitors. -% Use migrated content from existing pages that map to this page: +For more information about security-sensitive use cases, refer to [Working with secrets and sensitive values](../../../solutions/observability/apps/work-with-params-secrets.md#synthetics-secrets-sensitive). -% - [ ] ./raw-migrated-files/observability-docs/observability/synthetics-params-secrets.md -% - [ ] ./raw-migrated-files/docs-content/serverless/observability-synthetics-params-secrets.md -% Internal links rely on the following IDs being on this page (e.g. as a heading ID, paragraph ID, etc): +## Define params [synthetics-params-secrets-define] -$$$synthetics-cli-params$$$ +Param values can be declared by any of the following methods: -$$$synthetics-dynamic-configs$$$ +* In the *Global parameters* tab of the [Synthetics Settings page in an Observability project](../../../solutions/observability/apps/configure-synthetics-settings.md#synthetics-settings-global-parameters). +* Declaring a default value for the parameter in a [configuration file](../../../solutions/observability/apps/work-with-params-secrets.md#synthetics-dynamic-configs). +* Passing the `--params` [CLI argument](../../../solutions/observability/apps/work-with-params-secrets.md#synthetics-cli-params). -$$$synthetics-secrets-sensitive$$$ \ No newline at end of file +::::{note} +If you are creating and managing synthetic monitors using a [Synthetics project](../../../solutions/observability/apps/create-monitors-with-project-monitors.md), you can also use regular environment variables via the standard node `process.env` global object. + +:::: + + +The values in the configuration file are read in the following order: + +1. **Global parameters in an Observability project**: The *Global parameters* set using the UI are read first. +2. **Configuration file**: Then the *Global parameters* are merged with any parameters defined in a configuration file. If a parameter is defined in both the Observability UI **and** a Synthetics project configuration file, the value in the configuration file will be used. +3. **CLI**: Then the parameters defined in the configuration are merged with any parameters passed to the CLI `--params` argument. If a parameter is defined in a Synthetics project configuration file **and** using the CLI argument, the value defined using the CLI will be used. When running a script using the CLI, *Global parameters* defined in {{kib}} or the Observability serverless project have no impact on the test because it won’t have access to {{kib}} or the Observability project. + + +### Global parameters in your Observability project [observability-synthetics-params-secrets-global-parameters-in-your-observability-project] + +From any page in the Synthetics UI: + +1. Go to **Settings**. +2. Go to the **Global parameters** tab. +3. Define parameters. + +:::{image} ../../../images/observability-synthetics-params-secrets-kibana-define.png +:alt: Global parameters tab on the Synthetics Settings page +:class: screenshot +::: + + +### Synthetics project config file [synthetics-dynamic-configs] + +Use a `synthetics.config.js` or `synthetics.config.ts` file to define variables required by your tests. This file should be placed in the root of your Synthetics project. + +```js +export default (env) => { + let my_url = "http://localhost:8080"; + if (env === "production") { + my_url = "https://elastic.github.io/synthetics-demo/" + } + return { + params: { + my_url, + }, + }; +}; +``` + +The example above uses the `env` variable, which corresponds to the value of the `NODE_ENV` environment variable. + + +### CLI argument [synthetics-cli-params] + +To set parameters when running [`npx @elastic/synthetics` on the command line](../../../solutions/observability/apps/use-synthetics-cli.md), use the `--params` or `-p` flag. The provided map is merged over any existing variables defined in the `synthetics.config.{js,ts}` file. + +For example, to override the `my_url` parameter, you would run: + +```sh +npx @elastic/synthetics . --params '{"my_url": "http://localhost:8080"}' +``` + + +## Use params [synthetics-params-secrets-use] + +You can use params in both lightweight and browser monitors created in either a Synthetics project or the Synthetics UI. + + +### In a Synthetics project [synthetics-params-secrets-use-project] + +For lightweight monitors in a Synthetics project, wrap the name of the param in `${}` (for example, `${my_url}`). + +```yaml +- type: http + name: Todos Lightweight + id: todos-lightweight + urls: ["${my_url}"] + schedule: '@every 1m' +``` + +In browser monitors, parameters can be referenced via the `params` property available within the argument to a `journey`, `before`, `beforeAll`, `after`, or `afterAll` callback function. + +Add `params.` before the name of the param (for example, `params.my_url`): + +```js +beforeAll(({params}) => { + console.log(`Visiting ${params.my_url}`) +}) + +journey("My Journey", ({ page, params }) => { + step('launch app', async () => { + await page.goto(params.my_url) <1> + }) +}) +``` + +1. If you are using TypeScript, replace `params.my_url` with `params.my_url as string`. + + + +### In the UI [synthetics-params-secrets-use-ui] + +To use a param in a lightweight monitor that is created in the Synthetics UI, wrap the name of the param in `${}` (for example, `${my_url}`). + +:::{image} ../../../images/serverless-synthetics-params-secrets-kibana-use-lightweight.png +:alt: Use a param in a lightweight monitor created in the Synthetics UI +:class: screenshot +::: + +To use a param in a browser monitor that is created in the Synthetics UI, add `params.` before the name of the param (for example, `params.my_url`). + +:::{image} ../../../images/observability-synthetics-params-secrets-kibana-use-lightweight.png +:alt: Use a param in a lightweight monitor created in the Synthetics UI +:class: screenshot +::: + + +## Working with secrets and sensitive values [synthetics-secrets-sensitive] + +Your synthetics scripts may require the use of passwords or other sensitive secrets that are not known until runtime. + +::::{warning} +Params are viewable in plain-text by administrators and other users with `all` privileges for the Synthetics app. Also note that synthetics scripts have no limitations on accessing these values, and a malicious script author could write a synthetics journey that exfiltrates `params` and other data at runtime. Do **not** use truly sensitive passwords (for example, an admin password or a real credit card) in **any** synthetics tools. Instead, set up limited demo accounts, or fake credit cards with limited functionality. If you want to limit access to parameters, ensure that users who are not supposed to access those values do not have `all` privileges for the Synthetics app, and that any scripts that use those values do not leak them in network requests or screenshots. + +:::: + + +If you are managing monitors with a Synthetics project, you can use environment variables in your `synthetics.config.ts` or `synthetics.config.js` file. + +The example below uses `process.env.MY_URL` to reference a variable named `MY_URL` defined in the environment and assigns its value to a param. That param can then be used in both lightweight and browser monitors that are managed in the Synthetics project: + +```js +export default { + params: { + my_url: process.env.MY_URL + } +}; +``` \ No newline at end of file diff --git a/solutions/observability/apps/write-synthetic-test.md b/solutions/observability/apps/write-synthetic-test.md index ed7280fba4..f0b41878a4 100644 --- a/solutions/observability/apps/write-synthetic-test.md +++ b/solutions/observability/apps/write-synthetic-test.md @@ -4,27 +4,343 @@ mapped_urls: - https://www.elastic.co/guide/en/serverless/current/observability-synthetics-create-test.html --- -# Write a synthetic test +# Write a synthetic test [synthetics-create-test] -% What needs to be done: Align serverless/stateful +After [setting up a Synthetics project](../../../solutions/observability/apps/create-monitors-with-project-monitors.md), you can start writing synthetic tests that check critical actions and requests that an end-user might make on your site. -% Use migrated content from existing pages that map to this page: -% - [ ] ./raw-migrated-files/observability-docs/observability/synthetics-create-test.md -% - [ ] ./raw-migrated-files/docs-content/serverless/observability-synthetics-create-test.md +## Syntax overview [synthetics-syntax] -% Internal links rely on the following IDs being on this page (e.g. as a heading ID, paragraph ID, etc): +To write synthetic tests for your application, you’ll need to know basic JavaScript and [Playwright](https://playwright.dev/) syntax. -$$$before-after$$$ +::::{tip} +[Playwright](https://playwright.dev/) is a browser testing library developed by Microsoft. It’s fast, reliable, and features a modern API that automatically waits for page elements to be ready. +:::: -$$$synthetics-create-journey$$$ -$$$synthetics-create-step$$$ +The synthetics agent exposes an API for creating and running tests, including: -$$$synthetics-make-assertions$$$ +`journey` +: Tests one discrete unit of functionality. Takes two parameters: a `name` (string) and a `callback` (function). Learn more in [Create a journey](../../../solutions/observability/apps/write-synthetic-test.md#synthetics-create-journey). -$$$synthetics-playwright$$$ +`step` +: Actions within a journey that should be completed in a specific order. Takes two parameters: a `name` (string) and a `callback` (function). Learn more in [Add steps](../../../solutions/observability/apps/write-synthetic-test.md#synthetics-create-step). -$$$synthetics-request-param$$$ +`expect` +: Check that a value meets a specific condition. There are several supported checks. Learn more in [Make assertions](../../../solutions/observability/apps/write-synthetic-test.md#synthetics-make-assertions). -$$$synthetics-test-locally$$$ \ No newline at end of file +`beforeAll` +: Runs a provided function once, before any `journey` runs. If the provided function is a promise, the runner will wait for the promise to resolve before invoking the `journey`. Takes one parameter: a `callback` (function). Learn more in [Set up and remove a global state](../../../solutions/observability/apps/write-synthetic-test.md#before-after). + +`before` +: Runs a provided function before a single `journey` runs. Takes one parameter: a `callback` (function). Learn more in [Set up and remove a global state](../../../solutions/observability/apps/write-synthetic-test.md#before-after). + +`afterAll` +: Runs a provided function once, after all the `journey` runs have completed. Takes one parameter: a `callback` (function). Learn more in [Set up and remove a global state](../../../solutions/observability/apps/write-synthetic-test.md#before-after). + +`after` +: Runs a provided function after a single `journey` has completed. Takes one parameter: a `callback` (function). Learn more in [Set up and remove a global state](../../../solutions/observability/apps/write-synthetic-test.md#before-after). + +`monitor` +: The `monitor.use` method allows you to determine a monitor’s configuration on a journey-by-journey basis. If you want two journeys to create monitors with different intervals, for example, you should call `monitor.use` in each of them and set the `schedule` property to different values in each. Note that this is only relevant when using the `push` command to create monitors in {{kib}} an Observability serverless project. Learn more in [Configure individual monitors](../../../solutions/observability/apps/configure-individual-browser-monitors.md). + + +## Create a journey [synthetics-create-journey] + +Create a new file using the `.journey.ts` or `.journey.js` file extension or edit one of the example journey files. + +A *journey* tests one discrete unit of functionality. For example, logging into a website, adding something to a cart, or joining a mailing list. + +The journey function takes two parameters: a `name` and a `callback`. The `name` helps you identify an individual journey. The `callback` argument is a function that encapsulates what the journey does. The callback provides access to fresh Playwright `page`, `params`, `browser`, and `context` instances. + +```js +journey('Journey name', ({ page, browser, context, params, request }) => { + // Add steps here +}); +``` + + +### Arguments [synthetics-journey-ref] + +**`name`** (*string*) +: A user-defined string to describe the journey. + +**`callback`** (*function*) +: A function where you will add steps. + + **Instances**: + + `page` + : A [page](https://playwright.dev/docs/api/class-page) object from Playwright that lets you control the browser’s current page. + + `browser` + : A [browser](https://playwright.dev/docs/api/class-playwright) object created by Playwright. + + `context` + : A [browser context](https://playwright.dev/docs/api/class-browsercontext) that doesn’t share cookies or cache with other browser contexts. + + `params` + : User-defined variables that allow you to invoke the Synthetics suite with custom parameters. For example, if you want to use a different homepage depending on the `env` (`localhost` for `dev` and a URL for `prod`). See [Work with params and secrets](../../../solutions/observability/apps/work-with-params-secrets.md) for more information. + + `request` + : A request object that can be used to make API requests independently of the browser interactions. For example, to get authentication credentials or tokens in service of a browser-based test. See [Make API requests](../../../solutions/observability/apps/write-synthetic-test.md#synthetics-request-param) for more information. + + + +## Add steps [synthetics-create-step] + +A journey consists of one or more *steps*. Steps are actions that should be completed in a specific order. Steps are displayed individually in the Synthetics UI along with screenshots for convenient debugging and error tracking. + +A basic two-step journey would look like this: + +```js +journey('Journey name', ({ page, browser, client, params, request }) => { + step('Step 1 name', () => { + // Do something here + }); + step('Step 2 name', () => { + // Do something else here + }); +}); +``` + +Steps can be as simple or complex as you need them to be. For example, a basic first step might load a web page: + +```js +step('Load the demo page', () => { + await page.goto('https://elastic.github.io/synthetics-demo/'); <1> +}); +``` + +1. Go to the [`page.goto` reference](https://playwright.dev/docs/api/class-page#page-goto) for more information. + + + +### Arguments [synthetics-step-ref] + +| | | +| --- | --- | +| **`name`** (*string*) | A user-defined string to describe the journey. | +| **`callback`** (*function*) | A function where you simulate user workflows using Synthetics and [Playwright](../../../solutions/observability/apps/write-synthetic-test.md#synthetics-playwright) syntax. | + +::::{note} + +If you want to generate code by interacting with a web page directly, you can use the **Synthetics Recorder**. + +The recorder launches a [Chromium browser](https://www.chromium.org/Home/) that will listen to each interaction you have with the web page and record them internally using Playwright. When you’re done interacting with the browser, the recorder converts the recorded actions into JavaScript code that you can use with Elastic Synthetics or {{heartbeat}}. + +For more details on getting started with the Synthetics Recorder, refer to [Use the Synthetics Recorder](../../../solutions/observability/apps/use-synthetics-recorder.md). + +:::: + + + +### Playwright syntax [synthetics-playwright] + +Inside the callback for each step, you’ll likely use a lot of Playwright syntax. Use Playwright to simulate and validate user workflows including: + +* Interacting with the [browser](https://playwright.dev/docs/api/class-browser) or the current [page](https://playwright.dev/docs/api/class-page) (like in the example above). +* Finding elements on a web page using [locators](https://playwright.dev/docs/api/class-locator). +* Simulating [mouse](https://playwright.dev/docs/api/class-mouse), [touch](https://playwright.dev/docs/api/class-touchscreen), or [keyboard](https://playwright.dev/docs/api/class-keyboard) events. +* Making assertions using [`@playwright/test`'s `expect` function](https://playwright.dev/docs/test-assertions). Read more in [Make assertions](../../../solutions/observability/apps/write-synthetic-test.md#synthetics-make-assertions). + +Visit the [Playwright documentation](https://playwright.dev/docs) for information. + +::::{note} +Do not attempt to run in headful mode (using `headless:false`) when running through Elastic’s global managed testing infrastructure or Private Locations as this is not supported. + +:::: + + +However, not all Playwright functionality should be used with Elastic Synthetics. In some cases, there are alternatives to Playwright functionality built into the Elastic Synthetics library. These alternatives are designed to work better for synthetic monitoring. Do *not* use Playwright syntax to: + +* **Make API requests.** Use Elastic Synthetic’s `request` parameter instead. Read more in [Make API requests](../../../solutions/observability/apps/write-synthetic-test.md#synthetics-request-param). + +There is also some Playwright functionality that is not supported out-of-the-box in Elastic Synthetics including: + +* [Videos](https://playwright.dev/docs/api/class-video) +* The [`toHaveScreenshot`](https://playwright.dev/docs/api/class-locatorassertions#locator-assertions-to-have-screenshot-1) and [`toMatchSnapshot`](https://playwright.dev/docs/api/class-snapshotassertions) assertions + +::::{note} +Captures done programmatically via [`screenshot`](https://playwright.dev/docs/api/class-page#page-screenshot) or [`video`](https://playwright.dev/docs/api/class-page#page-video) are not stored and are not shown in the Synthetics application. Providing a `path` will likely make the monitor fail due to missing permissions to write local files. + +:::: + + + +## Make assertions [synthetics-make-assertions] + +A more complex `step` might wait for a page element to be selected and then make sure that it matches an expected value. + +Elastic Synthetics uses `@playwright/test`'s `expect` function to make assertions and supports most [Playwright assertions](https://playwright.dev/docs/test-assertions). Elastic Synthetics does *not* support [`toHaveScreenshot`](https://playwright.dev/docs/api/class-locatorassertions#locator-assertions-to-have-screenshot-1) or any [Snapshot Assertions](https://playwright.dev/docs/api/class-snapshotassertions). + +For example, on a page using the following HTML: + +```html +<header class="header"> + <h1>todos</h1> + <input class="new-todo" + autofocus autocomplete="off" + placeholder="What needs to be done?"> +</header> +``` + +You can verify that the `input` element with class `new-todo` has the expected `placeholder` value (the hint text for `input` elements) with the following test: + +```js +step('Assert placeholder text', async () => { + const input = await page.locator('input.new-todo'); <1> + expect(await input.getAttribute('placeholder')).toBe( + 'What needs to be done?' + ); <2> +}); +``` + +1. Find the `input` element with class `new-todo`. +2. Use the assertion library provided by the Synthetics agent to check that the value of the `placeholder` attribute matches a specific string. + + + +## Make API requests [synthetics-request-param] + +You can use the `request` parameter to make API requests independently of browser interactions. For example, you could retrieve a token from an HTTP endpoint and use it in a subsequent webpage request. + +```js +step('make an API request', async () => { + const response = await request.get(params.url); + // Do something with the response +}) +``` + +The Elastic Synthetics `request` parameter is similar to [other request objects that are exposed by Playwright](https://playwright.dev/docs/api/class-apirequestcontext) with a few key differences: + +* The Elastic Synthetics `request` parameter comes built into the library so it doesn’t have to be imported separately, which reduces the amount of code needed and allows you to make API requests in [inline journeys](../../../solutions/observability/apps/create-monitors-in-synthetics-app.md#synthetics-get-started-ui-add-a-browser-monitor). +* The top level `request` object exposed by Elastic Synthetics has its own isolated cookie storage unlike Playwright’s `context.request` and `page.request`, which share cookie storage with the corresponding [`BrowserContext`](https://playwright.dev/docs/api/class-browsercontext). +* If you want to control the creation of the `request` object, you can do so by passing options via [`--playwright-options`](../../../solutions/observability/apps/use-synthetics-cli.md#elastic-synthetics-command) or in the [`synthetics.config.ts` file](../../../solutions/observability/apps/configure-synthetics-projects.md). + +For a full example that shows how to use the `request` object, refer to the [Elastic Synthetics demo repository](https://github.com/elastic/synthetics-demo/blob/main/advanced-examples/journeys/api-requests.journey.ts). + +::::{note} +The `request` parameter is not intended to be used for writing pure API tests. Instead, it is a way to support writing plain HTTP requests in service of a browser-based test. +:::: + + + +## Set up and remove a global state [before-after] + +If there are any actions that should be done before or after journeys, you can use `before`, `beforeAll`, `after`, or `afterAll`. + +To set up global state or a server that will be used for a **single** `journey`, for example, use a `before` hook. To perform this setup once before **all** journeys, use a `beforeAll` hook. + +```js +before(({ params }) => { + // Actions to take +}); + +beforeAll(({ params }) => { + // Actions to take +}); +``` + +You can clean up global state or close a server used for a **single** `journey` using an `after` hook. To perform this cleanup once after all journeys, use an `afterAll` hook. + +```js +after(({ params }) => { + // Actions to take +}); + +afterAll(({ params }) => { + // Actions to take +}); +``` + + +## Import NPM packages [synthetics-import-packages] + +You can import and use other NPM packages inside journey code. Refer to the example below using the external NPM package `is-positive`: + +```js +import { journey, step, monitor, expect } from '@elastic/synthetics'; +import isPositive from 'is-positive'; + +journey('bundle test', ({ page, params }) => { + step('check if positive', () => { + expect(isPositive(4)).toBe(true); + }); +}); +``` + +When you [create a monitor](../../../solutions/observability/apps/create-monitors-with-project-monitors.md) from a journey that uses external NPM packages, those packages will be bundled along with the journey code when the `push` command is invoked. + +However there are some limitations when using external packages: + +* Bundled journeys after compression should not be more than 800 Kilobytes. +* Native node modules will not work as expected due to platform inconsistency. + + +## Sample synthetic test [synthetics-sample-test] + +A complete example of a basic synthetic test might look like this: + +```js +import { journey, step, expect } from '@elastic/synthetics'; + +journey('Ensure placeholder is correct', ({ page }) => { + step('Load the demo page', async () => { + await page.goto('https://elastic.github.io/synthetics-demo/'); + }); + step('Assert placeholder text', async () => { + const placeholderValue = await page.getAttribute( + 'input.new-todo', + 'placeholder' + ); + expect(placeholderValue).toBe('What needs to be done?'); + }); +}); +``` + +You can find more complex examples in the [Elastic Synthetics demo repository](https://github.com/elastic/synthetics-demo/blob/main/advanced-examples/journeys/api-requests.journey.ts). + + +## Test locally [synthetics-test-locally] + +As you write journeys, you can run them locally to verify they work as expected. Then, you can create monitors to run your journeys at a regular interval. + +To test all the journeys in a Synthetics project, navigate into the directory containing the Synthetics project and run the journeys in there. By default, the `@elastic/synthetics` runner will only run files matching the filename `*.journey.(ts|js)*`. + +```sh +# Run tests on the current directory. The dot `.` indicates +# that it should run all tests in the current directory. +npx @elastic/synthetics . +``` + + +### Test an inline monitor [synthetics-test-inline] + +To test an inline monitor’s journey locally, pipe the inline journey into the `npx @elastic/synthetics` command. + +Assume, for example, that your inline monitor includes the following code: + +```js +step('load homepage', async () => { + await page.goto('https://www.elastic.co'); +}); +step('hover over products menu', async () => { + await page.hover('css=[data-nav-item=products]'); +}); +``` + +To run that journey locally, you can save that code to a file and pipe the file’s contents into `@elastic-synthetics`: + +```sh +cat path/to/sample.js | npx @elastic/synthetics --inline +``` + +And you’ll get a response like the following: + +```sh +Journey: inline + ✓ Step: 'load homepage' succeeded (1831 ms) + ✓ Step: 'hover over products menu' succeeded (97 ms) + + 2 passed (2511 ms) \ No newline at end of file diff --git a/solutions/observability/apps/writer-role.md b/solutions/observability/apps/writer-role.md index 62a0aa7f12..a0ff10dc9e 100644 --- a/solutions/observability/apps/writer-role.md +++ b/solutions/observability/apps/writer-role.md @@ -5,7 +5,7 @@ mapped_pages: # Writer role [synthetics-role-write] -::::{important} +::::{important} To minimize the privileges required by the writer role, use the [setup role](setup-role.md) to enable Monitor Management. This section assumes another user has already enabled Monitor Management. :::: @@ -16,13 +16,13 @@ For users who need to create, modify, and delete monitors, provide write access. * **Limited write access**: If you want to limit write access to the {{synthetics-app}} only, you can use [Limited write access](#synthetics-write-privileges-limited), which requires additional configuration. -## General write access [synthetics-write-privileges-general] +## General write access [synthetics-write-privileges-general] Create a **writer role**, called something like `synthetics_writer`: 1. Start with the `editor` [built-in role](../../../deploy-manage/users-roles/cluster-or-deployment-auth/built-in-roles.md). This role grants full access to all features in {{kib}} (including the {{observability}} solution) and read-only access to data indices. - ::::{note} + ::::{note} The `editor` [built-in role](../../../deploy-manage/users-roles/cluster-or-deployment-auth/built-in-roles.md) will grant write access to *all* {{kib}} apps. If you want to limit write access to the {{synthetics-app}} only, refer to [Limited write access](#synthetics-write-privileges-limited). :::: @@ -36,7 +36,7 @@ Create a **writer role**, called something like `synthetics_writer`: -## Limited write access [synthetics-write-privileges-limited] +## Limited write access [synthetics-write-privileges-limited] If you want to limit write access to the {{synthetics-app}} only, do *not* use the `editor` [built-in role](../../../deploy-manage/users-roles/cluster-or-deployment-auth/built-in-roles.md). @@ -51,7 +51,7 @@ Instead to you can create a writer role, called something like `synthetics_write Additional privileges will depend on the factors below. -### To restrict using Elastic’s global managed infrastructure [disable-managed-locations] +### To restrict using Elastic’s global managed infrastructure [disable-managed-locations] To restrict users assigned this role from using monitors hosted on Elastic’s global managed infrastructure: @@ -60,7 +60,7 @@ To restrict users assigned this role from using monitors hosted on Elastic’s g 3. Uncheck *Elastic managed locations enabled*. -### If using Private Locations [synthetics-role-write-private-locations] +### If using Private Locations [synthetics-role-write-private-locations] The user who initially sets up a new Private Location needs additional privileges. Users who create or update monitors hosted on that Private Location do not need any additional permissions. @@ -72,9 +72,9 @@ The user who is setting up a new Private Location will need the following privil | [Kibana](../../../deploy-manage/users-roles/cluster-or-deployment-auth/kibana-privileges.md) | `Integrations`: `read` | Access to Integrations in {{kib}}. | -### If using projects [_if_using_projects] +### If using projects [_if_using_projects] -If the user should be able to create and update monitors using [projects](get-started.md#choose-projects), add *at least one* of following privileges: +If the user should be able to create and update monitors using [projects](get-started.md#observability-synthetics-get-started-synthetics-project), add *at least one* of following privileges: | Type | Privilege | Purpose | | --- | --- | --- |