Skip to content

Commit

Permalink
Merge pull request #2900 from newrelic/pixie-ga-2
Browse files Browse the repository at this point in the history
Pixie ga 2
  • Loading branch information
mmfred committed Jul 15, 2021
2 parents e588127 + f5d076e commit 1a05651
Show file tree
Hide file tree
Showing 11 changed files with 142 additions and 46 deletions.
Original file line number Diff line number Diff line change
@@ -1,18 +1,15 @@
---
title: Query Auto-telemetry with Pixie data
title: Query Pixie data
tags:
- Pixie Auto-telemetry
- Service monitoring
- Kubernetes
- eBPF
metaDescription:
watermark: BETA
redirects:
- docs/integrations/kubernetes-integration/understand-use-data/auto-telemetry-pixie-data-model/
---

<Callout variant="important">
[Auto-telemetry with Pixie](/docs/full-stack-observability/observe-everything/get-started/get-started-auto-telemetry-pixie/) is a supported public beta. Therefore, the features and functionality are subject to change. By joining the beta for Auto-telemetry with Pixie, you agree to the terms in the [New Relic Pre-Release Policy](/docs/licenses/license-information/referenced-policies/new-relic-pre-release-policy/).
</Callout>

[Auto-telemetry with Pixie](/docs/full-stack-observability/observe-everything/get-started/get-started-auto-telemetry-pixie/) pulls data from the Pixie Cloud API and sends it to the New Relic OpenTelemetry endpoint. You can build your own charts and [query](/docs/using-new-relic/data/understand-data/query-new-relic-data) your Auto-telemetry with Pixie data using the query builder and the NerdGraph API.

Find out more about getting started with Auto-telemetry with Pixie [here](/docs/full-stack-observability/observe-everything/get-started/get-started-auto-telemetry-pixie).
Expand Down
Original file line number Diff line number Diff line change
@@ -1,44 +1,43 @@
---
title: Use Auto-telemetry with Pixie to get instant Kubernetes observability
title: Use Auto-telemetry with Pixie for instant Kubernetes observability
tags:
- Pixie Auto-telemetry
- Service monitoring
- Kubernetes
- eBPF
metaDescription:
watermark: BETA
redirects:
- /docs/full-stack-observability/get-started-auto-telemetry-pixie
---

<Callout variant="important">
Auto-telemetry with Pixie is a supported public beta. Therefore, the features and functionality are subject to change. By joining the beta for Auto-telemetry with Pixie, you agree to the terms in the [New Relic Pre-Release Policy](/docs/licenses/license-information/referenced-policies/new-relic-pre-release-policy/).
</Callout>

When we say auto-telemetry, we’re not talking about cars &mdash; we’re talking about automatically collecting telemetry data for your Kubernetes clusters. With our [Pixie](https://pixielabs.ai/) integration, you no longer have to manually instrument your clusters or change code. With one install command, you get observability into your Kubernetes clusters and workloads. No language agents required.

Pixie collects telemetry data using [eBPF](https://ebpf.io/), a virtual machine-like construct in the Linux kernel. eBPF makes it easy to collect similar data about your clusters and workloads as with traditional New Relic language agents, but now without needing to update your code.
Pixie, New Relic's open source observability tool for Kubernetes applications, collects telemetry data using [eBPF](https://ebpf.io/), a virtual machine-like construct in the Linux kernel. eBPF makes it easy to collect similar data about your clusters and workloads as with traditional New Relic language agents, but now without needing to update your code.

![Pixie-explorer](./images/pixie-explorer.png "Pixie explorer")
![service-graph](./images/service-graph.png "service graph in live debugger")
<figcaption>
**Live debugging with Pixie** shows a service graph listing the namespaces and the node that are available on the current cluster.
</figcaption>

Simply put, Auto-telemetry with Pixie offers the quickest option for getting observability into your Kubernetes services.

## Why it matters

Our Pixie integration gives you the best of both worlds: Pixie’s fast and simple Kubernetes observability coupled with New Relic One’s incident correlation, intelligent alerting, and long-term retention.

You’ll get visibility into HTTP services using golden signals, HTTP transactions, database transactions, distributed tracing, and JVM metrics. You can operate, debug, and scale your Kubernetes clusters based on the information you learn about how your clusters and services are running. Using the New Relic [Explorer](/docs/new-relic-one/use-new-relic-one/core-concepts/new-relic-explorer-view-performance-across-apps-services-hosts/), you can see key metrics and events at every level, starting with the cluster, and diving down into namespaces, deployments, and pods. You can quickly spot anomalous behavior, and where it’s happening. If you need further insight, you can easily link over to Pixie's native UI for advanced debugging with PXL scripts.
You’ll get visibility into HTTP services using golden signals, HTTP transactions, database transactions, distributed tracing, and JVM metrics. You can operate, debug, and scale your Kubernetes clusters based on the information you learn about how your clusters and services are running. Using the New Relic [Explorer](/docs/new-relic-one/use-new-relic-one/core-concepts/new-relic-explorer-view-performance-across-apps-services-hosts/), you can see key metrics and events at every level, starting with the cluster, and diving down into namespaces, deployments, and pods. You can quickly spot anomalous behavior, and where it’s happening.

And then dive deeper using embedded visualizations of your Pixie data. Quickly identify hot spots with [Flamegraph](#flamegraph). On the [Live debugging with Pixie tab](#live-debugging), answer questions like what SQL requests your app is making or which services are talking to each other.

## Install Auto-telemetry with Pixie

Use our [guided installation](/docs/full-stack-observability/observe-everything/get-started/new-relic-guided-install-overview/) process to install Auto-telemetry with Pixie. This deploys Pixie with New Relic's Kubernetes integration on your cluster. You don't need to do any further configuration or installation to start using Pixie.

If you want to install Auto-telemetry with Pixie on multiple clusters, re-run the guided install for each additional cluster.

### Beta limitations
### Limitations

The following are not supported in this beta version of Auto-telemetry with Pixie:
The following are not currently supported in Auto-telemetry with Pixie:
* OpenShift. While our New Relic infrastructure agent supports OpenShift, Pixie does not.
* RBAC. New Relic RBAC controls cannot be used to gate access to Pixie. This means the following:
* Auto-telemetry with Pixie creates one project for all of your Kubernetes clusters
Expand All @@ -47,43 +46,38 @@ The following are not supported in this beta version of Auto-telemetry with Pixi

### Before you begin

The guided install requires the following (you can get these set up before you begin, or during the install steps):

* a [Pixie account](https://work.withpixie.ai/admin/api-keys)
You need a Google account to sign up for Pixie.
* a Pixie API key
* Sufficient memory: Pixie requires 2Gb of memory per node.
* Review this [Pixie data security overview](/docs/auto-telemetry-pixie/pixie-data-security-overview) for actions to take to secure your data.
* Make sure you have sufficient memory: Pixie requires 2Gb of memory per node.

### Start the guided installation
### Install from the beginning of the guided install process

1. Open our [New Relic One guided install](https://one.newrelic.com/launcher/nr1-core.home?pane=eyJuZXJkbGV0SWQiOiJucjEtY29yZS5ob21lLXNjcmVlbiJ9&cards[0]=eyJuZXJkbGV0SWQiOiJucjEtaW5zdGFsbC1uZXdyZWxpYy5ucjEtaW5zdGFsbC1uZXdyZWxpYyIsImFjdGlvbkluZGV4IjoxLCJpc0xvYWRpbmdUcmVhdG1lbnRzIjpmYWxzZSwicHJldkFjdGl2ZUNvbXBvbmVudCI6bnVsbCwiYWN0aXZlQ29tcG9uZW50IjoiVlRTT1NlbGVjdEFjY291bnQiLCJhY3RpdmVFbnZpcm9ubWVudCI6bnVsbCwidXNlckhhc011bHRpcGxlQWNjb3VudHMiOnRydWUsInNob3duRXhwZXJpbWVudCI6dHJ1ZSwicGF0aCI6Imd1aWRlZCJ9).
2. Select the account you want to use for the guided install, and click **Continue**.
Note: if you have a single account, you won't see this option.
3. Select **Kubernetes**.
4. Type in a name for your cluster, and select the check box to **Gather telemetry automatically with Pixie beta**.
3. Select **Kubernetes** and then continue with step one in the next section.

### Install from the Configure the HELM command/manifest (yaml) file

If you arrived in the guided installation process by following a link from Pixie or from within New Relic, your steps begin here.

1. Select the account and cluster for the install. If needed, select a namespace.

<Callout variant="important">
Currently, Pixie performs best on clusters with up to 100 nodes (exceeding 100 nodes can lead to excessive memory usage and scripts failing to run). Friendly reminder: autoscaling can quickly drive up your node numbers.
</Callout>

If Pixie is already running on your cluster, expand the **Advanced options** section and select the appropriate check box. We strongly recommend enabling Pixie auto update to maintain the latest Pixie version.

5. Click **Continue**.

6. Provide your Pixie API key. If you already have a key, simply paste it in the **API key** box, and click **Continue**.

If you don't have an account and existing API key, click the link to [create a Pixie account](https://work.withpixie.ai/admin/api-keys). Then, on the [API key page](https://work.withpixie.ai/admin/api-keys), click **New key** to add a new key. In the **Actions** column, click the ellipsis, and then click **Copy value**. Switch back to the guided install UI, paste the value into the **API key** box, and click **Continue**.

When you click **Continue**, the API key is added to your New Relic account, and a deploy key is created for Pixie. Your Pixie account information is now stored on your New Relic account; if you follow the install steps again, you won't be asked for this information.

7. On the **Choose install method** page, copy the Helm command that's provided, and then run it on your command line. Helm installs a bundle containing the New Relic infrastructure agent, an integration to gather Prometheus metrics and Kubernetes events, and the Pixie integration. The deployment takes a few minutes to complete.
2. Click **Continue**.
3. Select the data you want to gather, observe, and debug, and click **Continue**.
4. On the **Choose install method** page, copy the Helm command that's provided, and then run it on your command line. Helm installs a bundle containing the New Relic infrastructure agent, an integration to gather Prometheus metrics and Kubernetes events, and the Pixie integration. The deployment takes a few minutes to complete.

![Pixie-Helm](./images/helm-pixie.png "Pixie Helm")

To see the status of the install to the cluster, run `kubectl get pods -n newrelic`.

8. In the New Relic install UI, click **Continue** to open the **Listening for data** page.
9. When you get the message, **See your data**, click **Kubernetes Cluster Explorer** to see your cluster.
5. Click **Continue** to open the **Listening for data** page.
6. When you get the message, **See your data**, click **Kubernetes Cluster Explorer** to see your cluster.

Note that Auto-telemetry with Pixie might restart after installation. This is caused by the auto update feature.

Expand All @@ -95,6 +89,28 @@ In the cluster explorer, you can get a quick overview of the nodes in your clust
Containers might be listed for up to four hours after they get decommissioned.
</Callout>

You can query the Pixie data that's available in New Relic One and create dashboards for at-a-glance monitoring. Find the data model and sample queries [here](/docs/integrations/kubernetes-integration/understand-use-data/auto-telemetry-pixie-data-model).
You can query the Pixie data in New Relic One and create dashboards for at-a-glance monitoring. Find the data model and sample queries [here](/docs/integrations/kubernetes-integration/understand-use-data/auto-telemetry-pixie-data-model).

## Investigate usage spikes with Flamegraph [#flamegraph]

![flamegraph](./images/flamegraph.png "Pixie flamegraph")

Debugging is orders of magnitude easier when you can quickly see what your application is doing. Flamegraph, a Pixie always-on visualization, requires no instrumentation, redeploying, or recompiling. It works for compiled languages - Go, C+, Rust, to name a few. Flamegraph tells you in a glance what functions your application is spending time on, and where you have hot spots. Flamegraph is especially useful for hierarchical resource use, like disk usage and CPU utilization. For more information on how to read Flamegraph, see [the Pixie documentation](https://docs.pixielabs.ai/tutorials/profiler#reading-the-flamegraph).


## Debug live [#live-debugging]

On the **Live debugging with Pixie** tab, run PxL scripts &mdash; scripts written in Pixie's PxL language &mdash; to view live data captured through eBPF. Select the **script** drop-down and then select a script to run in the tab. (For best results, select a time range that is recent in the time picker.)

Scripts enable you to debug:

* Traffic in multiple formats: HTTP and HTTPs (including encrypted), DNS, Postgres, MySQL, Cassandra, Redis (currently supporting SQL and HTTP in beta)
* Kubernetes services and their throughput, error rate, and latency statistics
* Service maps to learn which services are talking to each other
* Network traffic maps to learn which nodes are talking to each other
* JVM data

If you don't find the script you need, you can [write your own PxL scripts](https://docs.pixielabs.ai/tutorials/pxl-scripts/).
Find more information about available scripts in [Pixie's open source repo](https://github.com/pixie-labs/pixie/tree/main/src/pxl_scripts/px).


In addition to analyzing the performance of your clusters in New Relic One, you might want to use Pixie's advanced debugging workflows. In Pixie's native UI, you can drill down further into header and content requests. Click **Explore service in Pixie** to see information about your service in the Pixie UI. Learn more about [writing PXL scripts in the Pixie docs](https://docs.pixielabs.ai/tutorials/pxl-scripts/).
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
---
title: Pixie data and security
tags:
- Pixie Auto-telemetry data
- Service monitoring
- Kubernetes
- eBPF
- Pixie data
metaDescription:
---

New Relic hosts Pixie, our open source observability tool for Kubernetes applications. Pixie fetches data flowing through a systems kernel via [eBPF](https://ebpf.io/) probes from a privileged state. If an application contains sensitive data, this data flow can potentially collect that data in plain text and save it to [NRDB](/docs/telemetry-data-platform/get-started/nrdb-horsepower-under-hood/). For example, Pixie gives insight into any data on your system, including raw HTTP requests in the kernel, allowing you to view all headers, query parameters, and HTTP bodies of a request.

New Relic, on the other hand, fetches and stores a subset of data that only revolves around an application's performance. These performance metrics include, but are not limited to:

* Golden metrics (throughput, latency, error rate) for HTTP-based services
* HTTP transaction data
* Database transaction data (for MySQL & PostgreSQL)
* Distributed tracing
* JVM metrics


The data you view on the **Live debugging** tab comes via Pixie, and is therefore potentially sensitive. It is not stored by New Relic and is ephemeral, queryable for only 24 hours.

## Control who has access to Pixie data

If you want to manage which members of your organization can view Pixie data, as well as install and delete Pixie links, you can [create a custom role](/docs/accounts/accounts-billing/new-relic-one-user-management/tutorial-add-new-user-groups-roles-new-relic-one-user-model/#roles). Note that this option is available only to Enterprise and Pro level customers.
For more information, see [New Relic's user model](/docs/accounts/accounts-billing/new-relic-one-user-management/new-relic-one-user-model-understand-user-structure/).

## Manage auto-upate and two-way communication

Pixie maintains an active two-way communication channel from your host system to Pixie at [withpixie.ai](https://work.withpixie.ai/). Pixie uses this communication channel to query data, push updates, and retrieve metadata and health checks about Pixie and your Kubernetes cluster.

By default, Pixie queries withpixie.ai to check if new updates have been pushed and then automatically installs them if they’re present. To disable auto updates, you must set a flag prior to the install process using either Helm or in the `newrelic-manifest.yaml` file. To disable automatic updates, choose one:

<CollapserGroup>
<Collapser
className="freq-link"
id=""
title="Helm option"
>
Add `--set pixie-chart.disableAutoUpdate=true` to your Helm command.

</Collapser>
<Collapser
className="freq-link"
id=""
title="newrelic-manifest.yaml option"
>
in your newrelic-manifest.yaml file under the pl-cluster-config section, add `PL_DISABLE_AUTO_UPDATE: "true"` to the `data` directive. Example:
```
---
apiVersion: v1
data:
PL_CUSTOM_ANNOTATIONS: ""
PL_CUSTOM_LABELS: ""
PL_DISABLE_AUTO_UPDATE: "true"
PL_ETCD_OPERATOR_ENABLED: "false"
PL_MD_ETCD_SERVER: "https://etcd.newrelic.svc:2379"
PX_MEMORY_LIMIT: ""
kind: ConfigMap
metadata:
annotations:
creationTimestamp: null
labels:
name: pl-cluster-config
namespace: newrelic
---
```
</Collapser>
</CollapserGroup>
1 change: 1 addition & 0 deletions src/data/homepage.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ fso:
- /docs/serverless-function-monitoring
- /docs/synthetics
- /docs/new-relic-one/use-new-relic-one/workloads/workloads-isolate-resolve-incidents-faster
- /docs/auto-telemetry-pixie/get-started-auto-telemetry-pixie

ai:
title: Alerts and Applied Intelligence (AI)
Expand Down
6 changes: 5 additions & 1 deletion src/i18n/translations/en/translation.json
Original file line number Diff line number Diff line change
Expand Up @@ -112,6 +112,10 @@
"t10": {
"title": "Workloads",
"description": "Group or monitor entities across your entire stack based on a team or a set of responsibilities."
},
"t11": {
"title": "Auto-telemetry with Pixie",
"description": "Get instant Kubernetes observability."
}
},
"ai": {
Expand Down Expand Up @@ -177,4 +181,4 @@
"placeholder": "Filter navigation"
}
}
}
}
13 changes: 9 additions & 4 deletions src/nav/full-stack-observability.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,10 +23,6 @@ pages:
path: /docs/full-stack-observability/monitor-everything/observability-solutions/distributed-tracing
- title: Infrastructure monitoring
path: /docs/full-stack-observability/monitor-everything/observability-solutions/infrastructure-monitoring
- title: Auto-Telemetry with Pixie
path: /docs/full-stack-observability/observe-everything/get-started/get-started-auto-telemetry-pixie
- title: Update your Pixie account in New Relic
path: /docs/full-stack-observability/observe-everything/get-started/pixie-account-link
- title: Kubernetes monitoring
path: /docs/integrations/kubernetes-integration/get-started/introduction-kubernetes-integration/
- title: Logs in context
Expand Down Expand Up @@ -1639,6 +1635,15 @@ pages:
path: /docs/apm/new-relic-apm/maintenance/remove-applications-new-relic
- title: Record and monitor deployments
path: /docs/apm/new-relic-apm/maintenance/record-monitor-deployments
- title: Auto-telemetry with Pixie
path: /docs/auto-telemetry-pixie
pages:
- title: Use Auto-telemetry with Pixie
path: /docs/auto-telemetry-pixie/get-started-auto-telemetry-pixie
- title: Query Pixie data
path: /docs/auto-telemetry-pixie/auto-telemetry-pixie-data-model
- title: Pixie data and security
path: /docs/auto-telemetry-pixie/pixie-data-security-overview
- title: Browser
path: /docs/browser
pages:
Expand Down

0 comments on commit 1a05651

Please sign in to comment.