Skip to content

Commit

Permalink
chore: Includes instructions to enable the collection of internal Flu…
Browse files Browse the repository at this point in the history
…ent Bit metrics in K8s clusters and how to represent them in a Dashboard
  • Loading branch information
jsubirat committed Apr 29, 2024
1 parent 6f38c33 commit 58f6eaf
Showing 1 changed file with 26 additions and 1 deletion.
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ To forward your Kubernetes logs to New Relic with our plugin:
2. Optionally, you can further tune your installation in [Step 4 from the guided install](/docs/kubernetes-pixie/kubernetes-integration/installation/kubernetes-integration-install-configure/#kubernetes-install-navigation) using the numerous configuration options available in the [newrelic-logging repository](https://github.com/newrelic/helm-charts/tree/master/charts/newrelic-logging#configuration). However, we recommend the standard setup, as it is valid for most users.

<Callout variant="important">

If you're [using a Kubernetes secret](https://github.com/newrelic/helm-charts/blob/master/charts/newrelic-logging/values.yaml#L8-L25) to store the New Relic license key, the `newrelic-logging` chart defaults to sending logs to the US API endpoint. If the license key belongs to an EU or FedRAMP account, and a secret is used for key storage, you must update the endpoint setting with the appropriate value from the [API reference docs](/docs/logs/log-api/introduction-log-api/#endpoint). Here's an example of how to set this for EU accounts:

```
Expand All @@ -39,6 +39,31 @@ To forward your Kubernetes logs to New Relic with our plugin:

<InstallFeedback />

## Troubleshoot your Kubernetes plugin for log forwarding installation

Sometimes, despite correctly installing the Kubernetes plugin for log forwarding (`newrelic-logging` [Helm chart](https://github.com/newrelic/helm-charts/blob/master/charts/newrelic-logging)), you may encounter performance issues that affect the correct delivery of logs. In such circumstances, it can be helpful to look at the log forwarder ([Fluent Bit](https://fluentbit.io/)) internal metrics to understand what is the cause of a potential bottleneck.

To this end, the `newrelic-logging` Helm chart provides a configuration setting to enable the collection of such metrics for a given Kubernetes cluster. Moreover, we provide a dashboard template in JSON format to easily display all these metrics in New Relic.

To configure your Kubernetes cluster to send the log forwarder internal metrics and represent them in a dashboard, follow these steps:
1. Install the Helm chart with the following extra configuration setting:
```
newrelic-logging:
fluentBit:
sendMetrics: true
```
2. Download this dashboard template file. Open it in a text editor and replace all the `YOUR_ACCOUNT_ID` occurrences (42 in total) by your [New Relic Account ID](/docs/accounts/accounts-billing/account-structure/account-id/).

3. Import the resulting dashboard in JSON format by following [these instructions](/docs/query-your-data/explore-query-data/dashboards/dashboards-charts-import-export-data/#import-json).

4. The imported dashboard will be available in your [Dashboards page](https://one.newrelic.com/dashboards) as `Kubernetes Fluent Bit monitoring`.

5. (Optional) It is highly recommended to enable [facet filtering] based on the "Fluent Bit version" table. To do so, simply click on the "Edit" submenu of the "Fluent Bit version" table and select "Filter the current dashboard" under "Facet Linking".

<Callout variant="tip">
We capture Fluent Bit's internal metrics by using its [prometheus_scrape INPUT plugin](https://docs.fluentbit.io/manual/pipeline/inputs/prometheus-scrape-metrics) in conjunction with its [prometheus_remote_write OUTPUT plugin](https://docs.fluentbit.io/manual/pipeline/outputs/prometheus-remote-write). All the Prometheus `counter` metrics are actually _cumulative counters_, but we automatically perform a _delta conversion_ when they are ingested at New Relic to ease querying them using NRQL later. You can find more details [here](/docs/data-apis/understand-data/metric-data/cumulative-metrics/).
</Callout>

## View log data [#find-data]

If everything is configured correctly and your data is being collected, you should see log data in both of these places:
Expand Down

0 comments on commit 58f6eaf

Please sign in to comment.