Skip to content

Commit

Permalink
fix(OTEL): old link to nrIntegrationError doc
Browse files Browse the repository at this point in the history
  • Loading branch information
@ally-sassman
ally-sassman committed May 6, 2024
1 parent d451c5c commit 4bf0ea3
Showing 1 changed file with 7 additions and 7 deletions.
14 changes: 7 additions & 7 deletions ...ntegrations/opentelemetry/best-practices/opentelemetry-otlp-troubleshooting.mdx
View file
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ When you encounter an issue with the New Relic OTLP endpoint, first follow these

1. **Enable diagnostic logs.** Ensure that your OTLP client has logging enabled such that you can see details about any errors which may be occruing. The mechanism for enabling logs varies by implementation. Please consult relevant docs.
2. **Test that New Relic's OTLP endpoint is reachable.** A simple shell command `curl http://otlp.nr-data.net` from the machine in question can help determine if some local network configuration issue (i.e. firewall restriction) is preventing connectivity to New Relic.
3. **Check for [`NrIntegrationErrors`](/docs/telemetry-data-platform/manage-data/nrintegrationerror/).** New Relic OTLP ingest performs minimal validation synchronously before returning a success status code. If you don't see indications of export errors in your application logs, but don't see data in New Relic, try querying for `NrIntegrationErrors`. There may be issues with your data which were detected during asynchronous validation.
3. **Check for [`NrIntegrationError`](/docs/data-apis/manage-data/nrintegrationerror/) events.** New Relic OTLP ingest performs minimal validation synchronously before returning a success status code. If you don't see indications of export errors in your application logs, but don't see data in New Relic, try querying for `NrIntegrationError`. There may be issues with your data which were detected during asynchronous validation.
4. **Determine if the problem is localized.** Often errors are localized to a specific application or environment. In these cases, it's useful to evaluate the differences between the areas which are problematic and properly functioning.
5. **Look for signs of invalid API key.** The New Relic OTLP endoint [requires setting an `api-key` header](/docs/more-integrations/open-source-telemetry-integrations/opentelemetry/best-practices/opentelemetry-otlp/#api-key). Invalid or missing API keys are a common issue which present with HTTP 403 or 401 status codes, or gRPC Unauthenticated or PermissionDenied status codes. If you see these, check that your API key is valid and is being properly set.
6. **Check if the export succeeds after retry.** We [recommend that retry is enabled](/docs/more-integrations/open-source-telemetry-integrations/opentelemetry/best-practices/opentelemetry-otlp/#retry), and expect export attempts to occasionally initially fail with transient errors but succeed after retrying. However, we do have an [SLA](/docs/licenses/license-information/referenced-policies/service-level-availability-commitment/). If you suspect that transient errors are frequent enough that they exceed our SLA, please open a support case.
Expand All @@ -42,10 +42,10 @@ The table below catalogs issues we've seen customers encounter with the New Reli
| HTTP | 401 - Unauthorized | Java | `io.opentelemetry.exporter.internal.http.HttpExporter - Failed to export spans. Server responded with HTTP status code 401.` | [Include API Key](/docs/more-integrations/open-source-telemetry-integrations/opentelemetry/best-practices/opentelemetry-otlp/#api-key) | Missing `api-key` header |
| HTTP | 401 - Unauthorized | Collector | `Exporting failed. The error is not retryable. Dropping data. {"kind": "exporter", "data_type": "traces", "name": "otlphttp", "error": "Permanent error: error exporting items, request to https://otlp.nr-data.net/v1/traces responded with HTTP Status Code 401, Message=, Details=[]", "dropped_items": 4}` | [Include API Key](/docs/more-integrations/open-source-telemetry-integrations/opentelemetry/best-practices/opentelemetry-otlp/#api-key) | Missing `api-key` header |
| HTTP | 401 - Unauthorized | Go | `failed to upload metrics: failed to send metrics to https://otlp.nr-data.net/v1/metrics: 401 Unauthorized` | [Include API Key](/docs/more-integrations/open-source-telemetry-integrations/opentelemetry/best-practices/opentelemetry-otlp/#api-key) | Missing `api-key` header |
| HTTP | 403 - Forbidden | Java | `io.opentelemetry.exporter.internal.http.HttpExporter - Failed to export spans. Server responded with HTTP status code 403.` | [Verify API Key](/docs/more-integrations/open-source-telemetry-integrations/opentelemetry/best-practices/opentelemetry-otlp/#api-key) | Invalid `api-key` header |
| HTTP | 403 - Forbidden | Java | `Exporting failed. The error is not retryable. Dropping data. {"kind": "exporter", "data_type": "traces", "name": "otlphttp", "error": "Permanent error: error exporting items, request to https://otlp.nr-data.net/v1/traces responded with HTTP Status Code 403, Message=, Details=[]", "dropped_items": 14}` | [Verify API Key](/docs/more-integrations/open-source-telemetry-integrations/opentelemetry/best-practices/opentelemetry-otlp/#api-key) | Invalid `api-key` header |
| HTTP | 403 - Forbidden | Go | `traces export: failed to send to https://otlp.nr-data.net/v1/traces: 403 Forbidden` | [Verify API Key](/docs/more-integrations/open-source-telemetry-integrations/opentelemetry/best-practices/opentelemetry-otlp/#api-key) | Invalid `api-key` header |
| HTTP | 403 - Forbidden | .NET | `Exporter failed send data to collector to {0} endpoint. Data will not be sent. Exception: {1}{https://otlp.nr-data.net:4317/v1/traces}{System.Net.Http.HttpRequestException: Response status code does not indicate success: 403 (Forbidden).` | [Verify API Key](/docs/more-integrations/open-source-telemetry-integrations/opentelemetry/best-practices/opentelemetry-otlp/#api-key) | Invalid `api-key` header |
| HTTP | 403 - Forbidden | Java | `io.opentelemetry.exporter.internal.http.HttpExporter - Failed to export spans. Server responded with HTTP status code 403.` | [Verify API key](/docs/more-integrations/open-source-telemetry-integrations/opentelemetry/best-practices/opentelemetry-otlp/#api-key) | Invalid `api-key` header |
| HTTP | 403 - Forbidden | Java | `Exporting failed. The error is not retryable. Dropping data. {"kind": "exporter", "data_type": "traces", "name": "otlphttp", "error": "Permanent error: error exporting items, request to https://otlp.nr-data.net/v1/traces responded with HTTP Status Code 403, Message=, Details=[]", "dropped_items": 14}` | [Verify API key](/docs/more-integrations/open-source-telemetry-integrations/opentelemetry/best-practices/opentelemetry-otlp/#api-key) | Invalid `api-key` header |
| HTTP | 403 - Forbidden | Go | `traces export: failed to send to https://otlp.nr-data.net/v1/traces: 403 Forbidden` | [Verify API key](/docs/more-integrations/open-source-telemetry-integrations/opentelemetry/best-practices/opentelemetry-otlp/#api-key) | Invalid `api-key` header |
| HTTP | 403 - Forbidden | .NET | `Exporter failed send data to collector to {0} endpoint. Data will not be sent. Exception: {1}{https://otlp.nr-data.net:4317/v1/traces}{System.Net.Http.HttpRequestException: Response status code does not indicate success: 403 (Forbidden).` | [Verify API key](/docs/more-integrations/open-source-telemetry-integrations/opentelemetry/best-practices/opentelemetry-otlp/#api-key) | Invalid `api-key` header |
| HTTP | Timeout | Java | `io.opentelemetry.exporter.internal.http.HttpExporter - Failed to export spans. The request could not be executed. Full error message: timeout` | [Tune batching / timeout](/docs/more-integrations/open-source-telemetry-integrations/opentelemetry/best-practices/opentelemetry-otlp/#payload) | Occurs after export times out. Check timeout settings and client network status.<br></br>If you've ruled out client side network and configuration, open support case. |
| HTTP | Timeout | Collector | `max elapsed time expired failed to make an HTTP request: Post \"https://otlp.nr-data.net/v1/traces\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)` | [Tune batching / timeout](/docs/more-integrations/open-source-telemetry-integrations/opentelemetry/best-practices/opentelemetry-otlp/#payload) | Typically occurs after retry attempts fail and export times out. Can be related to client network, client retry / timeout configuration, or New Relic network / servers.<br></br>If you've ruled out client side network and configuration, open support case. |
| HTTP | Timeout | Go | `failed to upload metrics: context deadline exceeded: retry-able request failure` | [Tune batching / timeout](/docs/more-integrations/open-source-telemetry-integrations/opentelemetry/best-practices/opentelemetry-otlp/#payload) | Occurs after export times out. Check timeout settings and client network status.<br></br>If you've ruled out client side network and configuration, open support case. |
Expand All @@ -62,5 +62,5 @@ The table below catalogs issues we've seen customers encounter with the New Reli
| gRPC | Code 14 - Unavailable<br></br>Connection reset | Collector | `rpc error: code = Unavailable desc = error reading from server: read tcp 100.127.0.171:47470->162.247.241.110:4317: read: connection reset by peer` | [Tune retry](/docs/more-integrations/open-source-telemetry-integrations/opentelemetry/best-practices/opentelemetry-otlp/#retry) | Should solve with retry. Ensure collector has sufficient resources to handle retry backpressure. |
| gRPC | Code 14 - Unavailable<br></br>HTTP 502 | Collector | `rpc error: code = Unavailable desc = unexpected HTTP status code received from server: 502 (Bad Gateway); transport: received unexpected content-type "text/html"` | [Tune retry](/docs/more-integrations/open-source-telemetry-integrations/opentelemetry/best-practices/opentelemetry-otlp/#retry) | Should solve with retry. Ensure collector has sufficient resources to handle retry backpressure. |
| gRPC | Code 14 - Unavailable<br></br>HTTP 503 | Collector | `rpc error: code = Unavailable desc = unexpected HTTP status code received from server: 503 (Service Unavailable)` | [Tune retry](/docs/more-integrations/open-source-telemetry-integrations/opentelemetry/best-practices/opentelemetry-otlp/#retry) | Should solve with retry. Ensure collector has sufficient resources to handle retry backpressure. |
| gRPC | Code 16 - PermissionDenied | Java | `io.opentelemetry.exporter.internal.grpc.GrpcExporter - Failed to export spans. Server responded with gRPC status code 16.` | [Verify API Key](/docs/more-integrations/open-source-telemetry-integrations/opentelemetry/best-practices/opentelemetry-otlp/#api-key) | Invalid `api-key` header |
| gRPC | Code 16 - PermissionDenied | .NET | `Exporter failed send data to collector to {0} endpoint. Data will not be sent. Exception: {1}{https://otlp.nr-data.net:4317/}{Grpc.Core.RpcException: Status(StatusCode="PermissionDenied", Detail="")` | [Verify API Key](/docs/more-integrations/open-source-telemetry-integrations/opentelemetry/best-practices/opentelemetry-otlp/#api-key) | Invalid `api-key` header |
| gRPC | Code 16 - PermissionDenied | Java | `io.opentelemetry.exporter.internal.grpc.GrpcExporter - Failed to export spans. Server responded with gRPC status code 16.` | [Verify API key](/docs/more-integrations/open-source-telemetry-integrations/opentelemetry/best-practices/opentelemetry-otlp/#api-key) | Invalid `api-key` header |
| gRPC | Code 16 - PermissionDenied | .NET | `Exporter failed send data to collector to {0} endpoint. Data will not be sent. Exception: {1}{https://otlp.nr-data.net:4317/}{Grpc.Core.RpcException: Status(StatusCode="PermissionDenied", Detail="")` | [Verify API key](/docs/more-integrations/open-source-telemetry-integrations/opentelemetry/best-practices/opentelemetry-otlp/#api-key) | Invalid `api-key` header |

0 comments on commit 4bf0ea3

Please sign in to comment.