From 1415b3567c6c302474955d811b41ce1e034718d0 Mon Sep 17 00:00:00 2001 From: Esther Kim Date: Tue, 4 Nov 2025 11:19:01 -0500 Subject: [PATCH 1/4] Add TCP disclaimer shortcode --- content/en/logs/guide/log-collection-troubleshooting-guide.md | 2 ++ layouts/shortcodes/logs-tcp-disclaimer.en.md | 2 ++ 2 files changed, 4 insertions(+) create mode 100644 layouts/shortcodes/logs-tcp-disclaimer.en.md diff --git a/content/en/logs/guide/log-collection-troubleshooting-guide.md b/content/en/logs/guide/log-collection-troubleshooting-guide.md index 8f40bcaa06ee0..c26f29b3d1782 100644 --- a/content/en/logs/guide/log-collection-troubleshooting-guide.md +++ b/content/en/logs/guide/log-collection-troubleshooting-guide.md @@ -25,6 +25,8 @@ Changes in the configuration of the `datadog-agent` won't be taken into account ## Outbound traffic on port 10516 is blocked +{{% logs-tcp-disclaimer %}} + The Datadog Agent sends its logs to Datadog over TCP using port 10516. If that connection is not available, logs fail to be sent and an error is recorded in the `agent.log` file to that effect. You can manually test your connection using OpenSSL, GnuTLS, or another SSL/TLS client. For OpenSSL, run the following command: diff --git a/layouts/shortcodes/logs-tcp-disclaimer.en.md b/layouts/shortcodes/logs-tcp-disclaimer.en.md new file mode 100644 index 0000000000000..8007b2075f918 --- /dev/null +++ b/layouts/shortcodes/logs-tcp-disclaimer.en.md @@ -0,0 +1,2 @@ +
TCP log collection is not supported. Datadog provides no delivery or reliability guarantees when using TCP, and log data may be lost without notice. +For reliable ingestion, use the HTTP intake endpoint, an official Datadog Agent, or forwarder integration instead. For more information, see Log Collection.
\ No newline at end of file From 4a020bbf29b5f12fcaf4c7870c9a1d02d25e512d Mon Sep 17 00:00:00 2001 From: Esther Kim Date: Fri, 7 Nov 2025 17:16:01 -0500 Subject: [PATCH 2/4] Add tcp disclaimer and remove options --- content/en/agent/logs/log_transport.md | 7 ++- content/en/agent/logs/proxy.md | 2 + ...r-logs-collection-troubleshooting-guide.md | 24 --------- .../log-collection-troubleshooting-guide.md | 33 ------------ content/en/logs/log_collection/csharp.md | 52 ------------------- 5 files changed, 7 insertions(+), 111 deletions(-) diff --git a/content/en/agent/logs/log_transport.md b/content/en/agent/logs/log_transport.md index 02af59985a8b7..5766c6a36f880 100644 --- a/content/en/agent/logs/log_transport.md +++ b/content/en/agent/logs/log_transport.md @@ -20,7 +20,7 @@ further_reading: --- -## Default agent behavior +## Default Agent behavior For Agent v6.19+/v7.19+, the default transport used for your logs is compressed HTTPS instead of TCP for the previous versions. When the Agent starts, if log collection is enabled, it runs a HTTPS connectivity test. If successful, then the Agent uses the compressed HTTPS transport, otherwise the Agent falls back to a TCP transport. @@ -33,7 +33,7 @@ To check which transport is used by the Agent, run the [Agent status command][1] **Notes**: -* For older Agent versions, TCP transport is used by default. Datadog strongly recommends you to enforce HTTPS transport if you are running v6.14+/v7.14+ and HTTPS compression if you are running v6.16+/v7.16+. +* For older Agent versions, TCP transport is used by default. **Datadog strongly recommends** you to enforce HTTPS transport if you are running v6.14+/v7.14+ and HTTPS compression if you are running v6.16+/v7.16+. * Always enforce a specific transport (either TCP or HTTPS) when using a proxy to forwards logs to Datadog ## Enforce a specific transport @@ -111,6 +111,9 @@ When logs are sent through HTTPS, use the same [set of proxy settings][3] as the {{% tab "TCP" %}} {{< site-region region="us,eu,us3,us5,ap1,ap2" >}} +{{% logs-tcp-disclaimer %}} + + To enforce TCP transport, update the Agent's [main configuration file][1] (`datadog.yaml`) with: ```yaml diff --git a/content/en/agent/logs/proxy.md b/content/en/agent/logs/proxy.md index 2afff5ba24c0c..6d5a34dd62f6f 100644 --- a/content/en/agent/logs/proxy.md +++ b/content/en/agent/logs/proxy.md @@ -13,6 +13,8 @@ further_reading: text: "Collect your traces" --- +{{% logs-tcp-disclaimer %}} + {{% site-region region="us3,eu,us5,gov,ap1,ap2" %}}
TCP is not available for the {{< region-param key="dd_site_name" >}} site. Contact support for more information. diff --git a/content/en/logs/guide/docker-logs-collection-troubleshooting-guide.md b/content/en/logs/guide/docker-logs-collection-troubleshooting-guide.md index c2ce3943766da..79caf00606368 100644 --- a/content/en/logs/guide/docker-logs-collection-troubleshooting-guide.md +++ b/content/en/logs/guide/docker-logs-collection-troubleshooting-guide.md @@ -134,30 +134,6 @@ If the Logs Agent status looks like the example in [Check the Agent status](#che * The required port (10516) for sending logs to Datadog is being blocked. * Your container is using a different logging driver than the Agent expects. -#### Outbound traffic on port 10516 is blocked - -The Datadog Agent sends its logs to Datadog over TCP using port 10516. If that connection is not available, logs fail to be sent and an error is recorded in the `agent.log` file to that effect. - -You can manually test your connection using OpenSSL, GnuTLS, or another SSL/TLS client. For OpenSSL, run the following command: - -```shell -openssl s_client -connect intake.logs.datadoghq.com:10516 -``` - -For GnuTLS, run the following command: - -```shell -gnutls-cli intake.logs.datadoghq.com:10516 -``` - -And then by sending a log like the following: - -```text - this is a test message -``` - -If opening the port 10516 is not an option, it is possible to configure the Datadog Agent to send logs through HTTPS by setting the `DD_LOGS_CONFIG_FORCE_USE_HTTP` environment variable to `true`: - #### Your containers are not using the JSON logging driver Docker's default is the json-file logging driver so the Container Agent tries to read from this first. If your containers are set to use a different logging driver, the Logs Agent indicates that it is able to successfully find your containers but it isn't able to collect their logs. The Container Agent can also be configured to read from the journald logging driver. diff --git a/content/en/logs/guide/log-collection-troubleshooting-guide.md b/content/en/logs/guide/log-collection-troubleshooting-guide.md index c26f29b3d1782..2962aaf414807 100644 --- a/content/en/logs/guide/log-collection-troubleshooting-guide.md +++ b/content/en/logs/guide/log-collection-troubleshooting-guide.md @@ -23,39 +23,6 @@ There are a number of common issues that can get in the way when [sending new lo Changes in the configuration of the `datadog-agent` won't be taken into account until you have [restarted the Agent][3]. -## Outbound traffic on port 10516 is blocked - -{{% logs-tcp-disclaimer %}} - -The Datadog Agent sends its logs to Datadog over TCP using port 10516. If that connection is not available, logs fail to be sent and an error is recorded in the `agent.log` file to that effect. - -You can manually test your connection using OpenSSL, GnuTLS, or another SSL/TLS client. For OpenSSL, run the following command: - -```shell -openssl s_client -connect intake.logs.datadoghq.com:10516 -``` - -For GnuTLS, run the following command: - -```shell -gnutls-cli intake.logs.datadoghq.com:10516 -``` - -And then by sending a log like the following: - -```text - this is a test message -``` - -- If opening the port 10516 is not an option, it is possible to configure the Datadog Agent to send logs through HTTPS by adding the following in `datadog.yaml`: - -```yaml -logs_config: - force_use_http: true -``` - -See the [HTTPS log forwarding section][4] for more information. - ## Check the status of the Agent Often, checking the [Agent status command][5] results will help you troubleshoot what is happening. diff --git a/content/en/logs/log_collection/csharp.md b/content/en/logs/log_collection/csharp.md index 5536bf2481683..784e07797ff68 100644 --- a/content/en/logs/log_collection/csharp.md +++ b/content/en/logs/log_collection/csharp.md @@ -494,58 +494,6 @@ using (var log = new LoggerConfiguration() } ``` -{{< site-region region="us" >}} - -You can also override the default behavior and forward logs in TCP by manually specifying the following required properties: `url`, `port`, `useSSL`, and `useTCP`. Optionally, [specify the `source`, `service`, `host`, and custom tags.][1] - -For instance to forward logs to the Datadog US region in TCP you would use the following sink configuration: - -```csharp -var config = new DatadogConfiguration(url: "intake.logs.datadoghq.com", port: 10516, useSSL: true, useTCP: true); -using (var log = new LoggerConfiguration() - .WriteTo.DatadogLogs( - "", - source: "", - service: "", - host: "", - tags: new string[] {":", ":"}, - configuration: config - ) - .CreateLogger()) -{ - // Some code -} -``` - -[1]: /logs/log_configuration/attributes_naming_convention/#reserved-attributes - -{{< /site-region >}} -{{< site-region region="eu" >}} - -You can also override the default behavior and forward logs in TCP by manually specifying the following required properties: `url`, `port`, `useSSL`, and `useTCP`. Optionally, [specify the `source`, `service`, `host`, and custom tags.][1] - -For instance to forward logs to the Datadog EU region in TCP you would use the following sink configuration: - -```csharp -var config = new DatadogConfiguration(url: "tcp-intake.logs.datadoghq.eu", port: 443, useSSL: true, useTCP: true); -using (var log = new LoggerConfiguration() - .WriteTo.DatadogLogs( - "", - source: "", - service: "", - host: "", - tags: new string[] {":", ":"}, - configuration: config - ) - .CreateLogger()) -{ - // Some code -} -``` -[1]: /logs/log_configuration/attributes_naming_convention/#reserved-attributes - -{{< /site-region >}} - New logs are now directly sent to Datadog. ## Further Reading From 0f0d28041708266ab744ac608192659641366a03 Mon Sep 17 00:00:00 2001 From: Esther Kim Date: Mon, 10 Nov 2025 11:51:18 -0500 Subject: [PATCH 3/4] Move tcp endpint to separate section, remove trouleshooting info --- content/en/logs/log_collection/_index.md | 118 +++++------------------ 1 file changed, 22 insertions(+), 96 deletions(-) diff --git a/content/en/logs/log_collection/_index.md b/content/en/logs/log_collection/_index.md index 9bb083cb6cf7a..858406c3e6fa7 100644 --- a/content/en/logs/log_collection/_index.md +++ b/content/en/logs/log_collection/_index.md @@ -154,11 +154,6 @@ Use the [site][13] selector dropdown on the right side of the page to see suppor | US | HTTPS | `agent-http-intake.logs.datadoghq.com` | 443 | Used by the Agent to send logs in JSON format over HTTPS. See the [Host Agent Log collection documentation][2]. | | US | HTTPS | `lambda-http-intake.logs.datadoghq.com` | 443 | Used by Lambda functions to send logs in raw, Syslog, or JSON format over HTTPS. | | US | HTTPS | `logs.`{{< region-param key="browser_sdk_endpoint_domain" code="true" >}} | 443 | Used by the Browser SDK to send logs in JSON format over HTTPS. | -| US | TCP | `agent-intake.logs.datadoghq.com` | 10514 | Used by the Agent to send logs without TLS. -| US | TCP and TLS | `agent-intake.logs.datadoghq.com` | 10516 | Used by the Agent to send logs with TLS. -| US | TCP and TLS | `intake.logs.datadoghq.com` | 443 | Used by custom forwarders to send logs in raw, Syslog, or JSON format over an SSL-encrypted TCP connection. | -| US | TCP and TLS | `functions-intake.logs.datadoghq.com` | 443 | Used by Azure functions to send logs in raw, Syslog, or JSON format over an SSL-encrypted TCP connection. **Note**: This endpoint may be useful with other cloud providers. | -| US | TCP and TLS | `lambda-intake.logs.datadoghq.com` | 443 | Used by Lambda functions to send logs in raw, Syslog, or JSON format over an SSL-encrypted TCP connection. | [1]: /api/latest/logs/#send-logs [2]: /agent/logs/#send-logs-over-https @@ -172,9 +167,6 @@ Use the [site][13] selector dropdown on the right side of the page to see suppor | EU | HTTPS | `agent-http-intake.logs.datadoghq.eu` | 443 | Used by the Agent to send logs in JSON format over HTTPS. See the [Host Agent Log collection documentation][2]. | | EU | HTTPS | `lambda-http-intake.logs.datadoghq.eu` | 443 | Used by Lambda functions to send logs in raw, Syslog, or JSON format over HTTPS. | | EU | HTTPS | `logs.`{{< region-param key="browser_sdk_endpoint_domain" code="true" >}} | 443 | Used by the Browser SDK to send logs in JSON format over HTTPS. | -| EU | TCP and TLS | `agent-intake.logs.datadoghq.eu` | 443 | Used by the Agent to send logs in protobuf format over an SSL-encrypted TCP connection. | -| EU | TCP and TLS | `functions-intake.logs.datadoghq.eu` | 443 | Used by Azure functions to send logs in raw, Syslog, or JSON format over an SSL-encrypted TCP connection. **Note**: This endpoint may be useful with other cloud providers. | -| EU | TCP and TLS | `lambda-intake.logs.datadoghq.eu` | 443 | Used by Lambda functions to send logs in raw, Syslog, or JSON format over an SSL-encrypted TCP connection. | [1]: /api/latest/logs/#send-logs [2]: /agent/logs/#send-logs-over-https @@ -252,95 +244,9 @@ Use the [site][13] selector dropdown on the right side of the page to see suppor ### Custom log forwarding -Any custom process or logging library able to forward logs through **TCP** or **HTTP** can be used in conjunction with Datadog Logs. - -{{< tabs >}} -{{% tab "HTTP" %}} - -You can send logs to Datadog platform over HTTP. Refer to the [Datadog Log HTTP API documentation][1] to get started. - -[1]: /api/latest/logs/#send-logs -{{% /tab %}} -{{% tab "TCP" %}} - -{{< site-region region="us" >}} - -You can manually test your connection using OpenSSL, GnuTLS, or another SSL/TLS client. For GnuTLS, run the following command: - -```shell -gnutls-cli intake.logs.datadoghq.com:10516 -``` - -For OpenSSL, run the following command: - -```shell -openssl s_client -connect intake.logs.datadoghq.com:10516 -``` - -You must prefix the log entry with your [Datadog API Key][1] and add a payload. - -``` - Log sent directly using TLS -``` - -Your payload, or `Log sent directly using TLS` as written in the example, can be in raw, Syslog, or JSON format. If your payload is in JSON format, Datadog automatically parses its attributes. - -```text - {"message":"json formatted log", "ddtags":"env:my-env,user:my-user", "ddsource":"my-integration", "hostname":"my-hostname", "service":"my-service"} -``` - -[1]: /account_management/api-app-keys/#api-keys - -{{< /site-region >}} - -{{< site-region region="eu" >}} - -You can manually test your connection using OpenSSL, GnuTLS, or another SSL/TLS client. For GnuTLS, run the following command: - -```shell -gnutls-cli tcp-intake.logs.datadoghq.eu:443 -``` - -For OpenSSL, run the following command: - -```shell -openssl s_client -connect tcp-intake.logs.datadoghq.eu:443 -``` - -You must prefix the log entry with your [Datadog API Key][1] and add a payload. +Any custom process or logging library able to forward logs through **HTTP** can be used in conjunction with Datadog Logs. -``` - Log sent directly using TLS -``` - -Your payload, or `Log sent directly using TLS` as written in the example, can be in raw, Syslog, or JSON format. If your payload is in JSON format, Datadog automatically parses its attributes. - -```text - {"message":"json formatted log", "ddtags":"env:my-env,user:my-user", "ddsource":"my-integration", "hostname":"my-hostname", "service":"my-service"} -``` - -[1]: /account_management/api-app-keys/#api-keys - -{{< /site-region >}} - -{{< site-region region="us3" >}} -The TCP endpoint is not recommended for this site. Contact [support][1] for more information. - -[1]: /help -{{< /site-region >}} - -{{< site-region region="gov,us5,ap1,ap2" >}} - -The TCP endpoint is not supported for this site. - -[1]: /help -{{< /site-region >}} - - -[1]: https://app.datadoghq.com/organization-settings/api-keys -[2]: https://app.datadoghq.com/logs/livetail -{{% /tab %}} -{{< /tabs >}} +You can send logs to Datadog platform over HTTP. Refer to the [Datadog Log HTTP API documentation][15] to get started. **Notes**: @@ -357,6 +263,24 @@ Log events that do not comply with these limits might be transformed or truncate There is an additional truncation in fields that applies only to indexed logs: the value is truncated to 75 KiB for the message field and 25 KiB for non-message fields. Datadog still stores the full text, and it remains visible in regular list queries in the Logs Explorer. However, the truncated version will be displayed when performing a grouped query, such as when grouping logs by that truncated field or performing similar operations that display that specific field. +{{% collapse-content title="TCP" level="h3" expanded=false %}} + +{{% logs-tcp-disclaimer %}} + + +| Site | Type | Endpoint | Port | Description | +|------|-------------|---------------------------------------------------------------------------|--------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| US | TCP | `agent-intake.logs.datadoghq.com` | 10514 | Used by the Agent to send logs without TLS. +| US | TCP and TLS | `agent-intake.logs.datadoghq.com` | 10516 | Used by the Agent to send logs with TLS. +| US | TCP and TLS | `intake.logs.datadoghq.com` | 443 | Used by custom forwarders to send logs in raw, Syslog, or JSON format over an SSL-encrypted TCP connection. | +| US | TCP and TLS | `functions-intake.logs.datadoghq.com` | 443 | Used by Azure functions to send logs in raw, Syslog, or JSON format over an SSL-encrypted TCP connection. **Note**: This endpoint may be useful with other cloud providers. | +| US | TCP and TLS | `lambda-intake.logs.datadoghq.com` | 443 | Used by Lambda functions to send logs in raw, Syslog, or JSON format over an SSL-encrypted TCP connection. | +| EU | TCP and TLS | `agent-intake.logs.datadoghq.eu` | 443 | Used by the Agent to send logs in protobuf format over an SSL-encrypted TCP connection. | +| EU | TCP and TLS | `functions-intake.logs.datadoghq.eu` | 443 | Used by Azure functions to send logs in raw, Syslog, or JSON format over an SSL-encrypted TCP connection. **Note**: This endpoint may be useful with other cloud providers. | +| EU | TCP and TLS | `lambda-intake.logs.datadoghq.eu` | 443 | Used by Lambda functions to send logs in raw, Syslog, or JSON format over an SSL-encrypted TCP connection. | + +{{% /collapse-content %}} + ### Attributes and tags Attributes prescribe [logs facets][9], which are used for filtering and searching in Log Explorer. See the dedicated [attributes and aliasing][10] documentation for a list of reserved and standard attributes and to learn how to support a naming convention with logs attributes and aliasing. @@ -407,3 +331,5 @@ Once logs are collected and ingested, they are available in **Log Explorer**. Lo [12]: /logs/explore/ [13]: /getting_started/site/ [14]: /logs/log_configuration/pipelines/?tab=date#date-attribute +[15]: /api/latest/logs/#send-logs + From 33a616d1a26a823f72870ef2cd4bb8d3e777a01d Mon Sep 17 00:00:00 2001 From: Esther Kim Date: Mon, 10 Nov 2025 16:00:26 -0500 Subject: [PATCH 4/4] Update content/en/agent/logs/proxy.md --- content/en/agent/logs/proxy.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/en/agent/logs/proxy.md b/content/en/agent/logs/proxy.md index 6d5a34dd62f6f..76a4b9d44aeda 100644 --- a/content/en/agent/logs/proxy.md +++ b/content/en/agent/logs/proxy.md @@ -15,7 +15,7 @@ further_reading: {{% logs-tcp-disclaimer %}} -{{% site-region region="us3,eu,us5,gov,ap1,ap2" %}} +{{% site-region region="us3,us5,gov,ap1,ap2" %}}
TCP is not available for the {{< region-param key="dd_site_name" >}} site. Contact support for more information.