fix: use spec.tls for telemetry backends instead of BackendTLSPolicy

[codefromthecrypt]

Description

EG processBackendRefs does not look up BackendTLSPolicy for telemetry backends (e.g. OTLP collectors), so TLS never gets configured. Use Backend spec.tls with wellKnownCACertificates instead, which processBackendRefs does support. Route backends continue to use BackendTLSPolicy as before.

Here's a screenshot from elastic on this working now

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 84.22%. Comparing base (7f77c8a) to head (73a8d05).

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #1964   +/-   ##
=======================================
  Coverage   84.22%   84.22%           
=======================================
  Files         128      128           
  Lines       17828    17829    +1     
=======================================
+ Hits        15016    15017    +1     
  Misses       1868     1868           
  Partials      944      944           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[codefromthecrypt]

FYI, I will chase up a root fix in EG later, but this will help avoid folks wanting to try remote otel clusters and failing with aigw

[nacx]

/retest

[dosubot]

Documentation Updates

1 document(s) were updated by changes in this PR:

tracing

View Changes

@@ -197,6 +197,29 @@
 --unset extProc.extraEnvVars`}
 </CodeBlock>
 
+## TLS Configuration for OTLP Endpoints
+
+When using HTTPS endpoints for your OTLP collector (e.g., `https://production-collector:4317`),
+the AI Gateway automatically configures TLS using the system's trusted CA certificates
+(`wellKnownCACertificates: System`). This differs from route backends, which use
+`BackendTLSPolicy` for TLS configuration.
+
+To use a secure OTLP collector, simply specify an `https://` URL in your
+`OTEL_EXPORTER_OTLP_ENDPOINT` environment variable. The system handles TLS
+configuration automatically—no additional BackendTLSPolicy is required.
+
+For example:
+
+```yaml
+extProc:
+  extraEnvVars:
+    - name: OTEL_EXPORTER_OTLP_ENDPOINT
+      value: "https://secure-collector.example.com:4317"
+```
+
+The examples in this guide use HTTP endpoints for simplicity, but production
+deployments should use HTTPS for secure trace transmission.
+
 ## Per-Gateway Configuration
 
 For deployments with multiple Gateways that need different tracing configurations,
@@ -218,7 +241,7 @@
   extProc:
     env:
       - name: OTEL_EXPORTER_OTLP_ENDPOINT
-        value: "http://production-collector:4317"
+        value: "https://production-collector:4317"
       - name: OTEL_SERVICE_NAME
         value: "ai-gateway-production"
 ---

^{How did I do? Any feedback?}  

[codefromthecrypt]

envoyproxy/gateway#8545 for the tech debt in EG