-
Notifications
You must be signed in to change notification settings - Fork 7.7k
Proxy Based Stackdriver Telemetry
Since 1.4, Istio supports exporting Stackdriver telemetry (metrics, logs, trace spans, and context graph) directly from proxy instead.
For version before 1.8 (< 1.8), to enable metrics and logs, add the following options into istioctl
control plane configuration, or set them in istioctl
command line:
values:
telemetry:
enabled: true
v2:
enabled: true
stackdriver:
enabled: true # This enables Stackdriver metrics
logging: true # This enables Stackdriver server access logging
For version after 1.8 (>= 1.8), istio Stackdriver adds client side access log reporting, to enable logging on both sides,
values:
telemetry:
enabled: true
v2:
enabled: true
stackdriver:
enabled: true # This enables Stackdriver metrics
inboundAccessLogging: FULL # This enables full Stackdriver server access logging, the other two options are `ERRORS_ONLY` and `NONE`
outboundAccessLogging: FULL # This enables Stackdriver full Stackdriver client access logging, the other two options are `ERRORS_ONLY` and `NONE`
Note: values.telemetry.v2.stackdriver.logging
option is still supported post 1.7, which could only control enablement of server access log.
You can add the equivalent command line arguments to istioctl
command --set values.telemetry.enabled=true --set values.telemetry.v2.enabled=true --set values.telemetry.v2.stackdriver.enabled=true --set values.telemetry.v2.stackdriver.logging=true
.
This will apply several EnvoyFilters to your cluster to enabled Metadata exchange filter and Stackdriver filter inside proxy. After enabling, in Stackdriver UI you will find:
- istio.io prefixed metrics be reported
- Server access log named as
server-accesslog-stackdriver
for every mesh request. Client access log named asclient-accesslog-stackdriver
Here is an example log entry
{
"insertId": "1j84zg8g68vb62z",
"httpRequest": {
"requestMethod": "GET",
"requestUrl": "http://35.235.89.201:80/productpage",
"requestSize": "795",
"status": 200,
"responseSize": "7005",
"remoteIp": "10.168.0.26:0",
"serverIp": "10.36.3.153:9080",
"latency": "0.229384205s",
"protocol": "http"
},
"resource": {
"type": "k8s_container",
"labels": {
"cluster_name": "istio-e2e22",
"namespace_name": "istio-bookinfo-1-68819",
"container_name": "productpage",
"project_id": "***",
"location": "us-west2-a",
"pod_name": "productpage-v1-64794f5db4-8xbtf"
}
},
"timestamp": "2020-08-13T21:37:42.963881Z",
"severity": "INFO",
"labels": {
"protocol": "http",
"upstream_host": "127.0.0.1:9080",
"source_canonical_service": "istio-ingressgateway",
"source_namespace": "istio-system",
"x-envoy-original-path": "",
"source_canonical_revision": "latest",
"connection_id": "32",
"upstream_cluster": "inbound|9080|http|productpage.istio-bookinfo-1-68819.svc.cluster.local",
"requested_server_name": "outbound_.9080_._.productpage.istio-bookinfo-1-68819.svc.cluster.local",
"destination_version": "v1",
"destination_workload": "productpage-v1",
"source_workload": "istio-ingressgateway",
"destination_canonical_revision": "v1",
"mesh_uid": "cluster.local",
"source_principal": "spiffe://cluster.local/ns/istio-system/sa/istio-ingressgateway-service-account",
"x-envoy-original-dst-host": "",
"service_authentication_policy": "MUTUAL_TLS",
"destination_principal": "spiffe://cluster.local/ns/istio-bookinfo-1-68819/sa/bookinfo-productpage",
"response_flag": "-",
"log_sampled": "false",
"destination_service_host": "productpage.istio-bookinfo-1-68819.svc.cluster.local",
"destination_name": "productpage-v1-64794f5db4-8xbtf",
"destination_canonical_service": "productpage",
"destination_namespace": "istio-bookinfo-1-68819",
"source_name": "istio-ingressgateway-6845f6d664-lnfvp",
"source_app": "istio-ingressgateway",
"destination_app": "productpage",
"request_id": "39013650-4e62-9be2-9d25-78682dd27ea4",
"route_name": "default"
},
"logName": "projects/***/logs/server-accesslog-stackdriver",
"trace": "projects/***t/traces/466d77d15753cb4d7749ba5413b5f70f",
"receiveTimestamp": "2020-08-13T21:37:48.758673203Z",
"spanId": "633831cb1fda4fd5",
"traceSampled": true
}
Helm installation does not support these options. If you are using Helm, alternatively you could generate the manifest with istioctl
, extract out the necessary EnvoyFilter
configuration, and apply them manually. Specifically run istioctl manifest generate
with the aforementioned values, look for and apply versioned Stackdriver and Metadata Exchange EnvoyFilter, e.g. stackdriver-filter-1.5
, metadata-exchange-1.5
.
To enable Stackdriver telemetry for a subset of workloads or a namespace, an EnvoyFilter has to be applied in the namespace, or with desired workload selector. Here is an example EnvoyFilter for 1.7. Please refer to EnvoyFilter and Stackdriver Extension configuration reference for configuration composition.
To get the EnvoyFilter for a particular release as start point for configuration composition, use istioctl
to generate the manifest, and get Stackdriver
EnvoyFilter (stackdriver-filter-x.x
) from the generated manifest.
To enable tracing, set the following options in istioctl
installation: --set values.global.enableTracing=true --set values.global.proxy.tracer=stackdriver
. This will enable Envoy proxy to export traces directly to Stackdriver. You can adjust sampling rate using pilot.traceSampling
option as other tracers.
To enable topology, set the following options in istioctl
installation: --set value s.telemetry.v2.stackdriver.topology=true
. This will enable Envoy proxy to export context graph edge to Stackdriver.
Visit istio.io to learn how to use Istio.
- Preparing for Development Mac
- Preparing for Development Linux
- Troubleshooting Development Environment
- Repository Map
- GitHub Workflow
- Github Gmail Filters
- Using the Code Base
- Developing with Minikube
- Remote Debugging
- Verify your Docker Environment
- Istio Test Framework
- Working with Prow
- Test Grid
- Code Coverage FAQ
- Writing Good Integration Tests
- Test Flakes
- Release Manager Expectations
- Preparing Istio Releases
- 1.5 Release Information
- 1.6 Release Information
- 1.7 Release Information
- 1.8 Release Information
- 1.9 Release Information
- 1.10 Release Information
- 1.11 Release Information
- 1.12 Release Information
- 1.13 Release Information
- 1.14 Release Information
- 1.15 Release Information
- 1.16 Release Information
- 1.17 Release Information
- 1.18 Release Information
- 1.19 Release Information
- 1.20 Release Information
- 1.21 Release Information
- 1.22 Release Information
- 1.23 Release Information
- 1.24 Release Information
- Collecting Logs and Debug Info
- Dependency FAQ
- Working with discuss.istio.io
- Developing with and hosting upon OpenShift
- Adapter Dev Guide
- Adapter Walkthrough
- Attribute Generating Adapter Walkthrough
- Route Directive Adapter Development Guide
- Out of Tree Adapter Walkthrough
- Running a Local Instance
- Template Dev Guide
- Using a Custom Adapter
- Publishing Adapters and Templates to istio.io
- Enabling Envoy Authorization Service and gRPC Access Log Service With Mixer