New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Access log with JSON encoding #12232
Comments
@njyeti can you grab the format string from envoy in the
|
Here you go. Encoding: TEXT Encoding: JSON |
This is still present in rc4. I put up a PR to fix the first part here: #12473 (I'm not actually sure what the second problem is referring to) |
Hi @dwradcliffe, Regarding the 2nd problem, if we want to capture the HTTP request or response header in the access log, currently, it is does not capture custom value the JSON format but works fine when we use the TEXT format. For example, if we have the below access log pattern where we have defined custom header for request and response: But, JSON encoding will output only: whereas Envoy supports customer header as mentioned https://www.envoyproxy.io/docs/envoy/latest/configuration/access_log#format-dictionaries |
What's your custom accessLogFormat for JSON? |
hmmm, is it the same format being used for both TEXT and JSON? Snippet from the 'istio" configMap |
If you want to use custom vars and JSON you need to put the mapping in accessLogFormat in json, not the same string you used for text. I’m guessing it failed to decide json and failed back to the default format. |
got it. Will try it out quickly. Thank you!! |
bug fix for the timestamp merged - marking as closed unless further issues detected |
Hello @njyeti, |
@my3sons,
If I remembered correctly there Is a way to update these using helm install as well but didn't remember the cmd and args top of my head |
@njyeti, |
@dwradcliffe Quick question: how come Istio decided to use string interpolation (which doesn't properly escape values) when constructing the JSON value? Isn't that obviously bugged? |
I am having the exact same issue. Using the following in
has no effect at all....It keeps printing the default log... Any suggestions? |
When the accessLogEncoding is set to JSON, accessLogFormat is not used (whether that is a bug or by design, I cannot say). This makes sense, as if you want to overrule elements in the accessLogFormat and are expecting JSON, you have to write a JSON alike template anyway. This is what worked for me
This did not work for me
And also the following did not work
Bottom line, if you want to modify the fields (add/modify/remove), you just fall back on accessLogFormat TEXT, where you have full control anyway. The accessLogEncoding JSON seems a shortcut feature to quickly switch from TEXT to JSON, but do not use it when tweaking is needed. Note: this way my JSON fields are ordered in alphabetical order, which I like more. |
This worked for me: meshConfig:
accessLogFormat: |
{
"authority": "%REQ(:AUTHORITY)%",
"bytes_received": "%BYTES_RECEIVED%",
"bytes_sent": "%BYTES_SENT%",
"downstream_local_address": "%DOWNSTREAM_LOCAL_ADDRESS%",
"downstream_peer_cert_v_end": "%DOWNSTREAM_PEER_CERT_V_END%",
"downstream_peer_cert_v_start": "%DOWNSTREAM_PEER_CERT_V_START%",
"downstream_remote_address": "%DOWNSTREAM_REMOTE_ADDRESS%",
"downstream_tls_cipher": "%DOWNSTREAM_TLS_CIPHER%",
"downstream_tls_version": "%DOWNSTREAM_TLS_VERSION%",
"duration": "%DURATION%",
"hostname": "%HOSTNAME%",
"istio_policy_status": "%DYNAMIC_METADATA(istio.mixer:status)%",
"method": "%REQ(:METHOD)%",
"path": "%REQ(X-ENVOY-ORIGINAL-PATH?:PATH)%",
"protocol": "%PROTOCOL%",
"request_duration": "%REQUEST_DURATION%",
"request_id": "%REQ(X-REQUEST-ID)%",
"requested_server_name": "%REQUESTED_SERVER_NAME%",
"response_code": "%RESPONSE_CODE%",
"response_duration": "%RESPONSE_DURATION%",
"response_tx_duration": "%RESPONSE_TX_DURATION%",
"response_flags": "%RESPONSE_FLAGS%",
"route_name": "%ROUTE_NAME%",
"start_time": "%START_TIME%",
"upstream_cluster": "%UPSTREAM_CLUSTER%",
"upstream_host": "%UPSTREAM_HOST%",
"upstream_local_address": "%UPSTREAM_LOCAL_ADDRESS%",
"upstream_service_time": "%RESP(X-ENVOY-UPSTREAM-SERVICE-TIME)%",
"upstream_transport_failure_reason": "%UPSTREAM_TRANSPORT_FAILURE_REASON%",
"user_agent": "%REQ(USER-AGENT)%",
"x_forwarded_for": "%REQ(X-FORWARDED-FOR)%",
"x_graphql_operation": "%REQ(X-GRAPHQL-OPERATION)%",
"x_graphql_operation_name": "%REQ(X-GRAPHQL-OPERATION-NAME)%",
"x_graphql_response_status": "%RESP(X-GRAPHQL-RESPONSE-STATUS)%",
"x_graphql_root_operation": "%REQ(X-PROP-GRAPHQL-OPERATION)%",
"x_graphql_root_operation_name": "%REQ(X-PROP-GRAPHQL-OPERATION-NAME)%"
}
accessLogEncoding: "JSON"
accessLogFile: "/dev/stdout" |
Since this issue not solved officially, I suppose this ticket should be re-open. @GregHanson verified not working on Istio 1.7.4. |
Hi guys, is there any way to define JSON ordering for access log in JSON format?
@boeboe What do you mean "ordered" in this message? |
BTW below config works for me, which is istio 1.8.1
|
I have problem with the mesh config, and I wonder how to apply I have two ingresses installed with different selector, due to the fact I route traffic in a way I need my ingresses totally separated. When I apply the This makes the JSON schema and the JSON encoding not to be applied on my ingresses, but only to work on default ingresses. Any ideas ? |
@yuliyantsvetkov what version of Istio are you using? You might be better off opening a new issue, and providing your meshConfig access log settings. The access log fields in meshconfig (spec here) are not conditional on the |
Istio: 1.1.0-rc.2
K8s: v1.11.5-gke.5
I'm currently seeing two issues while using the JSON encoding for the access log but works fine with the TEXT
Text encoding:
'2019-03-04T18:48:42.884Z' 'foo' 'GET' /prod/v1 HTTP/1.1 200 - 0 3173 3 2 10.138.0.20 'curl/7.54.0' b1dc5045-0975-46f5-b217-5fa1eb3876af website.com 10.52.4.52:80 outbound|80|version-2|website-path.path.svc.cluster.local - 10.52.1.77:80 10.138.0.20:53461
JSON encoding:
{ "upstream_cluster": "outbound|80|version-1|website-path.path.svc.cluster.local", "downstream_remote_address": "10.138.0.17:43587", "path": "\/prod\/v1", "authority": "website.com", "protocol": "HTTP\/1.1", "upstream_service_time": "2", "upstream_local_address": "-", "duration": "3", "downstream_local_address": "10.52.3.56:80", "response_code": "200", "user_agent": "curl\/7.54.0", "response_flags": "-", "start_time": "2019-03-04T18:50:40.527Z", "method": "2019-03-04T18:50:40.527Z", "request_id": "bf57fa95-3a15-4078-9353-ad5a74e2ce5e", "upstream_host": "10.52.2.15:80", "x_forwarded_for": "10.138.0.17", "requested_server_name": "-", "bytes_received": "0", "istio_policy_status": "-", "bytes_sent": "3176" }
istio-configmap-1.1.0-rc.2.yaml
The text was updated successfully, but these errors were encountered: