unknown-service-name in Jaeger when localEndpoint.serviceName is missing #39
Comments
What would you like to see for each of those spans instead? |
That is a great question, I hoped that you would have some answers to that :) But as a minimum it would be great to at least default to For spans that are children of auth-service in this case I would have expected that they would have the same serviceName as their parent. i.e. |
I have several answers, none of which I'm totally happy with 😛 One problem I think about is how kong is "pretending" to be the service. However from the POV of the client: it is the service. But from the POV of the service, kong is just a client. It can get a bit complex with how things might evolve: e.g. if within the |
I agree with you that it could get complex in some cases. Is it too early in the chain to get the consumer.id in the I would argue that it can be of interest to see the chain from the service point of view, including kong as a client to your service, i.e. setting the serviceName to |
Theoretically, yes it's too early. In practice, sometimes it's possible. Despite the span starting before we know where the request is going, we don't currently ship the log to zipkin until after the request has finished, which means we can edit it before it is sent. However not all requests will have a (known) service; and also in the future to support long running requests, we're considering sending the logs before the log phase. As a jaegar/zipkin user: what do you expect to see for in the service field(s) for those spans? e.g. should some contain "my-kong-proxy"? |
I would have expected the first two spans to be "my-kong-proxy", i.e. the servicename for the kong instance. For the spans which have auth-service as parent I would be totally fine with either "my-kong-proxy" or the service name (auth-service) |
Where does that string |
We are running kong in a kubernetes environment. So from our point of view I would have preferred an environment variable what we can specify from the outside. env:
- name: KONG_ZIPKIN_PLUGIN_SERVICE_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name This is also suggested in #23 |
Just saw this as I was hitting those unknown spans too, would it make sense to start the first one to populate with kong or a config value. Then once kong knows the service name to allow that tag to be the service name through the rest of the proxy phases? Plenty of possibilities though. Edit - Looking into the codebase a bit I decided to log to stdout the json that gets sent:
@james-callahan What do you think about just having the first span localEndpoint say Some configurable value from the schema, defaults to Kong, or null as it behaves now but could be renamed to whatever a team wants? Then rather than just carrying the "localEndpoint":{"serviceName":"F5MonitorService"} at the kong.proxy phase only why not set it at every phase other than the first "configed" one? I could probably get my group to PR something like that if yall see it as appropriate, any impl is better than unknown-service-name in almost all spans haha. |
From a jaeger-collector's point of view, logs are full of warning messages due to this issue:
|
Wouldn't you want Kong to have different service names depending on which service it is proxying for/to? |
IMHO I think it depends if Kong is proxying north/south traffic or if Kong is proxying east/west traffic. In the first case (i.e. Kong proxying traffic north/south), the suggestion proposed by @goober would be a good solution. |
@secat our current setup is a north-south type where only ingress requests are proxied through kong. All in-cluster communication between services are handled directly without going through kong. That is why we think it would solve our issue if the kong instance could be named separately to the service that was actually called. |
@james-callahan we are currently deploying three (3) kong gateway on our kubernetes cluster and we would like to discriminate from which Kong proxy the traffic is coming from. Having the possibility to configure the service name as poposed in @goober comment would be much appreciated. |
We are using Jaeger for collecting trace-information from our services and have just installed the zipkin-plugin for Kong (v1.0.3). However, the serviceName for kong is missing, and instead unknown-service-name is presented, as seen in the screenshot below.
As of my understanding, Jeager takes the
serviceName
from thelocalEndpoint
, which works fine for thekong.proxy
span that can be seen in the screenshot above and in reporter.lua#L42-L54.#12 resolved the above problem in cases where the the
peer.service
exists, but I expected that a default value for the serviceName would be set to kong in other cases.Maybe this is a duplicate of #23 and if that is the case we could continue the discussion in that issue?
Update
This is how Jaeger determines the serviceName from a zipkin span:
to_domain.go#L206-L233
The text was updated successfully, but these errors were encountered: