Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migration to 1.22 missed Zipkin port change #1441

Closed
gsagula opened this issue Apr 26, 2021 · 10 comments · Fixed by #1449
Closed

Migration to 1.22 missed Zipkin port change #1441

gsagula opened this issue Apr 26, 2021 · 10 comments · Fixed by #1449
Assignees
Labels
bug Something isn't working good first issue Good for newcomers help wanted Extra attention is needed

Comments

@gsagula
Copy link

gsagula commented Apr 26, 2021

Describe the bug
Starting on jaegertracing/jaeger-operator:1.22.1, jaeger-collector does not start Zipkin server automatically.

{"level":"info","ts":1619421380.7131286,"caller":"server/zipkin.go:48","msg":"Not listening for Zipkin HTTP traffic, port not configured"}

CR

apiVersion: jaegertracing.io/v1
kind: Jaeger
metadata:
  name: mon-jaeger
  namespace: observability
spec:
  log-level: debug
  strategy: streaming
  sampling:
    options:
      default_strategy:
        type: const
        param: 100 
  collector:
    options:
      kafka: 
        producer:
          topic: jaeger-spans
          brokers: knative-kafka-cluster.knative-kafka:9092
  ingester:
    options:
      kafka:
        consumer:
          topic: jaeger-spans
          brokers: knative-kafka-cluster.knative-kafka:9092
      ingester:
        deadlockInterval: 0s
  storage:
    type: elasticsearch
    options:
      es:
        server-urls: https://elastic-es-http:9200
        index-prefix: jaeger-tracing
        tls:
          ca: /es/certificates/ca.crt
    secretName: jaeger-secret
  volumeMounts:
    - name: certificates
      mountPath: /es/certificates/
      readOnly: true
  volumes:
    - name: certificates
      secret:
        secretName: elastic-es-http-certs-public

Discussion
jaegertracing/jaeger#2959

To Reproduce
Steps to reproduce the behavior:

  1. Install operator jaegertracing/jaeger-operator:1.22.1 with log-level: debug
  2. Fire a valid CR to deploy Jaeger.
  3. Grab the collector's logs

You should see the message "Not listening for Zipkin HTTP traffic, port not configured" and Jaeger does not take any Zipking trace.

Expected behavior
Collector should start Zipkin server automatically.

Version (please complete the following information):

  • OS: Ubuntu 20.04
  • Jaeger version: 1.22.1
  • Deployment: Kubernetes - 1.18.16-gke.502

What troubleshooting steps did you try?
The same CR works with the previous version jaegertracing/jaeger-operator:1.21.3.

{"level":"info","ts":1619449101.7558992,"caller":"server/zipkin.go:49","msg":"Listening for Zipkin HTTP traffic","zipkin host-port":":9411"} 

cc @jpkrohling

@jpkrohling
Copy link
Contributor

@rubenvp8510, could you please take a look at this one?

@jeremyzahner
Copy link

@jpkrohling @rubenvp8510 Debugged this quickly because i ran into the same problem.
Seems like the Operator still reconciles the deployments with the env variable COLLECTOR_ZIPKIN_HTTP_PORT.
Since this (or better, the underlying arg --collector.zipkin.http-port) has been changed (in https://github.com/jaegertracing/jaeger/releases/tag/v1.22.0) to ...host-port instead, this is probably what's causing the issue.

@jeremyzahner
Copy link

jeremyzahner commented Apr 29, 2021

@gsagula You can fix it (for now) by using the following spec:

apiVersion: jaegertracing.io/v1
kind: Jaeger
metadata:
  name: tracing
spec:
  ...
  collector:
    options:
      collector.zipkin.host-port: ':9411'
     ...

(I've only verified this using the allInOne strategy.)

@gsagula
Copy link
Author

gsagula commented Apr 30, 2021

Thanks @jeremyzahner

I did that but with "9411" instead of ":9411". It's working now!

@jpkrohling jpkrohling transferred this issue from jaegertracing/jaeger Apr 30, 2021
@github-actions github-actions bot added the needs-triage New issues, in need of classification label Apr 30, 2021
@jpkrohling jpkrohling changed the title Collector does not start Zipkin server automatically Migration to 1.22 missed Zipkin port change Apr 30, 2021
@jpkrohling jpkrohling added bug Something isn't working good first issue Good for newcomers help wanted Extra attention is needed and removed needs-triage New issues, in need of classification labels Apr 30, 2021
@jpkrohling
Copy link
Contributor

Would one of you be interested in contributing a fix? Looks like we should have migrated the Zipkin parameter and value. This is where the code needs to be added:

func upgrade1_22_0(ctx context.Context, client client.Client, jaeger v1.Jaeger) (v1.Jaeger, error) {
flagMapCollector := []deprecationFlagMap{{
from: "jaeger.tags",
to: "collector.tags",
}}
flagMapAgent := []deprecationFlagMap{{
from: "jaeger.tags",
to: "agent.tags",
}}
flagMapQuery := []deprecationFlagMap{
{
from: "downsampling.hashsalt",
to: "",
},
{
from: "downsampling.ratio",
to: "",
},
}
j := &jaeger
j.Spec.AllInOne.Options = migrateDeprecatedOptions(j, j.Spec.AllInOne.Options, flagMapCollector)
j.Spec.Collector.Options = migrateDeprecatedOptions(j, j.Spec.Collector.Options, flagMapCollector)
j.Spec.Agent.Options = migrateDeprecatedOptions(j, j.Spec.Agent.Options, flagMapAgent)
j.Spec.Query.Options = migrateDeprecatedOptions(j, j.Spec.Query.Options, flagMapQuery)
return migrateCassandraVerifyFlagv1_22_0(jaeger), nil
}

@gsagula
Copy link
Author

gsagula commented May 3, 2021

@jpkrohling You can assign this issue to me if nobody else is taking it.

@jpkrohling
Copy link
Contributor

Thanks for volunteering, @gsagula!

@gsagula
Copy link
Author

gsagula commented May 12, 2021

Sorry guys, I just started looking into this one today but I see that @rubenvp8510 already did it.

@jpkrohling
Copy link
Contributor

That's ok, we have plenty of other issues if you are interested :-)

@gsagula
Copy link
Author

gsagula commented May 12, 2021

Awesome, I can give this one a shot if nobody else is taking it. #1422

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working good first issue Good for newcomers help wanted Extra attention is needed
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants