jaegertracing/jaeger

Backend Changes

Bug fixes, Minor Improvements

• Make agent timeout for reporting configurable and fix flags overriding (#1034, @gouthamve)
• Fix metrics handler registration in agent (#1178, @pavolloffay)

Backend Changes

Breaking Changes

• Refactor agent configuration (#1092, @pavolloffay)

The following agent flags has has been deprecated in order to support multiple reporters:

--collector.host-port
--discovery.conn-check-timeout
--discovery.min-peers

New flags:

--reporter.tchannel.host-port
--reporter.tchannel.discovery.conn-check-timeout
--reporter.tchannel.discovery.min-peers

• Various changes around metrics produced by jaeger-query: Names scoped to the query component, generated for all span readers (not just ES), consolidate query metrics and include result tag (#1074, #1075 and #1096, @objectiser)

For example, sample of metrics produced for find_traces operation before:

jaeger_find_traces_attempts 1
jaeger_find_traces_errLatency_bucket{le="0.005"} 0
jaeger_find_traces_errors 0
jaeger_find_traces_okLatency_bucket{le="0.005"} 0
jaeger_find_traces_responses_bucket{le="0.005"} 1
jaeger_find_traces_successes 1

And now:

jaeger_query_latency_bucket{operation="find_traces",result="err",le="0.005"} 0
jaeger_query_latency_bucket{operation="find_traces",result="ok",le="0.005"} 2
jaeger_query_requests{operation="find_traces",result="err"} 0
jaeger_query_requests{operation="find_traces",result="ok"} 2
jaeger_query_responses_bucket{operation="find_traces",le="0.005"} 2

New Features

• Configurable deadlock detector interval for ingester (#1134, @marqc)
• Emit spans for elastic storage backend (#1128, @annanay25)
• Allow to use TLS certificates for Elasticsearch authentication (#1139, @clyang82)
• Add ingester metrics, healthcheck and rename Kafka cli flags (#1094, @ledor473)
• Add a metric for number of partitions held (#1154, @vprithvi)
• Log jaeger-collector tchannel port (#1136, @mindaugasrukas)
• Support tracer env based initialization in hotrod (#1115, @eundoosong)
• Publish ingester as binaries and docker image (#1086, @ledor473)
• Use Go 1.11 (#1104, @isaachier)
• Tag images with commit SHA and publish to -snapshot repository (#1082, @pavolloffay)

Bug fixes, Minor Improvements

• Fix child span context while tracing cassandra queries (#1131, @annanay25)
• Deadlock detector hack for Kafka driver instability (#1087, @vprithvi)
• Fix processor overriding data in a buffer (#1099, @pavolloffay)

UI Changes

New Features

• Span Search - Highlight search results (#238), @davit-y
• Span Search - Improve search logic (#237), @davit-y
• Span Search - Add result count, navigation and clear buttons (#234), @davit-y

Bug Fixes, Minor Improvements

• Use correct duration format for scatter plot (#266), @tiffon)
• Fix collapse all issues (#264), @tiffon)
• Use a moderately sized canvas for the span graph (#257), @tiffon)

UI Changes

• Compare two traces (#228, @tiffon)
• Make tags clickable (#223, @divdavem)
• Directed graph as React component (#224, @tiffon)
• Timeline Expand and Collapse Features (#221, @davit-y)
• Integrate Google Analytics into Search Page (#220, @davit-y)

Backend Changes

Breaking changes

• jaeger-standalone binary has been renamed to jaeger-all-in-one. This change also includes package rename from standalone to all-in-one (#1062, @pavolloffay)

New Features

• (Experimental) Allow storing tags as object fields in Elasticsearch for better Kibana support(#1018, @pavolloffay)
• Enable tracing of Cassandra queries (#1038, @yurishkuro)
• Make Elasticsearch index configurable (#1009, @pavolloffay)
• Add flags to allow changing ports for HotROD services (#951, @cboornaz17)
• (Experimental) Kafka ingester (#952, #942, #944, #940, @davit-y and @vprithvi)
• Use tags in agent metrics (#950, @eundoosong)
• Add support for Cassandra reconnect interval (#934, @nyanshak)

Backend Changes

Breaking Changes!!!

• The storage implementations no longer write the parentSpanID field to storage (#856).
  If you are upgrading to this version, you must upgrade query service first!

• Update Dockerfiles to reference executable via ENTRYPOINT (#815) by Zachary DiCesare (@zdicesare)

  It is no longer necessary to specify the binary name when passing flags to containers.
  For example, to execute the help command of the collector, instead of

  $ docker run -it --rm jaegertracing/jaeger-collector /go/bin/collector-linux help
  

  run

  $ docker run -it --rm jaegertracing/jaeger-collector help
  

• Detect HTTP payload format from Content-Type (#916) by Yuri Shkuro (@yurishkuro)

  When submitting spans in Thrift format to HTTP endpoint /api/traces,
  the format argument is no longer required, but the Content-Type header
  must be set to "application/vnd.apache.thrift.binary".

• Change metric tag from "service" to "svc" (#883) by Won Jun Jang (@black-adder)

New Features

• Add Kafka as a Storage Plugin (#862) by David Yeghshatyan (@davit-y)

  The collectors can be configured to write spans to Kafka for further data mining.

• Package static assets inside the query-service binary (#918) by Yuri Shkuro (@yurishkuro)

  It is no longer necessary (but still possible) to pass the path to UI static assets
  to jaeger-query and jaeger-standalone binaries.

• Replace domain model with Protobuf/gogo-generated model (#856) by Yuri Shkuro (@yurishkuro)

  First step towards switching to Protobuf and gRPC.

• Include HotROD binary in the distributions (#917) by Yuri Shkuro (@yurishkuro)

• Improve HotROD demo (#915) by Yuri Shkuro (@yurishkuro)

• Add DisableAutoDiscovery param to cassandra config (#912) by Bill Westlin (@whistlinwilly)

• Add connCheckTimeout flag to agent (#911) by Henrique Rodrigues (@Henrod)

• Ability to use multiple storage types (#880) by David Yeghshatyan (@davit-y)

Minor Improvements

• [ES storage] Log number of total and failed requests (#902) by Tomasz Adamski (@tmszdmsk)
• [ES storage] Do not log requests on error (#901) by Tomasz Adamski (@tmszdmsk)
• [ES storage] Do not exceed ES _id length limit (#905) by Łukasz Harasimowicz (@harnash) and Tomasz Adamski (@tmszdmsk)
• Add cassandra index filter (#876) by Won Jun Jang (@black-adder)
• Close span writer in standalone (#863) (4 weeks ago) by Pavol Loffay (@pavolloffay)
• Log configuration options for memory storage (#852) (6 weeks ago) by Juraci Paixão Kröhling (@jpkrohling)
• Update collector metric counters to have a name (#886) by Won Jun Jang (@black-adder)
• Add CONTRIBUTING_GUIDELINES.md (#864) by (@PikBot)

Backend Changes

• Add bounds to memory storage (#845) by Juraci Paixão Kröhling (@jpkrohling)
• Add metric for debug traces (#796) by Won Jun Jang (@black-adder)
• Change metrics naming scheme (#776) by Juraci Paixão Kröhling (@jpkrohling)
• Remove ParentSpanID from domain model (#831) by Yuri Shkuro (@yurishkuro)
• Add ability to adjust static sampling probabilities per operation (#827) by Won Jun Jang (@black-adder)
• Support log-level flag on agent (#828) by Won Jun Jang (@black-adder)
• Add healthcheck to standalone (#784) by Eundoo Song (@eundoosong)
• Do not use KeyValue fields directly and use KeyValues as decorator only (#810) by Yuri Shkuro (@yurishkuro)
• Upgrade to go 1.10 (#792) by Prithvi Raj (@vprithvi)
• Do not create Cassandra index if it already exists (#782) by Greg Swift (@gregswift)

UI Changes

• None

Backend Changes

• Publish binaries for Linux, Darwin, and Windows (#765) - thanks to @grounded042

UI Changes

New Features

• View Trace JSON buttons return formatted JSON (fixes #199)

Backend Changes

New Features

• Support traces with >10k spans in Elasticsearch (#668) - thanks to @sramakr

Fixes

• Allow slash '/' in service names (#586)
• Log errors from HotROD services (#769)

Highlights

HotROD Demo

By popular demand, the HotROD demo is now available as a Docker image, jaegertracing/example-hotrod.

Centrally Configurable Sampling Policies

Jaeger clients have long supported centrally controlled per-service sampling policies. We are still working on rolling out full adaptive sampling to the open source version, but now the collectors allow configuring sampling strategies via a configuration file (see Sampling section in the documentation) that can be pushed to the clients without manual re-configuration of the microservices.

Running Jaeger UI Behind Reverse Proxy

Another popular request is to support running Jaeger UI with a custom site prefix, such as mycompany.com/jaeger. Now possible with the new query.base-path command line argument or the QUERY_BASE_PATH environment variable.

Google Analytics Support in the UI

If you are running a tracing team in your company, you probably want to measure how other engineers are using Jaeger. Jaeger UI supports Google Analytics, see Deployment documentation on how to configure it.

Metrics Improvements

We are still tweaking how the backend emits metrics, check the issues list below.

Backend Changes

New Features

• Add sampling handler with file-based configuration for agents to query (#720) (#674)
• Allow overriding base path for UI/API routes and remove --query.prefix (#748)
• Add Dockerfile for hotrod example app (#694) <Guilherme Baufaker Rêgo>
• Publish hotrod image to docker hub (#702)
• Dockerize es-index-cleaner script (#741)
• Add a flag to control Cassandra consistency level (#700)
• Collect metrics from ES bulk service (#688)
• Allow zero replicas for Elasticsearch (#754)

Fixes

• Apply namespace when creating Prometheus metrics factory (fix for #732) (#733)
• Disable double compression on Prom Handler - fixes #697 (#735) <Juraci Paixão Kröhling>
• Use the default metricsFactory if not provided (#739)
• Avoid duplicate expvar metrics - fixes #716 (#726)
• Make sure different tracers in HotROD process use different random generator seeds (#718)
• Test that processes with identical tags are deduped (#708)
• When converting microseconds to time.Time ensure UTC timezone (#712)
• Add to WaitGroup before the goroutine creation (#711)
• Pin testify version to ^1.2.1 (#710)

UI Changes

New Features

• Support running Jaeger behind a reverse proxy (fixes #42)
• Track Javascript errors via Google Analytics (fixes #39)
• Add Google Analytics event tracking for actions in trace view (#191)

Fixes

• Clearly identify traces without a root span (fixes #190)
• Fix #166 JS error on search page after viewing 404 trace

Backend Changes

New Features

• Use elasticsearch bulk API (#656)
• Support archive storage in the query-service (#604)
• Introduce storage factory framework and composable CLI (#625)
• Make agent host port configurable in hotrod (#663)
• Add signal handling to standalone (#657)

Fixes

• Remove the override of GOMAXPROCS (#679)
• Use UTC timezone for ES indices (#646)
• Fix elasticsearch create index race condition error (#641)

UI Changes

New Features

• Use Ant Design instead of Semantic UI (jaegertracing/jaeger-ui#169)
  • Fix #164 - Use Ant Design instead of Semantic UI
  • Fix #165 - Search results are shown without a date
  • Fix #69 - Missing endpoints in jaeger ui dropdown

Fixes

• Fix 2 digit lookback (12h, 24h) parsing (jaegertracing/jaeger-ui#167)