[META] How to support breaking changes prior to 2.0.0

[black-adder]

We have a couple of "breaking" changes in the pipeline #793 and #776

The former requires a c* schema update and the latter requires users to setup their metrics anew.

I feel hesitant landing them in master in case we have bug fixes that we want to release. What do people think about having a branch for 2.0.0 and doing the unenviable job of keeping master and 2.0.0 in sync?

@jpkrohling @pavolloffay @objectiser @yurishkuro @vprithvi @tiffon @isaachier

[objectiser]

Why not do a final release, e.g. 1.4.2 or 1.5.0 whichever is more appropriate, and then all further work on master will be for 2.0.0.

If there are any bug fixes required to the 1.x version, we create a 1.x branch from the last release and only apply the fixes.

[yurishkuro]

My personal view is that these changes are not that significant to warrant a major rev.

[jpkrohling]

I agree with @yurishkuro about the metric names change, but not sure about the other one. Would using a newer code with an older schema cause problems?

[vprithvi]

I agree with @yurishkuro as well.

The dependencies schema change isn't backward compatible, and users are expected to run a migration script to move to the new schema. They will also need to use a newer version of the spark job to write dependencies. As such, these are internal details of jaeger, and I don't think it warrants a major rev.

[benley]

Regarding the metrics naming changes, here is a partial set of prometheus metrics relabel rules to transform 1.3 / 1.4 metrics into the new naming scheme:

- action: replace
  regex: jaeger_collector:(?:.+:)?[^:]+_by_svc:([^:]+)
  replacement: $1
  source_labels: ["__name__"]
  target_label: service
- action: replace
  regex: (jaeger_collector:(?:.+:)?[^:]+_by_svc):[^:]+
  replacement: $1
  source_labels: ["__name__"]
  target_label: __name__
- action: replace
  regex: jaeger_collector:([^:]+):((?:debug_)?(?:spans|traces)(?:_recd|_rejected|_saved)?(?:_by_svc)?)
  replacement: $1
  source_labels: ["__name__"]
  target_label: format
- action: replace
  regex: jaeger_collector:([^:]+):((?:debug_)?(?:spans|traces)(?:_recd|_rejected|_saved)?(?:_by_svc)?)
  replacement: jaeger_collector:$2
  source_labels: ["__name__"]
  target_label: __name__
- regex: jaeger_collector:([^:]+)
  replacement: jaeger_collector_$1
  source_labels: ["__name__"]
  target_label: __name__

This doesn't touch the :cassandra:.* metrics, just the core jaeger ones. I haven't touched jaeger-agent or jaeger-query yet, but those are equally doable if anyone would find it useful.

[jpkrohling]

This is awesome, @benley! One would just need to swap the source/target in order to get the future 1.5 to report under the same name as 1.4, right?

[benley]

More or less, yeah. You would need to adjust the rules to match the new labels and add them back into the metric names, and tweak the regexes to match underscores and replace them with colons again. It's definitely doable.