Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Roadmap updates per meeting this week #498

Closed
wants to merge 5 commits into from
Closed

Roadmap updates per meeting this week #498

wants to merge 5 commits into from

Conversation

jkowall
Copy link
Collaborator

@jkowall jkowall commented Apr 23, 2021

This is an attempt to update and re-organize the roadmap. I am sure things are not where the maintainers would like them, so please comment and I will adjust accordingly.

Which problem is this PR solving?

Roadmap Updates

Short description of the changes

Re-organized existing roadmap, and added several items which are in active development today. Added more to the wish list and longer term goals.

This is an attempt to update and re-organize the roadmap. I am sure things are not where the maintainers would like them, so please comment and I will adjust accordingly.

## Which problem is this PR solving?
Roadmap Updates

## Short description of the changes
Re-organized existing roadmap, and added several items which are in active development today. Added more to the wish list and longer term goals.

Signed-off-by: Jonah Kowall <jkowall@kowall.net>
@jkowall
Copy link
Collaborator Author

jkowall commented Apr 23, 2021

Not sure what I did wrong on DCO, so please advise too :)

@albertteoh
Copy link
Contributor

Not sure what I did wrong on DCO, so please advise too :)

The commands from the "Details" link should resolve this :) https://github.com/jaegertracing/documentation/pull/498/checks?check_run_id=2423531679

jkowall and others added 2 commits April 23, 2021 18:23
This is an attempt to update and re-organize the roadmap. I am sure things are not where the maintainers would like them, so please comment and I will adjust accordingly.

## Which problem is this PR solving?
Roadmap Updates

## Short description of the changes
Re-organized existing roadmap, and added several items which are in active development today. Added more to the wish list and longer term goals.

Signed-off-by: Jonah Kowall <jkowall@kowall.net>
Signed-off-by: Jonah Kowall <jkowall@gmail.com>

[OpenTelemetry collector](https://opentelemetry.io/docs/collector/getting-started/) is a vendor-agnostic service for receiving, processing and exporting telemetry data. We have decided to rebuild the Jaeger backed components (agent, collector, ingester, all-in-one) on top of OpenTelemetry collector which has several benefits:
Aggregated trace metrics can be exported by the OpenTelemetry Collector [/opentelemetry-collector-contrib/tree/main/processor/spanmetricsprocessor](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/processor/spanmetricsprocessor) this allows metrics to be calculated off trace data and metrics to be exported from the Collector. We are adapting Jaeger to be able to read metrics from a Prometheus compatible backend, but additional backends may be supported. For additional information see [jaeger/pull/2946](https://github.com/jaegertracing/jaeger/pull/2946).
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Aggregated trace metrics can be exported by the OpenTelemetry Collector [/opentelemetry-collector-contrib/tree/main/processor/spanmetricsprocessor](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/processor/spanmetricsprocessor) this allows metrics to be calculated off trace data and metrics to be exported from the Collector. We are adapting Jaeger to be able to read metrics from a Prometheus compatible backend, but additional backends may be supported. For additional information see [jaeger/pull/2946](https://github.com/jaegertracing/jaeger/pull/2946).
Aggregated trace metrics can be exported from the OpenTelemetry Collector by [spanmetricsprocessor](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/processor/spanmetricsprocessor) that allows metrics to be calculated from trace data. We are adapting Jaeger to be able to read metrics from a Prometheus compatible backend, but additional backends may be supported. For additional information see [jaeger/pull/2946](https://github.com/jaegertracing/jaeger/pull/2946).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's better to link to an issue rather than pull request: jaegertracing/jaeger-ui#736

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That issue is for the UI, not the Otel processor. I linked to jaegertracing/jaeger#2954 which is the master issue for the UI change.


* Multi-Tenancy ([mailgroup thread](https://groups.google.com/forum/#!topic/jaeger-tracing/PcxftflO4_o))
* Cloud and Multi-DC strategy
* Anomaly detection of flagging of traces
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

flagging?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will show you a POC of the flagging in a forthcoming community meeting. We did a POC of this already. Just haven't discussed it yet :)


* Multi-Tenancy ([mailgroup thread](https://groups.google.com/forum/#!topic/jaeger-tracing/PcxftflO4_o))
* Cloud and Multi-DC strategy
* Anomaly detection of flagging of traces
* Baselining of trace metrics
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

elaborate?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Haven't discussed this one yet with you, but the idea would be to provide a proactive way to identify problems happening outside of a normal pattern. Building some basic baseline of the metrics and showing deviations. This is something Prometheus doesn't do today. It could belong as a Prometheus feature, but it depends how it's used inside Jaeger.

content/roadmap.md Outdated Show resolved Hide resolved

* Multi-Tenancy ([mailgroup thread](https://groups.google.com/forum/#!topic/jaeger-tracing/PcxftflO4_o))
* Cloud and Multi-DC strategy
* Anomaly detection of flagging of traces
* Baselining of trace metrics
* Alerting
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmmm... What kind of alerting (considering that you already have anomaly detection above)?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Notifications, email/slack or maybe integrate with Alertmanager? I have some other ideas here potentially which I will bring up in a community call leveraging some of the OpenSearch capabilities which are plugins: https://opendistro.github.io/for-elasticsearch-docs/docs/alerting/ which might allow us to integrate. Of course these are all ideas we need to discuss.

jkowall and others added 2 commits April 24, 2021 12:39
Co-authored-by: Yuri Shkuro <yurishkuro@users.noreply.github.com>
Co-authored-by: Yuri Shkuro <yurishkuro@users.noreply.github.com>
@jkowall
Copy link
Collaborator Author

jkowall commented Apr 24, 2021

Botched this commit, so closing and replacing with #499

@jkowall jkowall closed this Apr 24, 2021
yurishkuro added a commit that referenced this pull request Apr 27, 2021
* Updates per discussion in #498 clarified several points

Signed-off-by: Jonah Kowall <jkowall@kowall.net>

* Update content/roadmap.md

Co-authored-by: Yuri Shkuro <yurishkuro@users.noreply.github.com>
Signed-off-by: Jonah Kowall <jkowall@kowall.net>

Co-authored-by: Yuri Shkuro <yurishkuro@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants