Incorporate an OTel-wide maturity matrix #290
Conversation
Per the header comment for the file under review here: OpenTelemetry is a complex project with many moving parts. Thankfully, we strive to keep those parts loosely coupled where we can (this has been a goal since the very announcement of OpenTelemetry, per this blog post: https://medium.com/opentracing/merging-opentracing-and-opencensus-f0fe9c7ca6f0) This file describes the structure of the project and its sub-components, definitions of API and stability maturity levels, and finally records the current maturity of each component. We maintain this file as YAML in order to make it easier to automate scripts (e.g., for the website) that present the data to OpenTelemetry end-users.
bhs
requested review from
bogdandrutu,
ccaraman,
lizthegrey,
mtwo,
sarahnovotny,
SergeyKanzhelev,
tedsuo and
yurishkuro
as code owners
add golang status, add additional states.
|
Fixed some typoes, updated golang status, added "unimplemented" as a state in addition to "alpha". we currently do not release major versions on breaking changes while we're in alpha for golang, as a FYI. our goal is to release major/minor versions corresponding to spec major/minor versions rather than incrementing in unpredictable ways that slew from the spec major/minor versions. |
|
We’ve been trying to track this via github milestones, and automatically scrape that data to update the status page on the website. Would be great if we could continue to have an automated process for keeping the website up to date. |
|
@tedsuo @austinlparker yes, agree – I like the idea of using repo metadata (milestones, tags, whatever) to track status. And also agree about the website. My goal with this PR is primarily to debate "the schema" for what we're telling end-users about maturity. If we figure that out, I am personally flexible about the mechanics as long as they allow for automation and don't require some crazy amount of upfront work. |
|
Once we've got the schema sorted out then it should be pretty easy to add some tags or whatever to auto-populate the website. |
|
Agree, the main change here is moving off of one-dimensional v0.1->v0.4 versioning to tagging each maturity stage per category and project. |
maturity-matrix.yaml
Outdated
| - traceSDK | ||
| - metricsSDK | ||
| - contextSDK | ||
| - automaticInstrumentation |
There was a problem hiding this comment.
should the integrations and exporters be also tracked? Zipking and Jaeger exporters, integrations like SpringBoot, etc.?
There was a problem hiding this comment.
IMO we should focus solely on SDK/API for trace/metric/context here and allow exporters and integrations to simply reference their stability or maturity against the lower level components.
There was a problem hiding this comment.
some integrations and exporters are the baseline for the stability as you cannot use API/SDK without ability to export it. Similar for integrations. I'm not sure how automaticInstrumentation here is different than integrations?
There was a problem hiding this comment.
Hm, that's a good point, although I don't know how to square it. Automatic instrumentation does seem to require integrations.
There was a problem hiding this comment.
+1, I think that end-users expect at least some of these (OT exporter, Jaeger exporter, Prometheus exporter, basic HTTP integrations) to be tracked
There was a problem hiding this comment.
I do not want to suggest that we try to maintain a full compatibility matrix with framework-level things like Spring. The "registry" seems like the right place to track that sort of thing.
By "automaticInstrumentation", I meant specifically the idea of "no-code" (literally) installation of the OpenTelemetry instrumentation. This is an important requirement for many brownfield services that are no longer actively maintained but nevertheless participate in transactions. As such, I would not like to extend the definition to "1-line code changes" that nevertheless require recompilation.
@SergeyKanzhelev re your comment:
some integrations and exporters are the baseline for the stability
I disagree – they may be the baseline for "usefulness", but not for stability.
Given the possibly large diversity of exporters in the future, I would personally prefer that we:
- Adopt the API and production maturity levels for exporters, and
- Track exporter maturity via the registry
Remember that we're not debating how this information is displayed on a website at the moment, we're just debating what should be considered "internal" to OpenTelemetry rather than a plugin/etc. The website can absolutely bring in the maturity levels for popular exporters in designated languages.
There was a problem hiding this comment.
We need to avoid situation when OT is only usable with the specific vendor exporter. This is why I think we should declare some baseline exporters here. It may be minimum functionality and not scalable, but must be reliable and stable.
There was a problem hiding this comment.
I wonder about the 'diversity of exporters' -- Yuri could probably weigh in here, but I suspect that many "vendors", both OSS and commercial, would like to simply accept OTel data format natively and thus not require an exporter at all.
With that in mind, this feels like a transient issue; Yes, there'll be maturity levels for exporters but over time those exporters simply won't matter since your backend will accept the data with no translation.
There was a problem hiding this comment.
I suspect that many "vendors", both OSS and commercial, would like to simply accept OTel data format natively and thus not require an exporter at all.
Not immediately. It may take a while
With that in mind, this feels like a transient issue; Yes, there'll be maturity levels for exporters but over time those exporters simply won't matter since your backend will accept the data with no translation.
Yes, but if we intent to track progress towards 1.0 I'd suggest to include tracking of this transient issue
There was a problem hiding this comment.
@SergeyKanzhelev I'm going to continue to push back on this... if I want to use Jaeger's exporter and it's the only thing that's mature for my given language, IMO that's fine. Now, whether the SIGs would feel comfortable marking their APIs as "stable" (or even "beta") with zero or one exporter implemented is a different question.
| golang: | ||
| traceAPI: | ||
| api: beta | ||
| production: beta |
There was a problem hiding this comment.
I'd suggest to include the link to the issues query on GitHub showing what's left for this area. It may be an optional attribute
There was a problem hiding this comment.
I'm a little hesitant to do this since in the early stages, that list will have lots of errors of omission and could lead to a false sense of security...
There was a problem hiding this comment.
can we declare it as optional so I can use it for .NET?
There was a problem hiding this comment.
IMO it'd be good to have a URL field in the schema that points to the repo.
|
Hi folks... I've updated the PR here in two ways:
Can reviewers either approve or request further changes? I can then fill in the gaps in other languages, potentially as follow-on PRs. cc @lizthegrey @SergeyKanzhelev @austinlparker @mtwo @tedsuo (who've all provided comments earlier – thanks). |
|
One more ping on this – still looking for an approval from an approver! |
|
@lizthegrey thanks. All: I'll merge this tomorrow or Tuesday if there are no further concerns. As mentioned on the most recent Gov Committee call, I'll also make a very crude sketch of how we could represent this on the marketing site and submit an issue to track that. (I don't think I'll have the cycles to actually implement the marketing site piece, though) (edit: here's that sketch for the otel.io website: open-telemetry/opentelemetry.io#109) |
|
Thanks, all... Happy to take PRs on this file as we bring it into sync with the various per-language SIGs that are presently left as TODOs, etc. |
Per the header comment for the file under review here:
This PR stems from the most recent technical and governance committee meetings. cc @mtwo who is doing a survey about blockers to a beta release in 5 languages for Kubecon EMEA.
All comments welcome, but especially those about the actual schema (formatting, repo location, naming, etc will be easy to change and I'm very flexible on those fronts!).