Proposal: Pathway to component stability

[MovieStoreGuy]

Context

In the most recent NA-APAC meeting, it was discussed on "what does stability look like?".
Which is an interesting question when you start to consider:

• Can a component be considered stable when that components dependancies are not considered stable?
• How do we know when a component should be considered stable?
• How do we ensure the stability of the component?

Proposed plan

While core maintainer and approvers work towards core component stability, the idea I would like to forward is a template for component stability.

As we currently ask for new components to provide an outline of:

• expected configuration
• supported pipelines
• if it is a vendor owned or community owned component

I would like to extend the definition to include:

• What does stability look like for the component
  • Breakdown of features (potentially grouped by alpha, beta, stable)
  • What feature sets are in scope or out of scope for the component
• What are the core features for the component?
  • Can be as simple as writing to a vendor's solution
  • Should include clear tests that ensure that the core features are working
    • Could be prefixed with TestCoreFeature... or some other means of grouping.
    • Failing to meet the core feature functionality after X would be grounds to move component to unmaintained.

I assume there is other parts to this but I think if we encourage some sort of plan on component owners, we can all reach stability at the same time. I am keen to hear all feedback on this.

[atoulme]

I have been thinking that we could cement stability levels with different test requirements. Those tests should be reusable across components so it's as simple as adding them. The cmd/otelcontribcol location is where we can add those sanity tests. Right now, we test for the lifecycle of the components for example.

We also have moved to use a metadata.yaml file for each component to fill in the component's status. I think we should use this approach to mandate that the component be added to cmd/otelcontribcol if it is declared to be of alpha or beta status.

This gives us a broad set of possibilities as to what we want to test to start with, so we can enforce whatever guidelines we come up with here as code and CI checks.

[jpkrohling]

What's the current state of this?

[atoulme]

We have finished automating with metadata.yaml such as that we can for contrib components at least now operate programmatic checks against a component based of this metadata. We have used it to check presence in cmd/otelcontribcol at this time.

We now have the means to enforce requirements per stability level, but not yet a good sense of the exhaustive requirements we want to apply.

[atoulme]

There are now some practical requirements for a component to be considered stable, see #9045 for an example. Here are the stability requirements I see:

• No open issues or PRs in the module that would require breaking changes
• No TODOs in the module code that would require breaking changes
• No deprecated symbols in the module
• No symbols marked as experimental in the module
• The module follows the Coding guidelines
• No unstable dependencies

Please also make sure to publicly announce our intent to stabilize the module on:

• The #otel-collector CNCF Slack Channel
• The #opentelemetry CNCF Slack channel
• A Collector SIG meeting (if unable to attend, just add to the agenda)