Skip to content

Remove pre-release OpenTelemetry pins from generated constraints #71176

Description

@potiuk

Our generated constraints currently pin two packages to pre-release (beta) versions:

opentelemetry-exporter-prometheus==0.65b0
opentelemetry-semantic-conventions==0.65b0

These are the only pre-releases in the constraints files we tag and ship, and both require a hardcoded exception in scripts/in_container/run_generate_constraints.py (additional_constraints_for_highest_resolution).

Why the exception exists

Constraints cut for a release candidate resolve with --prerelease explicit, which permits a pre-release only for packages some requirement explicitly marks as such, and drops the if-necessary fallback. Neither OpenTelemetry package has ever published a final release - every version on PyPI is 0.NNbM (61 releases for opentelemetry-semantic-conventions, 55 for opentelemetry-exporter-prometheus, zero non-pre-release among them). Without an explicit pre-release lower bound in the resolution, they cannot resolve at all and constraints generation fails with No solution found.

Where they come from

  • opentelemetry-exporter-prometheus - direct dependency of the otel extra of apache-airflow-core (which is part of the all extra), >=0.47b0.
  • opentelemetry-semantic-conventions - hard transitive dependency of opentelemetry-sdk, which comes in via opentelemetry-exporter-otlp (a core apache-airflow-core dependency) and directly via shared/observability and shared/listeners.

So neither pin can simply be dropped today: semantic-conventions is unavoidable as long as we depend on the OpenTelemetry SDK at all.

What needs to happen

Any of the following would let us delete the exception:

  1. Upstream stabilisation - OpenTelemetry Python graduates these components to 1.x finals. This is the real fix and is outside our control; it needs tracking against open-telemetry/opentelemetry-python.
  2. Drop opentelemetry-exporter-prometheus from the otel extra and document it as a user-installed optional dependency - removes one of the two pins, but not semantic-conventions.
  3. Reconsider whether shipping the OTel SDK in core dependencies is required, since it is what drags semantic-conventions in.

Option 1 is the only one that removes both pins without a functional trade-off, so this issue mainly serves as the tracking anchor referenced from the code, to be revisited whenever the OTel Python packages change their versioning scheme.

Acceptance criteria

  • No pre-release version appears in tagged constraints files.
  • additional_constraints_for_highest_resolution in scripts/in_container/run_generate_constraints.py no longer needs entries for either OpenTelemetry package, and constraints generation still succeeds with --allow-pre-releases (the RC path).

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:dependenciesIssues related to dependencies problemsarea:dev-envCI, pre-commit, pylint and other changes that do not change the behavior of the final codeupstream-dependency

    Type

    No type

    Projects

    Status
    Dependencies

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions