Skip to content

feat(codecs): introduce OTLP decoder#24003

Merged
pront merged 14 commits into
masterfrom
pront-otlp-decoder
Oct 16, 2025
Merged

feat(codecs): introduce OTLP decoder#24003
pront merged 14 commits into
masterfrom
pront-otlp-decoder

Conversation

@pront
Copy link
Copy Markdown
Member

@pront pront commented Oct 14, 2025

Summary

  • Introduces otlp decoder.
  • Uses it for the opentelemetry source

Implemented with flexibility in mind (iterative approach).
Follow-up PR will improve performance before we release this feature.

Vector configuration

For manual tests I used the following config:

api:
  enabled: true

sources:
  source0:
    type: opentelemetry
    grpc:
      address: 0.0.0.0:4317
    http:
      address: 0.0.0.0:4318
    use_otlp_decoding: true
sinks:
  console:
    type: console
    inputs:
      - source0.logs
      - source0.metrics
      - source0.traces
    encoding:
      codec: json
      pretty: true

How did you test this PR?

Unit tests and also used by existing OTEL E2E tests.

Manually with:

telemetrygen traces \
  --otlp-endpoint=localhost:4318 \
  --otlp-http \
  --otlp-insecure \
  --traces 1 \
  --service="telemetrygen-test"

Change Type

  • Bug fix
  • New feature
  • Non-functional (chore, refactoring, docs)
  • Performance

Is this a breaking change?

  • Yes
  • No

Does this PR include user facing changes?

  • Yes. Please add a changelog fragment based on our guidelines.
  • No. A maintainer will apply the no-changelog label to this PR.

References

closes: #23975

Notes

  • Please read our Vector contributor resources.
  • Do not hesitate to use @vectordotdev/vector to reach out to us regarding this PR.
  • Some CI checks run only after we manually approve them.
    • We recommend adding a pre-push hook, please see this template.
    • Alternatively, we recommend running the following locally before pushing to the remote branch:
      • make fmt
      • make check-clippy (if there are failures it's possible some of them can be fixed with make clippy-fix)
      • make test
  • After a review is requested, please avoid force pushes to help us review incrementally.
    • Feel free to push as many commits as you want. They will be squashed into one before merging.
    • For example, you can run git merge origin master and git push.
  • If this PR introduces changes Vector dependencies (modifies Cargo.lock), please
    run make build-licenses to regenerate the license inventory and commit the changes (if any). More details here.

@github-actions github-actions Bot added the domain: sources Anything related to the Vector's sources label Oct 14, 2025
@github-actions github-actions Bot added the domain: external docs Anything related to Vector's external, public documentation label Oct 14, 2025
@pront pront force-pushed the pront-otlp-decoder branch from 7ee0e73 to a0eb288 Compare October 14, 2025 22:10
@pront pront marked this pull request as ready for review October 15, 2025 14:00
@pront pront requested review from a team as code owners October 15, 2025 14:00
@drichards-87 drichards-87 self-assigned this Oct 15, 2025
@drichards-87 drichards-87 removed their assignment Oct 15, 2025
Copy link
Copy Markdown
Member

@thomasqueirozb thomasqueirozb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As discussed offline OtlpDeserializer could use some improvements to not blindly try all 3 deserializers but the current implementation should work just fine. It's a nice user facing improvement

@pront pront enabled auto-merge October 16, 2025 14:33
@pront pront added this pull request to the merge queue Oct 16, 2025
Merged via the queue into master with commit 73c468c Oct 16, 2025
46 checks passed
@pront pront deleted the pront-otlp-decoder branch October 16, 2025 15:30
titaneric pushed a commit to titaneric/vector that referenced this pull request Oct 21, 2025
* feat(codecs): introduce OTLP decoder and use it internally for the OTEL source

* make fmt

* add changelog

* generate docs

* clippy fix

* make fmt

* code golfing and tests

* fix schema definition

* make fmt

* remove unusused import

* improve docs

* improve test: validate trace ID and spand ID

* fmt linux
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

domain: external docs Anything related to Vector's external, public documentation domain: sources Anything related to the Vector's sources

Projects

None yet

Development

Successfully merging this pull request may close these issues.

invalid wire type: LengthDelimited (expected SixtyFourBit) use_otlp_decoding: true

3 participants