otel: add support for per tenant configuration for mapping otlp data to loki format #11143
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
sandeepsukhani
force-pushed
the
per-tenant-otlp-config
branch
from
930ad2d
to
3b0b374
Compare
slim-bean
reviewed
Jan 19, 2024
Comment on lines
+61
to
+63
| type Limits interface { | ||
| OTLPConfig(userID string) OTLPConfig | ||
| } |
There was a problem hiding this comment.
Not sure why a new interface here is helpful? Can't we use the existing overrides?
There was a problem hiding this comment.
It makes it easier to write tests, and we can't reference overrides since that would cause circular dependency.
|
This looks good to me @sandeepsukhani! |
# Conflicts: # pkg/validation/limits.go
github-actions
bot
added
the
type/docs
Gordejj
pushed a commit
to Gordejj/loki
that referenced
this pull request
Jan 29, 2024
…to loki format (grafana#11143) **What this PR does / why we need it**: In OTEL, we pick select Resource Attributes to identify the streams and store all the other attributes as Structured Metadata, as explained here. The problem however is that the list of Resource Attributes that are picked as Stream labels are hardcoded, and there is no way to drop unwanted data. This PR adds support for configuring how data is mapped from OTEL to Loki format per tenant. It also adds support for dropping unwanted data. We decided to make the config look similar to Prometheus's relabling config to make it familiar. **Special notes for your reviewer**: Opening a draft PR to get some initial feedback. I will add documentation in a separate PR. **Checklist** - [x] Tests updated - [x] `CHANGELOG.md` updated
1 task
sandeepsukhani
added a commit
that referenced
this pull request
Feb 5, 2024
**What this PR does / why we need it**: In PR #11143 we added support for per tenant otlp config. This PR adds the relevant documentation to explain how the config looks and works. **Checklist** - [x] Documentation added --------- Co-authored-by: J Stickler <julie.stickler@grafana.com>
rhnasc
pushed a commit
to inloco/loki
that referenced
this pull request
Apr 12, 2024
…to loki format (grafana#11143) **What this PR does / why we need it**: In OTEL, we pick select Resource Attributes to identify the streams and store all the other attributes as Structured Metadata, as explained here. The problem however is that the list of Resource Attributes that are picked as Stream labels are hardcoded, and there is no way to drop unwanted data. This PR adds support for configuring how data is mapped from OTEL to Loki format per tenant. It also adds support for dropping unwanted data. We decided to make the config look similar to Prometheus's relabling config to make it familiar. **Special notes for your reviewer**: Opening a draft PR to get some initial feedback. I will add documentation in a separate PR. **Checklist** - [x] Tests updated - [x] `CHANGELOG.md` updated
rhnasc
pushed a commit
to inloco/loki
that referenced
this pull request
Apr 12, 2024
**What this PR does / why we need it**: In PR grafana#11143 we added support for per tenant otlp config. This PR adds the relevant documentation to explain how the config looks and works. **Checklist** - [x] Documentation added --------- Co-authored-by: J Stickler <julie.stickler@grafana.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What this PR does / why we need it:
In OTEL, we pick select Resource Attributes to identify the streams and store all the other attributes as Structured Metadata, as explained here. The problem however is that the list of Resource Attributes that are picked as Stream labels are hardcoded, and there is no way to drop unwanted data. This PR adds support for configuring how data is mapped from OTEL to Loki format per tenant. It also adds support for dropping unwanted data. We decided to make the config look similar to Prometheus's relabling config to make it familiar.
Special notes for your reviewer:
Opening a draft PR to get some initial feedback. I will add documentation in a separate PR.
Checklist
CHANGELOG.mdupdated