Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

An ability to tell otlpreceiver that some logs were dropped and return a partialSuccess #7669

Open
zhkvia opened this issue May 10, 2023 · 0 comments

Comments

@zhkvia
Copy link

zhkvia commented May 10, 2023

Is your feature request related to a problem? Please describe.
We use the collector along with otlpreceiver to receive logs and we would also like to validate incoming data with our own rules (e.g. looking for special values in Resource().Attributes()). Depending on the values, some logs can be processed or dropped. It seems that the protocol supports a partialSuccess response with a number of processed and dropped spans / logs. However, if we write a processor with our validation logic, the only way to tell otlpreceiver (and a user) that some logs were dropped is to return an error with a string description. In that case, otlpreceiver does not return a partialSuccess. Instead it returns an error with the http.StatusInternalServerError code.

Is there a way to tell otlpreceiver that some logs were dropped but that it still needs to return a partialSuccess?

Describe the solution you'd like
I see two options:

  1. Extend the protocol so Logs.ConsumeLogs returns not only an error, but also information about processed and dropped logs with a description of a rejection. In that case, we will continue to use our processor which is executed after a receiver, and a user would get a partialSuccess with an error description if some logs were dropped, but the processing of valid logs would continue
  2. Support receiver extensions writing which developers could add their own validation

Describe alternatives you've considered

  1. Forking otlpreceiver with adding our custom validation which is probably not (or hardly) maintainable in the long-term
  2. Writing our own receiver that should comply with the changing OTLP format rules
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant