Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

malformed documents log too much #4461

Closed
fulmicoton opened this issue Jan 26, 2024 · 2 comments · Fixed by #4483
Closed

malformed documents log too much #4461

fulmicoton opened this issue Jan 26, 2024 · 2 comments · Fixed by #4483
Assignees
Labels
bug Something isn't working

Comments

@fulmicoton
Copy link
Contributor

fulmicoton commented Jan 26, 2024

Every single malformed doc logs a warning in the doc processor. We need a way to make sure we log at most N docs per minutes for this.

This ended up causing #4460.

This can be tested easily with ./simian "abu(rate=20mb)" --endpoint localhost:7280

The solution needs to be "frugal".

@fulmicoton
Copy link
Contributor Author

We have another risky call site. When the WAL is full we log a warning every try sends an ingest request.

@fulmicoton fulmicoton assigned trinity-1686a and unassigned PSeitz Jan 29, 2024
@fulmicoton
Copy link
Contributor Author

@trinity-1686a can you take of the PR. You can either use vector's solution, cook your own, or use a macro with a static as you suggested.

I kind of like the idea of the macro.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants