New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
need some clarification on what this plugin do exactly #97
Comments
You can confirm the behavior like the following:
Run command
The event that is not matched any rules will be consumed by this plugin.
Please read example carefully. |
so does that mean that the event gets stuck circulating in the pipeline and i should implement something like a garbage collector like this :
will this do the trick of getting rid of those events ? |
@okkez im still waiting for a reply please ? |
@SmittyCooger |
Would be really nice to have a rule (or just rule without a key that will automatically match everything. Using a wildcard pattern ( |
It's insane that this behaviour (dropping unmatched records) is not documented... It's quite reasonable based on the behaviour of other "rewrite" plugins to expect records that don't match a rule to be passed through unchanged. |
Also a log can be matched by multiple rules which results in multiple versions of it getting emitted. We were using config like:
which didn't work as intended because subsequent rules are still evaluated against after the While this behavior is implied by documentation, it's not really clear enough to people not understanding how fluentd works internally. |
@mlasevich Just came by accidentaly looking for something else, but I belive you can somehow utilize the "invert" option and some impossible pattern to rewrite the tag of all to this point unmatched events. Which is not nice, but I think it would work. |
I am not sure any match, invert or not, would match if key is missing. I solved this problem by using a key i guarantee exists. Ugly, but you do what you have to |
It's wildly annoying that the documentation doesn't clarify that the plugin drops unmatched event into abyss. I was stuck and frustrated for two days while writing a complex Fluentd configuration and not getting the desired output. Finally, I figured out that for unmatched events, they are simply dropped. This behavior should be different. The default should be to pass through the unmatched event instead of dropping it. We could have a parameter at each rule level or at the plugin level to indicate whether the unmatched event should be dropped, with the default value set to false. |
Please use |
@y-ken I think the pattern is not the issue, the issue is that I believe it will not even try that pattern if the key is not there at all |
I commented below on what I think needs clarification in the specifications. |
so from what i understood from the Overview this plugin re-emits events that matches a pattern and let the unmatched events keep throught the rest of the configuration, but when i try something like this :
the events with a
kalp
string in it get routed with the new tag, pass therewrite_tag_filter
and get catched bystdout
plugin, And the events that do not containkalp
will be re-emitted along with the matched events having the same tag and get stuck atrewrite_tag_filter
.so is this how it should work (and if yes, then does it drop the event or re-emit it ?) or is this a bug ? (Thank you)
The text was updated successfully, but these errors were encountered: