-
Notifications
You must be signed in to change notification settings - Fork 131
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
About parameter full_search_strategy
in drain match method
#68
Comments
What you suggest makes sense to me. However I checked also the original Drain code here: https://github.com/logpai/logparser/blob/e8d96cd4de1121c5d2b517982c6028cd06e643f1/logparser/Drain/Drain.py#L172 and the same max-param-count selection logic exists, so it's not a translation bug. I also read the part of the Drain paper https://jiemingzhu.github.io/pub/pjhe_icws2017.pdf on page 4 (step 4), but unfortunately, the text does not describe what happens in a case of equal similarity score. |
Actually while writing an issue to the Drain team, I think I understand the reason. Follow this example: For the two templates:
and log message:
Equal tokens to the 1st template = 4, param count in first message = 4 Here the message has a bettter match with 1st template. |
Also, something in your first example does not make sense. |
while enbale include_param=True,the similarity 1st template is (4+4)/ 8 = 1.0, the sim of 2st template is (4+2) / 8 = 0.75, it will not go to the logical comparsion |
for this case |
But the template has 9 tokens, not 8 ? |
what I want to talk about is when two template has same similarity, the one with less wildcards will be match. But in this case, when we enable include_param=True, the similariteis of 1st and 2nd template is not equal, it will not go to the logical |
include_param=True by default in |
I know, So that in my cases,
two template similarity is 1.0, we should choose 1st template which has no wildcard, but drain it return the 2st template.
|
This is an example where more params is the better choice. |
Got it, but it didn't work on my case. Is it something wrong in its logic? In other words, I think when two template both get 1.0 similarity score, the template has the less wildcard will be match, isn't it? But drain doesn't work on this case. |
If you want, you can provide a sample code that reproduces the problem in a branch, and I will check. |
As the comments said
(3) "always" is the slowest. It will select the best match among all known clusters, by always evaluating all clusters with the same token count, and selecting the cluster with perfect all token match and least count of wildcard matches.
I have two clusters as below, one has wildcard and another not
And I have a log to be matched
IPPROTO_TCP fd: 100, errno: 100, option: 100, value: 100
After masked it become
IPPROTO_TCP fd <NUM> errno <NUM> option <NUM> value <NUM>
And I use "always" strategy, as the comments said, I should get the cluster(id=1) which has least wildcard, but I get result
ID=2 : size=1 : IPPROTO_TCP fd <NUM> errno <NUM> option <NUM> value <*>
So I read the code in fast_match, and I found this code seg will always return the cluster which has most param_count, is it wrong?
Should I modified it like this
The text was updated successfully, but these errors were encountered: