Skip to content

Add fuzzy query for log content.#47

Merged
wu-sheng merged 4 commits intomasterfrom
wu-sheng-patch-1
Jan 8, 2021
Merged

Add fuzzy query for log content.#47
wu-sheng merged 4 commits intomasterfrom
wu-sheng-patch-1

Conversation

@wu-sheng
Copy link
Member

@wu-sheng wu-sheng commented Jan 8, 2021

Log content fuzzy query is a risky operation. What do you think about this?

@EvanLjp
Copy link
Member

EvanLjp commented Jan 8, 2021

Log queries are much more complex than other existing queries in SkyWalking now, such as more data size and need custom Tokenizer in es. We need to be careful with this.

@wu-sheng
Copy link
Member Author

wu-sheng commented Jan 8, 2021

That is another story about the stop word, word analyzer, etc. But none of them relate to the query abstract implementation and protocol.
Let's discuss one thing at a time.

@kezhenxu94
Copy link
Member

I'm big +1 to have this functionality, the performance concern will be discussed in specific implementations though.

@dmsolr
Copy link
Member

dmsolr commented Jan 8, 2021

I'm +1 at first of all.
I am thinking about the implementation of storage plugins. It seems that not very friendly except ES.

@wu-sheng
Copy link
Member Author

wu-sheng commented Jan 8, 2021

I am thinking about the implementation of storage plugins. It seems that not very friendly except ES.

@dmsolr Yes, I am thinking about this too. How about we add a response text when the implementation doesn't support that, they could add description for the UI.

Copy link
Member

@rainbend rainbend left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is a very useful functionality.

}

extend type Query {
supportQueryLogsByKeywords: Boolean!
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@arugal @kezhenxu94 @EvanLjp @dmsolr What about this? And I put keywords as List.

@EvanLjp
Copy link
Member

EvanLjp commented Jan 8, 2021

Maybe need to be enhanced the feature. We need not only grep but grep -v in the original log query. excludeKeywordsOfContent may need to be added.

# Fuzzy query conditions for the log content.
# Use these 2 keyword related condition, when supportQueryLogsByKeywords = TRUE.
keywordsOfContent: [!String]
excludingKeywordsOfContent: [!String]
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@EvanLjp You ask for this, right?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes

@wu-sheng wu-sheng merged commit 4ac87d5 into master Jan 8, 2021
@wu-sheng wu-sheng deleted the wu-sheng-patch-1 branch January 8, 2021 12:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants