Update Analytics Performance #3866
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
patel-bhavin
approved these changes
Jan 20, 2026
| | `drop_dm_object_name("All_Traffic")` | ||
| | `security_content_ctime(firstTime)` | ||
| | `security_content_ctime(lastTime)` | ||
| | lookup remote_access_software remote_appid AS app OUTPUT isutility, description as signature, comment_reference as desc, category |
Contributor
There was a problem hiding this comment.
Should we move these lookup by the where clause like other detection updates? eg : Windows DotNet Binary in Non Standard Path
Contributor
Author
There was a problem hiding this comment.
We can't just because the app field is inconsistent in that CIM it can contain both apps categories and binaries, but more importantly the lookup is big with 600 entries, which will results in a big OR clause that results in the same performance.
Contributor
|
Lets wait on merging this after the release is out! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR enhances the performance of a couple of detections as part of a continued effort that I was tracking from the issue #3809. Details are below.
Updated Analytics [11]
Common Ransomware Notes- From 164 seconds to 33.44 seconds - Switch to using a subsearch since the lookup is small enough. Added a warning in the description to inform the user about this too.File with Samsam Extension- From 160 seconds to 11 seconds - Moved the extension search to thewhereand kept the extraction for better output.Detect Rare Executables- From 300~ seconds to 100~ seconds - Reduces usage ofvaluesand switched them tolatestso that we can reduce MV calcs which are expensive.3CX Supply Chain Attack Network Indicators- Added a where clause withDNS.query=*to ensure the existence of the field, since it is used as a filter in the lookup. This is a best effort and will only reduce search time by some time. The same thing was applied to the followingDetect hosts connecting to dynamic domain providers,Detect Remote Access Software Usage DNSDetect Remote Access Software Usage URL- Added a where clause withWeb.url_domain=*to ensure the existence of the field, since it is used as a filter in the lookup.Detect Remote Access Software Usage Traffic- Added a where clause withAll_Traffic.app=*to ensure the existence of the field, since it is used as a filter in the lookup.Common Ransomware Extensions- To enhance this we added filter for known extensions to reduce the search space. By excluding stuff like.exe...etc from file creation we already get rid of a lot. This can ofc be further enhanced by adding more to the list.Detect Remote Access Software Usage File- In order to also enhance perf, we allow only the extensions that are referenced by the lookup itself. Because this is using an file creation event, this will reduce the search space by a lot.Windows DotNet Binary in Non Standard Path- From 852 seconds to 7 seconds - Using a subsearch to enhance performance with the accepted limitation just because the lookup is small.