[Rule Tuning] Tune Threat Indicator Match Rules

[w0rk3r]

Issues

Resolves #2954

Summary

• Threat Intel URL Indicator Match:
  • Drops the domain fields on this rule, as it is more noisy than expected, as a lot of commonly used web services are included in threat intel fields. Such as Github, dropbox, discord, google drive, etc.
• Threat Intel Hash Indicator Match
  • Drops the support to dll.pe.imphash, as integrations such as abusech contain events with this field empty, which can match the target field in specific conditions in which it is not populated by Elastic Endpoint, generating thousands of unintended matches:

Some other approaches considered:

I've looked at using QueryDSL as a filter in the Indicator index query but excluding indicators that don't populate certain fields will cause the other fields which contain the information to be excluded, causing FNs.

Other potential solutions to the problem:

As this matches a specific condition on the endpoint, we could add a filter that excludes the events which cause this, here is a working query that we could ship as a filter:

Query

{
  "bool": {
    "must": [],
    "filter": [],
    "should": [],
    "must_not": [
      {
        "match_phrase": {
          "dll.hash.md5": ""
        }
      },
      {
        "match_phrase": {
          "dll.hash.sha1": ""
        }
      },
      {
        "match_phrase": {
          "dll.hash.sha256": ""
        }
      },
      {
        "match_phrase": {
          "dll.pe.imphash": ""
        }
      }
    ]
  }
}

[MakoWish]

Could we consider changing the URL Indicator rule to use both url.full and url.original? I have found some events that include one without the other, and other events with the opposite. I did a PATCH on our copy of the rule to do exactly that, and it is working well so far.

[Aegrah]

LGTM

[DefSecSentinel]

LGTM