[Security Solution][Endpoint] Adds matches wildcard operator for file.path.text field for Event Filters

[ashokaditya]

Summary

Notable changes

• added matches and not match operators for file.path.text fields.
• changed artifact logic to include process.name entries for file.path.text when there's a simple filename at the end of the wildcard path
• moved path validations for TA into packages/kbn-securitysolution-utils in order to use in both Trusted Apps and Event Filters input validations.
• added relevant tests for the above changes

For creating/updating event filters, this change allows adding wildcard path values when the selected field is file.path.text via matches operator.

Screens

---

Artifact cards vs artifact entries
Windows

windows artifact entry

{
  "entries": [
    {
      "type": "simple",
      "entries": [
        {
          "field": "file.path.text",
          "operator": "included",
          "type": "wildcard_caseless",
          "value": "c:\\fol*\\sample.exe"
        },
        {
          "field": "process.name",
          "operator": "included",
          "type": "exact_caseless",
          "value": "sample.exe"
        },
        {
          "field": "file.path.text",
          "operator": "excluded",
          "type": "wildcard_caseless",
          "value": "a:\\filters\\fi*.exe"
        },
        {
          "field": "file.path.text",
          "operator": "excluded",
          "type": "wildcard_caseless",
          "value": "d:\\exac*\\this.exe"
        },
        {
          "field": "process.name",
          "operator": "excluded",
          "type": "exact_caseless",
          "value": "this.exe"
        },
        {
          "field": "file.path.text",
          "operator": "included",
          "type": "wildcard_caseless",
          "value": "c:\\dam*\\file.*"
        }
      ]
    }
  ]
}

Mac

mac artifact entry

{
  "entries": [
    {
      "type": "simple",
      "entries": [
        {
          "field": "file.path.text",
          "operator": "included",
          "type": "wildcard_caseless",
          "value": "/usr/l*/exact.app"
        },
        {
          "field": "process.name",
          "operator": "included",
          "type": "exact_caseless",
          "value": "exact.app"
        },
        {
          "field": "file.path.text",
          "operator": "included",
          "type": "wildcard_caseless",
          "value": "/opt/lib/fi*.dmg"
        },
        {
          "field": "file.path.text",
          "operator": "excluded",
          "type": "wildcard_caseless",
          "value": "/sy*/bin/readme.txt"
        },
        {
          "field": "process.name",
          "operator": "excluded",
          "type": "exact_caseless",
          "value": "readme.txt"
        },
        {
          "field": "file.path.text",
          "operator": "excluded",
          "type": "wildcard_caseless",
          "value": "/usr/lib/read*"
        }
      ]
    }
  ]
}

Linux

linux artifact entry

{
  "entries": [
    {
      "type": "simple",
      "entries": [
        {
          "field": "file.path.text",
          "operator": "included",
          "type": "wildcard_cased",
          "value": "/opt/*in/sys.dmg"
        },
        {
          "field": "process.name",
          "operator": "included",
          "type": "exact_cased",
          "value": "sys.dmg"
        },
        {
          "field": "file.path.text",
          "operator": "included",
          "type": "wildcard_cased",
          "value": " /sy*/file.*"
        },
        {
          "field": "file.path.text",
          "operator": "excluded",
          "type": "wildcard_cased",
          "value": "/usr/li*/readme.md"
        },
        {
          "field": "process.name",
          "operator": "excluded",
          "type": "exact_cased",
          "value": "readme.md"
        },
        {
          "field": "file.path.text",
          "operator": "excluded",
          "type": "wildcard_cased",
          "value": "/usr/bi*/arch*.txt"
        }
      ]
    }
  ]
}

Checklist

Delete any items that are not applicable to this PR.

• Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
• Documentation was added for features that require explanation or tutorials
• Unit or functional tests were updated or added to match the most common scenarios
• Any UI touched in this PR does not create any new axe failures (run axe in browser: FF, Chrome)
• This renders correctly on smaller devices using a responsive layout. (You can test this in your browser)
• This was checked for cross-browser compatibility

[intxgo]

actually, is it possible to combine duplicated entries into one like this:

        {
          "field": "file.path.text",
          "operator": "excluded",
          "type": "wildcard_cased",
          "value": [
             "/usr/li*/readme.md",
             "/opt/*in/sys.dmg"
           ]
        },
        {
          "field": "process.name",
          "operator": "excluded",
          "type": "exact_cased",
          "value": [
             "readme.md",
             "sys.dmg"
           ]
        },

We support such filters, here is an example:
https://github.com/elastic/endpoint-dev/blob/ce0d146dfeacd59789a890f5065019871fc4fb23/Libraries/FilterLib/Tests/filter_data/test-wildcard-7/exceptionlist.json

Having it this way is a performance improvement as the wildcard is converted to one regex with OR match pattern.

OK, looking again the the example, those two entries have different operator (included vs excluded) so couldn't be combined, but just keep this in mind in general.

[ashokaditya]

actually, is it possible to combine duplicated entries into one like this:

        {
          "field": "file.path.text",
          "operator": "excluded",
          "type": "wildcard_cased",
          "value": [
             "/usr/li*/readme.md",
             "/opt/*in/sys.dmg"
           ]
        },
        {
          "field": "process.name",
          "operator": "excluded",
          "type": "exact_cased",
          "value": [
             "readme.md",
             "sys.dmg"
           ]
        },

We support such filters, here is an example: https://github.com/elastic/endpoint-dev/blob/ce0d146dfeacd59789a890f5065019871fc4fb23/Libraries/FilterLib/Tests/filter_data/test-wildcard-7/exceptionlist.json

Having it this way is a performance improvement as the wildcard is converted to one regex with OR match pattern.

OK, looking again the the example, those two entries have different operator (included vs excluded) so couldn't be combined, but just keep this in mind in general.

This is insightful @intxgo Thanks for pointing this out. I'll make sure I check this as I finish up the PR.

[ashokaditya]

actually, is it possible to combine duplicated entries into one like this:

        {
          "field": "file.path.text",
          "operator": "excluded",
          "type": "wildcard_cased",
          "value": [
             "/usr/li*/readme.md",
             "/opt/*in/sys.dmg"
           ]
        },
        {
          "field": "process.name",
          "operator": "excluded",
          "type": "exact_cased",
          "value": [
             "readme.md",
             "sys.dmg"
           ]
        },

We support such filters, here is an example: https://github.com/elastic/endpoint-dev/blob/ce0d146dfeacd59789a890f5065019871fc4fb23/Libraries/FilterLib/Tests/filter_data/test-wildcard-7/exceptionlist.json
Having it this way is a performance improvement as the wildcard is converted to one regex with OR match pattern.
OK, looking again the the example, those two entries have different operator (included vs excluded) so couldn't be combined, but just keep this in mind in general.

This is insightful @intxgo Thanks for pointing this out. I'll make sure I check this as I finish up the PR.

@intxgo we talked offline and you pointed out that I was chasing down the wrong path. The grouping of values is only needed for ORed entries. Since event filters AND entries we don't need this improvement in this PR.
Although, we probabaly want to improve TA artifacts for ORed entries in a new PR.

In short, entries such as

{
    "entries": [
        {
            type: 'simple',
            entries: [
                {
                    "field": "file.path.text",
                    "operator": "excluded",
                    "type": "wildcard_caseless",
                    "value": "c:\\bin*\\group.exe"
                }
            ]
        },
        {
            type: 'simple',
            entries: [
                {
                    "field": "file.path.text",
                    "operator": "excluded",
                    "type": "wildcard_caseless",
                    "value":  "d:\\doc*\\another.md"
                }
            ]
        }        
    ]
}

can be transformed to

{
    "entries": [
        {
            type: 'simple',
            entries: [
                {
                    "field": "file.path.text",
                    "operator": "excluded",
                    "type": "wildcard_caseless",
                    "value": [
                        "c:\\bin*\\group.exe", 
                       "d:\\doc*\\another.md"
                    ]
                }
            ]
        }       
    ]
}

but entries that look like

{
	type: 'simple',
	entries: [
		{
			"field": "file.path.text",
			"operator": "excluded",
			"type": "wildcard_caseless",
			"value": "c:\\bin*\\group.exe"
		},
        {
			"field": "file.path.text",
			"operator": "excluded",
			"type": "wildcard_caseless",
			"value": "d:\\doc*\\another.md"
		}
	]
}

should not be transformed further to group values yet.

[FrankHassanabad]

LGTM. Appreciate all the work to make these matches work out and all the feedback you changed here. Really awesome.

[kibana-ci]

💚 Build Succeeded

• Buildkite Build
• Commit: 4a8cec3
• Storybooks Preview
• Webpack Bundle Analyzer

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
lists	298	300	+2
securitySolution	2886	2888	+2
total			+4

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id	before	after	diff
@kbn/securitysolution-autocomplete	34	35	+1
@kbn/securitysolution-list-utils	178	183	+5
@kbn/securitysolution-utils	4	26	+22
lists	161	162	+1
total			+29

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
lists	140.8KB	147.4KB	+6.6KB
securitySolution	4.7MB	4.7MB	+6.4KB
total			+13.0KB

Unknown metric groups

API count

id	before	after	diff
@kbn/securitysolution-autocomplete	47	50	+3
@kbn/securitysolution-list-utils	223	231	+8
@kbn/securitysolution-utils	6	28	+22
lists	197	198	+1
total			+34

History

• 💚 Build #26886 succeeded 9a2b2ed
• 💔 Build #26564 failed 3f8249d
• 💚 Build #26503 succeeded f9ab0e1
• 💚 Build #26237 succeeded d987ba3
• 💚 Build #26114 succeeded 7851f4d
• 💚 Build #25928 succeeded ca8ff13

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @ashokaditya

[kibanamachine]

Friendly reminder: Looks like this PR hasn’t been backported yet.
To create backports run node scripts/backport --pr 125202 or prevent reminders by adding the backport:skip label.