[windows] Fix parsing of event data fields for event 600 #9490
Conversation
marc-gr
added
Integration:Windows
bugfix
Team:Security-Windows Platform
marc-gr
force-pushed
the
fix/powershell-600-data
branch
from
69c4dc4
to
fdf18ee
Compare
|
Pinging @elastic/sec-windows-platform (Team:Security-Windows Platform) |
marc-gr
force-pushed
the
fix/powershell-600-data
branch
from
fdf18ee
to
15e2b90
Compare
|
fyi I was checking our "SEI Demo" cluster and I see that error for 600, 403, and 400. 
|
packages/windows/data_stream/powershell/_dev/test/pipeline/test-events.json-expected.json
Show resolved
Hide resolved
🚀 Benchmarks reportPackage
|
| Data stream | Previous EPS | New EPS | Diff (%) | Result |
|---|---|---|---|---|
applocker_msi_and_script |
9523.81 | 7352.94 | -2170.87 (-22.79%) | 💔 |
powershell |
2392.34 | 1506.02 | -886.32 (-37.05%) | 💔 |
To see the full report comment with /test benchmark fullreport
| Split Events 4xx and 600 event data fields. | ||
| Some events can contain multiline values containing also '\n', '\s', and '=' characters, | ||
| for this reason a simple KV processor is not reliable enough and we need a more specific parsing. | ||
| lang: painless |
There was a problem hiding this comment.
packages/windows/data_stream/powershell/elasticsearch/ingest_pipeline/default.yml
Outdated
Show resolved
Hide resolved
| source: |- | ||
| def p = ctx.winlog?.event_data[params["field"]]; | ||
| // Define the pattern that will match all keys | ||
| def pat = /(^|(^[\n]?))?\t([^\s\W]+)=/m; |
There was a problem hiding this comment.
I think this regex assumes that the powershell script will be pretty print with indentation.
The loop will skip over such line form the event example
\n\t\t\t\t\t\t\t[Parameter(ValueFromPipeline=\"\", ValueFromPipelineByPropertyName=\"\",
But if the script was not print with identation, could this became
\n\t[Parameter(ValueFromPipeline=\"\", ValueFromPipelineByPropertyName=\"\",
and break the parser?
There was a problem hiding this comment.
I assume that the idea was to put the whole function body under one key
[HostApplication] = [C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe function...]
💚 Build Succeeded
History
|
|
100.0% Coverage on New Code
0.0% Duplication on New Code
|
Package windows - 1.44.5 containing this change is available at https://epr.elastic.co/search?package=windows |
Proposed commit message
Some 600 powershell events can contain multiline values, meaning the current KV split is not enough to handle them. This adds specific logic to handle these.
Checklist
changelog.ymlfile.Closes #9469