[windows] Add Initial AppLocker Data Stream (EXE and DLL) #6977
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
modified: windows/changelog.yml modified: windows/data_stream/applocker_exe_and_dll/elasticsearch/ingest_pipeline/default.yml modified: windows/data_stream/applocker_exe_and_dll/fields/winlog.yml modified: windows/data_stream/applocker_exe_and_dll/manifest.yml modified: windows/data_stream/applocker_exe_and_dll/sample_event.json modified: windows/data_stream/sysmon_operational/_dev/test/pipeline/test-events.json modified: windows/docs/README.md modified: windows/manifest.yml
4 tasks
nicpenning
changed the title
|
Pinging @elastic/security-external-integrations (Team:Security-External Integrations) |
|
Ready for review. 👍🏻 |
efd6
reviewed
Jul 17, 2023
packages/windows/data_stream/applocker_exe_and_dll/agent/stream/winlog.yml.hbs
Outdated
Show resolved
Hide resolved
| field: winlog.event_data.User | ||
| target_field: "_temp.user_parts" | ||
| separator: '\\' | ||
| if: ctx?.winlog?.event_data?.User != null |
There was a problem hiding this comment.
What are we looking at here?
There was a problem hiding this comment.
Suggested change
| if: ctx?.winlog?.event_data?.User != null | |
| if: ctx.winlog?.event_data?.User != null |
There was a problem hiding this comment.
This should happen throughout. For clarity (I missed the mark), the original comment is a sed global search and replace expression: search for re:"ctx?." and replace all instances with "ctx.".
There was a problem hiding this comment.
This does not appear to have been generated by a system test.
There was a problem hiding this comment.
It was not. I must have missed that. Since there isn't any sample documents, perhaps that wasn't possible?
I have a hard time understanding how one obtains a sample document from a log source that lives in the Windows event viewer.
I essentially cloned the PowerShell data stream and tried to clean it up and generate where I could.
Thank you for the review and quick responses!
…m/winlog.yml.hbs Co-authored-by: Dan Kortschak <90160302+efd6@users.noreply.github.com>
Co-authored-by: Dan Kortschak <90160302+efd6@users.noreply.github.com>
💚 Build Succeeded
Expand to view the summary
Build stats
Test stats 🧪
🤖 GitHub commentsExpand to view the GitHub comments
To re-run your PR in the CI, just comment with:
|
…ch/ingest_pipeline/default.yml Co-authored-by: Dan Kortschak <90160302+efd6@users.noreply.github.com>
|
I will see what I can do for the pipeline and system tests and also generate the sample event from said tests. |
|
@efd6, do you have a link to some documentation or a recommendation on how to create the test-events.json? I have plenty of example events but unsure how to dump them before they get sent to the Integration. Do I run the agent in debug mode? Is there some configurations I can make to extract test-events? |
|
Yes, that is how I know it works and how I found some ingest errors when testing. |
|
Cool. So that I imagine is for the winlog case? Has the httpjson case been tested? I don't think we can expect to test the winlog case, but httpjson is just sending POST requests, so we should be able to simulate that. |
|
Correct, that is for the winlog input. I have not tested the httpjson case, but in theory, I should have all of the files in place to test that. |
|
OK. So let's test the HTTPJSON case and leave the winlog case, noting that it has been tested in the wild. |
|
The system tests fail. I am unsure how to run just the splunk test, but either way, it appears to be running since I see an agent get created in Kibana. This is from an filebeat event in regards to the splunk mock.
And here are the logs from the file path mentioned above from the docker container: I don't quite uderstand what I am looking at. What do you think our next steps are? |
efd6
reviewed
Jul 25, 2023
|
Just fixed file paths in xml. Ready to test splunk again! |
|
I went ahead and added the convert with removal of the field as described above as the pipeline, and it appears that the system test worked. 2023/07/25 08:59:26 INFO Write container logs to file: /home/napsta/integrations/build/container-logs/splunk-mock-1690293566616699674.log
--- Test results for package: windows - START ---
╭─────────┬───────────────────────┬───────────┬───────────┬────────┬───────────────╮
│ PACKAGE │ DATA STREAM │ TEST TYPE │ TEST NAME │ RESULT │ TIME ELAPSED │
├─────────┼───────────────────────┼───────────┼───────────┼────────┼───────────────┤
│ windows │ applocker_exe_and_dll │ system │ default │ PASS │ 25.715349609s │
╰─────────┴───────────────────────┴───────────┴───────────┴────────┴───────────────╯
--- Test results for package: windows - END ---
DoneHowever, I have not been able to generate the sample_event.json from the test. I tried to use: elastic-package test system --data-streams applocker_exe_and_dll generateHowever, that does not seem to do the trick (like it does for the pipeline tests). Please advise! Update: I was able to run the Added new sample_event.json file. Please check the convert processors I added. Otherwise, I think we are getting close! |
efd6
reviewed
Jul 25, 2023
packages/windows/data_stream/applocker_exe_and_dll/elasticsearch/ingest_pipeline/default.yml
Outdated
Show resolved
Hide resolved
|
/test |
|
Needs a |
On it |
|
Ran the build and then pushed. Good? |
|
/test |
🌐 Coverage report
|
efd6
approved these changes
Jul 26, 2023
|
🙏🏻 my first data stream added complete. Thank you @efd6 for the patience and guidance on this!! More PRs for refinement to come on these types of events. 🚀 |
|
Package windows - 1.27.0 containing this change is available at https://epr.elastic.co/search?package=windows |
gizas
pushed a commit
that referenced
this pull request
Sep 5, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What does this PR do?
This is the first iteration of adding the AppLocker event logs to the Windows Integration.
Resolves Part of - #6979
Improved ECS, pipelines, and dashboards will come with future PRs.
Checklist
changelog.ymlfile.