Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cef: make general purpose dashboards #3526

Merged
merged 5 commits into from
Jul 6, 2022
Merged

cef: make general purpose dashboards #3526

merged 5 commits into from
Jul 6, 2022

Conversation

efd6
Copy link
Contributor

@efd6 efd6 commented Jun 17, 2022 •
edited
Loading

What does this PR do?

This mirrors the ArcSight CEF dashboard where possible by mapping ArcSight extensions to ECS and removes visualisations and searches that are not expressible without those extensions.

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.

Author's Checklist

  • [ ]

How to test this PR locally

Related issues

Screenshots

@efd6 efd6 self-assigned this Jun 17, 2022
@elasticmachine
Copy link

elasticmachine commented Jun 17, 2022 •
edited
Loading

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2022-06-29T02:36:05.454+0000

  • Duration: 18 min 29 sec

Test stats 🧪

Test Results
Failed 0
Passed 159
Skipped 0
Total 159

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.

@elasticmachine
Copy link

elasticmachine commented Jun 17, 2022 •
edited
Loading

🌐 Coverage report

Name Metrics % (covered/total) Diff
Packages 100.0% (1/1) 💚
Files 100.0% (3/3) 💚 3.401
Classes 100.0% (3/3) 💚 3.401
Methods 95.455% (21/22) 👍 6.592
Lines 89.831% (424/472) 👎 -0.117
Conditionals 100.0% (0/0) 💚

@efd6 efd6 marked this pull request as ready for review June 20, 2022 02:41
@efd6 efd6 requested a review from a team as a code owner June 20, 2022 02:41
@elasticmachine
Copy link

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

andrewkroh
andrewkroh reviewed Jun 24, 2022
View reviewed changes
Copy link
Member

@andrewkroh andrewkroh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you please add screenshots to the PR description

Also consider embedding the visualizations in the dashboards by unlinking them or trying this inliner tool https://github.com/flash1293/legacy_vis_analyzer#inliner-usage.

efd6 added 5 commits June 29, 2022 12:04
@efd6
deep clone existing dashboard
90b4611 
The process for this was:

clonedash -src $(pwd) -dry-run=false
find kibana -name '*.json' -print0 | xargs -0 gsed -i -r 's/( via)? ArcSight//g; s!"/Attempt"!"unknown"!g; s!"/Success"!"success"!g; s!"/Failure"!"failure"!g; s!"query": "cef.extensions.categoryDeviceGroup:\\"/Operating System\\" OR cef.extensions.categoryDeviceGroup:\\"/IDS/Host\\" OR cef.extensions.categoryDeviceGroup:\\"/Application\\""!"query": "data_stream.dataset:\\"cef.log\\""!g; s/"cef\.extensions\.categoryBehavior"/"event.action"/g; s/"cef\.extensions\.categoryOutcome"/"event.outcome"/g; s/^(.*)"cef\.extensions\.categorySignificance"/\1"event.category",\n\1"event.type"/g;'
git reset --hard
@efd6
remove visualisations that depend on specific features
d931398
@efd6
remove orfans
a3dce38
@efd6
degrade operating system dashboard
56420eb 
This dashboard depends on the ArcSight category "Device Group" which has no
direct mapping in ECS or any fields in CEF that can be used to construct it in
the general case, so degrade it to all endpoints from only operating system
endpoints rather than losing the dashboard entirely.
@efd6
add changelog and manifest
16c495e
@efd6
Copy link
Contributor Author

efd6 commented Jul 5, 2022

There are some holes in the dashboards. Beyond removing them, I'm not sure what to do with that.

Network Overview
Screen Shot 2022-07-05 at 11 02 03

Suspicious Activity
Screen Shot 2022-07-05 at 11 00 30

Endpoint Overview
Screen Shot 2022-07-05 at 11 01 11

Endpoint Activity
Screen Shot 2022-07-05 at 11 00 10

Microsoft DNS Overview
Screen Shot 2022-07-05 at 11 33 09

@efd6
Copy link
Contributor Author

efd6 commented Jul 6, 2022

A note on the origin of the data used to simulate for the dashboards. This is a general purpose CEF simulator that I will clean up for spigot. I needed to be customised for CheckPoint since that appears to be what the integration (along with Forcepoint) seems to be aimed at.

@efd6 efd6 merged commit 6e75bd8 into elastic:main Jul 6, 2022
@andrewkroh andrewkroh mentioned this pull request Jul 13, 2022
Merged
@efd6 efd6 mentioned this pull request Aug 23, 2022
Merged
7 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@andrewkroh andrewkroh andrewkroh approved these changes

Assignees

@efd6 efd6

Labels
enhancement New feature or request Integration:CEF
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@efd6 @elasticmachine @andrewkroh