Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Remove discover - Implement embeddable dashboard on statistics module #6542

Merged
merged 20 commits into from
Apr 22, 2024
Merged

Remove discover - Implement embeddable dashboard on statistics module #6542

merged 20 commits into from
Apr 22, 2024

Conversation

jbiset
Copy link
Member

@jbiset jbiset commented Mar 20, 2024 •
edited
Loading

Description

This pull request implement the embeddable dashboard on Server Management / Statistics and deprecate any use of kibana-integrations components.
Also fixes an issue in manage-hosts.ts by fixing the cluster_info attribute that was not being added correctly. This caused the statistics index to not be generated.

Issues Resolved

Evidence

  • Each visualization, if applicable, must have interaction so that it adds the corresponding date range upon clicking.
Evidence_1.webm
  • The visualizations have to be updated according to the date range applied in the searchbar.
Evidence_2.webm
  • The opensearch request of Events processed and Events dropped visualizations must have the nodeName filter and nodeName.keyword key filter when a node is selected in cluster mode.

Events processed

image

Events dropped

image

  • The opensearch request of visualizations must have the nodeName filter when a node is selected in cluster mode.

image

  • Check that the node selector does not appear in manager mode.

Manager mode

image

Cluster Mode

image

  • Check that the apiName filter appears in the opensearch request of the visualizations .

image

  • Check that the statistics index is created correctly, consulting the indexes through devTools: GET _cat/indices

image

Test

Go to Server Management -> Statistics and check the following:

  • Each visualization, if applicable, must have interaction so that it adds the corresponding date range upon clicking.
  • The visualizations have to be updated according to the date range applied in the searchbar.
  • The opensearch request of Events processed and Events dropped visualizations must have the nodeName filter and nodeName.keyword key filter when a node is selected in cluster mode.
  • The opensearch request of visualizations must have the nodeName filter when a node is selected in cluster mode.
  • Check that the node selector does not appear in manager mode.
  • Check that the apiName filter appears in the opensearch request of the visualizations .
  • Check that the statistics index is created correctly, consulting the indexes through devTools: GET _cat/indices

Check List

  • All tests pass
    • yarn test:jest
  • New functionality includes testing.
  • New functionality has been documented.
  • Update CHANGELOG.md
  • Commits are signed per the DCO using --signoff

@jbiset jbiset self-assigned this Mar 20, 2024
@jbiset jbiset linked an issue Mar 20, 2024 that may be closed by this pull request
[Remove discover] Implement embeddable dashboard on Server Management / Statistics module #6530
Closed
18 tasks
@jbiset jbiset marked this pull request as ready for review April 11, 2024 20:33
@jbiset
Added No results message, fixed allow agents filters on request and c…
6a7636f 
…leaned obsolete code
@jbiset
Changed condition of apiName filter, validation of statistics-data-so…
87abf0e 
…urce-repository and abstracted behavior between DashboardListenerEngineStatistics and DashboardAnalysisEngineStatistics
@Tostti
Copy link
Member

Tostti commented Apr 15, 2024 •
edited
Loading

Test

  • Each visualization, if applicable, must have interaction so that it adds the corresponding date range upon clicking.
  • The visualizations have to be updated according to the date range applied in the searchbar.
  • The opensearch request of Events processed and Events dropped visualizations must have the nodeName filter and nodeName.keyword key filter when a node is selected in cluster mode.
  • The opensearch request of visualizations must have the nodeName filter when a node is selected in cluster mode.
  • Check that the node selector does not appear in manager mode.
  • Check that the apiName filter appears in the opensearch request of the visualizations .
  • Check that the statistics index is created correctly, consulting the indexes through devTools: GET _cat/indices

image
image
image
image
image

Tostti
Tostti previously approved these changes Apr 15, 2024
View reviewed changes
Copy link
Member

@Tostti Tostti left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Test ✅
CR ✅

LGTM

@Desvelao
Copy link
Member

Desvelao commented Apr 16, 2024 •
edited
Loading

Test 🟢

  • The visualizations have to be updated according to the date range applied in the searchbar.
    image
    image
  • The opensearch request of Events processed and Events dropped visualizations must have the nodeName filter and nodeName.keyword key filter when a node is selected in cluster mode.
    image
  • Check that the node selector does not appear in manager mode.
    image
  • Check that the apiName filter appears in the opensearch request of the visualizations .
    image
  • Check that the statistics index is created correctly, consulting the indexes through devTools: GET _cat/indices
    image

@github-actions GitHub Actions
Copy link
Contributor

Wazuh Core plugin code coverage (Jest) test % values
Statements 45.94% ( 397 / 864 )
Branches 42.2% ( 157 / 372 )
Functions 44.11% ( 135 / 306 )
Lines 46.14% ( 395 / 856 )

@github-actions GitHub Actions
Copy link
Contributor

Wazuh Check Updates plugin code coverage (Jest) test % values
Statements 76.44% ( 172 / 225 )
Branches 58.65% ( 61 / 104 )
Functions 61.7% ( 29 / 47 )
Lines 76.44% ( 172 / 225 )

@github-actions GitHub Actions
Copy link
Contributor

Main plugin code coverage (Jest) test % values
Statements 10.78% ( 3745 / 34723 )
Branches 7.22% ( 1643 / 22728 )
Functions 10.48% ( 869 / 8289 )
Lines 10.93% ( 3640 / 33281 )

@Desvelao
Copy link
Member

Desvelao commented Apr 18, 2024 •
edited
Loading

It seems the view of Statistics is not protected with the user permissions check, it could give problems. I do not know if we want to solve this on this pull request.

@jbiset
Copy link
Member Author

jbiset commented Apr 19, 2024

It seems the view of Statistics is not protected with the user permissions check, it could give problems. I do not know if we want to solve this on this pull request.

Added withUserAuthorizationPrompt to add protection with the user permissions check.

@Desvelao
Copy link
Member

Remove unused code: 7e12b80

Desvelao
Desvelao previously approved these changes Apr 22, 2024
View reviewed changes
Copy link
Member

@Desvelao Desvelao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Tostti
Tostti approved these changes Apr 22, 2024
View reviewed changes
Copy link
Member

@Tostti Tostti left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@Tostti Tostti merged commit 3310c8d into 4.9.0 Apr 22, 2024
1 check passed
@Tostti Tostti deleted the 6530-remove-discover-implement-embeddable-dashboard-on-statistics-module branch April 22, 2024 16:51
Machi3mfl pushed a commit that referenced this pull request Apr 25, 2024
@jbiset @Desvelao @Machi3mfl
Remove discover - Implement embeddable dashboard on statistics module (…
cc949f7 
…#6542)

* Migrated visualizations to embeddables

* Changed searchbar and node selector

* Cleaned dashboard obsolete code

* Added selectedNodeFilter

* Integrated new data source on statistics

* Fixed statistics index without data.

* Added apiName filter

* Fixed nodeName filter when a node is selected in cluster mode

* Added No results message, fixed allow agents filters on request and cleaned obsolete code

* Changed condition of apiName filter, validation of statistics-data-source-repository and abstracted behavior between DashboardListenerEngineStatistics and DashboardAnalysisEngineStatistics

* Deleted unused  use-build-statistics-visualizations hook

* Fixed information message depending on active tab

* Added withUserAuthorizationPrompt to add protection with the user permissions check

* remove(statistics): unused message definitions

* Deleted unused index.ts file on cluster integration-files visualizations

---------

Co-authored-by: Antonio David Gutiérrez <antonio.gutierrez@wazuh.com>
Tostti pushed a commit that referenced this pull request Apr 25, 2024
@JuanGarriuz @Machi3mfl
@jbiset @Desvelao @asteriscos @yenienserrano
Implement new data source feature on MITRE ATT&CK module (#6482)
1f183a0 
* Moved mitre intelligence to a subfolder

* fix imports

* moved mitre framework

* fix framework imports

* Changed dashboard

* updated snapshots

* Fixed typo

* Added pinned agent mode

* Changed mitre dashboard

* Fix framework tab

* Add mitre data source

* Change framework tab class components to functional components

* Added date picker in search bar

* Add aggregattions to search

* Add columns definition in data grid

* Fix cluster disabled error on filter

* Add receive filter manager like prop

* Create generic components to reuse in discover

* Add mitre attack data source

* Create flyout discover

* Refactor mitre with data source

* Add default fetch filters

* Linter in wz-discover

* Create component to reuse in discover

* Add expanded techniques row inner component

* Apply initial fetch filters on flyout

* Linter on mitre module

* Fix cluster filter

* Fix UI styles

* Add redirect links and apply filters links

* Update CHANGELOG

* Remove unused code and imports

* Fix table headers and cell links

* Move mitre lib to react-services

* Fix files with prettier

* Signed commit

* [Remove discover] Implement embeddable dashboard on server management cluster module (#6561)

* Migrated Server Management Cluster to embeddables without cluster controller

* Fixed width of Cluster Configuration section

* Rendering conditions are adjusted and clean code

* Deleted controller and visualization monitoring files

* Integrated data-source implementation

* Fixed configuration_cards itemsList, the code is improved and unnecessary code is removed

* Added HOC withGuardAsync

* Changed the source of clusterEnabled information to that provided by the API

* Changed ClusterOverview component class to functional and fixed error on checkClusterIsEnabledAndRunning

* Added more dependencies in useEffect on Cluster Dashboard

* Added error handling to cluster-disabled component

* Fixed alert.timestamp field on SampleData

* Remove discover - Implement embeddable dashboard on statistics module (#6542)

* Migrated visualizations to embeddables

* Changed searchbar and node selector

* Cleaned dashboard obsolete code

* Added selectedNodeFilter

* Integrated new data source on statistics

* Fixed statistics index without data.

* Added apiName filter

* Fixed nodeName filter when a node is selected in cluster mode

* Added No results message, fixed allow agents filters on request and cleaned obsolete code

* Changed condition of apiName filter, validation of statistics-data-source-repository and abstracted behavior between DashboardListenerEngineStatistics and DashboardAnalysisEngineStatistics

* Deleted unused  use-build-statistics-visualizations hook

* Fixed information message depending on active tab

* Added withUserAuthorizationPrompt to add protection with the user permissions check

* remove(statistics): unused message definitions

* Deleted unused index.ts file on cluster integration-files visualizations

---------

Co-authored-by: Antonio David Gutiérrez <antonio.gutierrez@wazuh.com>

* Fix changelog entries (#6612)

* changelog: fix entries

* Change enrollment.dns changelog entry

---------

Co-authored-by: Federico Rodriguez <federico.rodriguez@wazuh.com>

* Remove AngularJS component click-action.js (#6613)

* remove click-action.js

* update changelog

* Update changelog

---------

Co-authored-by: Federico Rodriguez <federico.rodriguez@wazuh.com>

* Refactor settings angularJS controller (#6580)

* Create settings component

* Parcial refactor into react component

* Remove angularJS controller

* Remove unnecessary properties

* Add changelog

* fix: category parameter to preselect the app category settings on App Settings from Statistics

* Remove optional chaining operator

---------

Co-authored-by: Antonio David Gutiérrez <antonio.gutierrez@wazuh.com>

* Fix mitre columns error in techniques on hover

* Hide remove filters on search bar

* Use not global time filter and query in search bar

* Fix filters in visualizations

* Implement journald log collection feature (#6572)

* Added tab

* Add journald tab

* Added changes to imposter

* journald values filters table

* Update changelog and redesign filters table

* Fixed styles

* Fixed hardcode bug

* Added filters groups

* Change to an accordion render

* Added helps-link

* Added changelog

* Fix changelog and message popover improve

* Update configuration-setting imports

* Fix macOS log title

* resolve comments

* Fixed info euitext render and header no render in journald tab

* Add verification to mac and journald agents and add condition to journald

---------

Co-authored-by: Federico Rodriguez <federico.rodriguez@wazuh.com>

* Remove global filter manager from common-data

* Fix date range parser on requests

* Fix error when remove query saved

* Add timestamp formatted in events data grid

* [Remove discover] Implement embeddable dashboard on Virustotal module (#6525)

* Migrated visualizations, added loadings and messages

* Fixed warning for expected a single ReactElement

* Added new virustotal data source. Changed NoResults and LoadingSpinner to commons components

* Fixed Events tab

* Integrated pinned agent functionality based on data source

* Added timeRange to dashboard useEffect dependencies and changed the way to get pinned agent

* Deleted unused component

* Changed AlertsVirustotalDataSource import in modules-defaults

* Fixed error message

* DashboardByRenderer timeRange params replaced by searchBarProps deconstruction

* Added wz-discover hide-filter-control classes to hide the button that allows you to affect all the filters in the search bar

* Removed unnecessary virus total in Filters tab in common data to remove duplicate filters

* Removed unused getImplicitPinnedAgent in modules-helper

* Added dateRange param to fetchData in dashboard useEffect

* Improved AlertsVirustotalDataSource import in modules-defaults and deleted wz-discover on SearchBar wrapper

* [Remove discover] Implement embeddable dashboard on Threat Hunting module (#6486)

* Migrated visualizations to embeddables

* Added withPinnedAgent HOC

* Added dashboard update mechanism depending on whether or not an agent has been pinned

* Pinned agent visualization definitions are migrated and aesthetic adjustments are made

* The interaction was added to the KPIs, the links to the lower table were added, as well as the change of columns when an agent is set

* Integrated new data source on Threat Hunting module

* DiscoverNoResults and LoadingSpinner components are replaced with common components

* Clean code and fixed dashboards conditions

* Improved condition for rendering the dashboard and SampleData message

* Removed unnecessary general/threat hunting in tabFilters in common data to remove duplicate filters

* Removed unused getImplicitPinnedAgent in modules-helper

* Added dateRange param to fetchData in dashboard useEffect, added wz-discover hide-filter-control classes to hide the button that allows you to affect all the filters in the search bar and use searchbarProps deconstruction

* Deleted unnecessary wz-discover class on SearchBar wrapper

* Changed Threat Hunting columns file name

* Deleted unused imports in modules-helper

---------

Co-authored-by: Ian Yenien Serrano <63758389+yenienserrano@users.noreply.github.com>
Co-authored-by: Federico Rodriguez <federico.rodriguez@wazuh.com>

* Renamed virus total data source

* Change order fixed filters virus total

* Clean dashboard code

---------

Co-authored-by: Maximiliano <maximiliano.ibarra@wazuh.com>
Co-authored-by: Maximiliano Ibarra <6089438+Machi3mfl@users.noreply.github.com>
Co-authored-by: Julio César Biset <43619595+jbiset@users.noreply.github.com>
Co-authored-by: Antonio David Gutiérrez <antonio.gutierrez@wazuh.com>
Co-authored-by: Antonio <34042064+Desvelao@users.noreply.github.com>
Co-authored-by: Federico Rodriguez <federico.rodriguez@wazuh.com>
Co-authored-by: Ian Yenien Serrano <63758389+yenienserrano@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Tostti Tostti Tostti approved these changes

@Desvelao Desvelao Desvelao left review comments

Assignees

@jbiset jbiset

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[Remove discover] Implement embeddable dashboard on Server Management / Statistics module
3 participants
@jbiset @Tostti @Desvelao