Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added statistics discover template #3111

Merged
merged 12 commits into from
Apr 13, 2021

Conversation

Machi3mfl
Copy link
Member

@Machi3mfl Machi3mfl commented Mar 29, 2021

Hi Team, this resolves

  • Adding wazuh-statistics index template and mapping

Test suit:

Note: in order to get statistics with a short frequency, you could set these configs in your wazuh.yml

cron.statistics.index.creation: 'h'
cron.statistics.interval: '0/5 * * * * *'
  • The new template wazuh-statistics should be generated
    image
  • The new template should support a custom cron.index and cron.statistics.index.name
  • The new statistics docs should be mapped correctly
    image
  • The statictis views should work correctly

Closes #3072

@Machi3mfl Machi3mfl changed the base branch from master to 4.1-7.10 March 29, 2021 20:08
@Machi3mfl Machi3mfl self-assigned this Mar 29, 2021
@Machi3mfl Machi3mfl changed the title Feature/3072 added statistics discover template Added statistics discover template Mar 29, 2021
@frankeros frankeros changed the base branch from 4.1-7.10 to 4.2-7.10 April 7, 2021 16:44
@frankeros frankeros marked this pull request as ready for review April 7, 2021 16:47
@frankeros frankeros self-assigned this Apr 7, 2021
Copy link
Contributor

@gabiwassan gabiwassan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

CHANGELOG.md Outdated
@@ -12,6 +12,7 @@ All notable changes to the Wazuh app project will be documented in this file.
- Added Agent Stats section [#3056](https://github.com/wazuh/wazuh-kibana-app/pull/3056)
- Add `logtest` PUT example on API Console [#3061](https://github.com/wazuh/wazuh-kibana-app/pull/3061)
- New buttom to check api again in health check [#3109](https://github.com/wazuh/wazuh-kibana-app/pull/3109)
- Added `wazuh-statistics` template and a new mapping for this indices [#3111](https://github.com/wazuh/wazuh-kibana-app/pull/3111)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fix typo this indices to these indices.

Copy link
Contributor

@pablomarga pablomarga left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These fields should be added to public/utils/statistics-fields.ts:

{ "name": "_id", "type": "string", "count": 0, "scripted": false, "searchable": true, "aggregatable": true, "readFromDocValues": false },
{ "name": "_index", "type": "string", "count": 0, "scripted": false, "searchable": true, "aggregatable": true, "readFromDocValues": false },
{ "name": "_score", "type": "number", "count": 0, "scripted": false, "searchable": false, "aggregatable": false, "readFromDocValues": false },
{ "name": "_source", "type": "_source", "count": 0, "scripted": false, "searchable": false, "aggregatable": false, "readFromDocValues": false },
{ "name": "_type", "type": "string", "count": 0, "scripted": false, "searchable": true, "aggregatable": true, "readFromDocValues": false },

Due to if you had a previously created index pattern, it may not have the _source mapping field and it will make the entire document look like a JSON in Discover instead of highlighting the fields of the object

@Desvelao
Copy link
Member

Desvelao commented Apr 8, 2021

Testing
✅ Create wazuh-statistics template
✅ Create indices with new mapping
❌ Some visualizations in Management > Statistics are not displayed. This maybe could related to the change in remoted and analisysd from type [{remoted|analisyd}] to {remoted|analisyd} ?
image
image
image

❗ The file with the known fields for statistics needs to be updated with the same field mapping public/utils/statistics-fields.ts. Some meta fields are missing in the know fields as _id, _index, _score, _source and _type. The _source metafield is important to display the formatted fields like the screenshot:
image

@pablomarga
Copy link
Contributor

Testing
✅ Create wazuh-statistics template
✅ Create indices with new mapping
❌ Some visualizations in Management > Statistics are not displayed. This maybe could related to the change in remoted and analisysd from type [{remoted|analisyd}] to {remoted|analisyd} ?
image
image
image

Everything mentioned by @Desvelao is also happening to me

@Desvelao Desvelao added the type/enhancement Enhancement issue label Apr 8, 2021
@Desvelao
Copy link
Member

Changes

  • Fixed the statistics visualizations expressions replacing all the instances of wazuh-statistics-* index placeholder instead of the first one.
  • Fixed the template for wazuh-statistics indices when using a custom name for the indices instead of statistics. Now add the index pattern to the template correctly.

Copy link
Contributor

@frankeros frankeros left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Copy link
Contributor

@pablomarga pablomarga left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested it and seems to work as expected. Great job! LGTM!

@Desvelao Desvelao merged commit a14ad36 into 4.2-7.10 Apr 13, 2021
@Desvelao Desvelao deleted the feature/3072-added-statistics-discover-template branch April 13, 2021 09:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type/enhancement Enhancement issue
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants