Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Disable unmapped fields filter #4929

Merged
merged 11 commits into from
Dec 22, 2022

Conversation

asteriscos
Copy link
Member

@asteriscos asteriscos commented Nov 28, 2022

Description

Team,
this PR disables filter buttons of unmapped fields in the Security Events table.

Prevents filtering errors when clicking unmapped fields filters:

Screenshot from 2022-11-28 19-26-20

Screenshot from 2022-11-28 19-25-20

Issues Resolved

Closes #4932
Related issue #4429

Evidence

Peek 2022-11-28 18-32

Test

  • Add alerts with unmapped fields.
Example alert
{"timestamp":"2022-08-31T16:13:21.285+0000","rule":{"level":3,"description":"AWS Cloudtrail: gefv2.amazonaws.com - UpdateWebACL.","id":"80202","firedtimes":1,"mail":false,"groups":["amazon","aws","aws_cloudtrail"],"gdpr":["IV_35.7.d"],"hipaa":["164.312.b"],"nist_800_53":["AU.6"],"pci_dss":["10.6.1"],"tsc":["CC7.2","CC7.3"]},"agent":{"id":"000","name":"wazuh_manager_filebeat_sources_cmake-v4.3.7-7.10.2"},"manager":{"name":"wazuh_manager_filebeat_sources_cmake-v4.3.7-7.10.2"},"id":"1661184801.1137630","cluster":{"name":"wazuh","node":"master-node"},"decoder":{"name":"json"},"data":{"integration":"aws","aws":{"log_info":{"log_file":"AWSLogs/166157441623/CloudTrail/us-west-1/2022/08/17/166157441623_CloudTrail_us-west-1_20220817T0000Z_HASDoKlxgfdkdIOa.json.txt","s3bucket":"wazuh-aws-wodle-waf"},"eventVersion":"1.08","userIdentity":{"type":"AssumedRole","principalId":"asdf@asdf.com","arn":"-","accountId":"166157441623","accessKeyId":"-","sessionContext":{"sessionIssuer":{"type":"Role","principalId":"-","arn":"-"},"attributes":{"creationDate":"2022-07-05T16:49:18Z","mfaAuthenticated":"false"}}},"eventTime":"2022-07-05T18:20:27Z","eventSource":"gefv2.amazonaws.com","eventName":"UpdateWebACL","awsRegion":"us-west-2","sourceIPAddress":"AWS Internal","userAgent":"AWS Internal","requestParameters":{"name":"ec-web-acl","scope":"REGIONAL","id":"xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx","description":"ec-web-acl","rules":[{"name":"X-Application-ID","priority":0,"statement":{"byteMatchStatement":{"searchString":{"hb":[115,121,110,116,104,101,116,105,99],"offset":0,"isReadOnly":false,"bigEndian":true,"nativeByteOrder":false,"mark":-1,"position":0,"limit":9,"capacity":9,"address":0},"fieldToMatch":{"singleHeader":{"name":"x-application-id"}},"textTransformations":[{"priority":0,"type":"LOWERCASE"}],"positionalConstraint":"CONTAINS"}},"action":{"count":{}},"ruleLabels":[{"name":"RulesCustom:Application:Synthetic"}],"visibilityConfig":{"sampledRequestsEnabled":true,"cloudWatchMetricsEnabled":true,"metricName":"X-Application-ID"}},{"name":"MS-Internos","priority":1,"statement":{"andStatement":{"statements":[{"sizeConstraintStatement":{"fieldToMatch":{"singleHeader":{"name":"x-application-id"}},"comparisonOperator":"GE","size":0,"textTransformations":[{"priority":0,"type":"LOWERCASE"}]}},{"notStatement":{"statement":{"byteMatchStatement":{"searchString":{"hb":[115,121,110,116,104,101,116,105,99],"offset":0,"isReadOnly":false,"bigEndian":true,"nativeByteOrder":false,"mark":-1,"position":0,"limit":9,"capacity":9,"address":0},"fieldToMatch":{"singleHeader":{"name":"x-application-id"}},"textTransformations":[{"priority":0,"type":"LOWERCASE"}],"positionalConstraint":"CONTAINS"}}}}]}},"action":{"count":{}},"visibilityConfig":{"sampledRequestsEnabled":true,"cloudWatchMetricsEnabled":true,"metricName":"MS-Internos"}},{"name":"blueteam-rules","priority":2,"statement":{"ruleGroupReferenceStatement":{"aRN":"arn:aws:gefv2:us-west-2:166157441623:regional/rulegroup/blueteam/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"}},"overrideAction":{"none":{}},"visibilityConfig":{"sampledRequestsEnabled":true,"cloudWatchMetricsEnabled":true,"metricName":"blueteam-rules"}},{"name":"MS-ECOM-PARTNER","priority":3,"statement":{"byteMatchStatement":{"searchString":{"hb":[47,101,99,111,109,45,112,97,114,116,110,101,114,115,47],"offset":0,"isReadOnly":false,"bigEndian":true,"nativeByteOrder":false,"mark":-1,"position":0,"limit":15,"capacity":15,"address":0},"fieldToMatch":{"uriPath":{}},"textTransformations":[{"priority":0,"type":"LOWERCASE"}],"positionalConstraint":"CONTAINS"}},"action":{"count":{}},"visibilityConfig":{"sampledRequestsEnabled":true,"cloudWatchMetricsEnabled":true,"metricName":"MS-ECOM-PARTNER"}},{"name":"RL_ALL","priority":4,"statement":{"rateBasedStatement":{"limit":1000,"aggregateKeyType":"FORWARDED_IP","forwardedIPConfig":{"headerName":"X-Forwarded-For","fallbackBehavior":"MATCH"}}},"action":{"count":{}},"ruleLabels":[{"name":"CustomRules:RateLimitMS"}],"visibilityConfig":{"sampledRequestsEnabled":true,"cloudWatchMetricsEnabled":true,"metricName":"HttpFlood"}},{"name":"Request_From_Country","priority":5,"statement":{"andStatement":{"statements":[{"notStatement":{"statement":{"geoMatchStatement":{"countryCodes":["AR","BR","CL","CO","CR","EC","MX","PE","UY","US"],"forwardedIPConfig":{"headerName":"X-Forwarded-For","fallbackBehavior":"MATCH"}}}}},{"byteMatchStatement":{"searchString":{"hb":[47,114,101,115,116,97,117,114,97,110,116,115,45,98,117,115,47],"offset":0,"isReadOnly":false,"bigEndian":true,"nativeByteOrder":false,"mark":-1,"position":0,"limit":17,"capacity":17,"address":0},"fieldToMatch":{"uriPath":{}},"textTransformations":[{"priority":0,"type":"LOWERCASE"}],"positionalConstraint":"CONTAINS"}}]}},"action":{"block":{}},"visibilityConfig":{"sampledRequestsEnabled":true,"cloudWatchMetricsEnabled":true,"metricName":"Request_From_Country"}},{"name":"rl-es-proxy","priority":6,"statement":{"rateBasedStatement":{"limit":750,"aggregateKeyType":"FORWARDED_IP","scopeDownStatement":{"byteMatchStatement":{"searchString":{"hb":[47,97,112,105,47,101,115,45,112,114,111,120,121,47,115,101,97,114,99,104,47,118,50,47,112,114,111,100,117,99,116,115],"offset":0,"isReadOnly":false,"bigEndian":true,"nativeByteOrder":false,"mark":-1,"position":0,"limit":32,"capacity":32,"address":0},"fieldToMatch":{"uriPath":{}},"textTransformations":[{"priority":0,"type":"LOWERCASE"}],"positionalConstraint":"STARTS_WITH"}},"forwardedIPConfig":{"headerName":"X-Forwarded-For","fallbackBehavior":"MATCH"}}},"action":{"block":{}},"visibilityConfig":{"sampledRequestsEnabled":true,"cloudWatchMetricsEnabled":true,"metricName":"rl-elasticsearch-proxy"}},{"name":"restaurant-bus-rl","priority":7,"statement":{"rateBasedStatement":{"limit":350,"aggregateKeyType":"FORWARDED_IP","scopeDownStatement":{"regexPatternSetReferenceStatement":{"aRN":"arn:aws:gefv2:us-west-2:166157441623:regional/regexpatternset/toppings-rl/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx","fieldToMatch":{"uriPath":{}},"textTransformations":[{"priority":0,"type":"LOWERCASE"}]}},"forwardedIPConfig":{"headerName":"X-Forwarded-For","fallbackBehavior":"MATCH"}}},"action":{"block":{}},"visibilityConfig":{"sampledRequestsEnabled":true,"cloudWatchMetricsEnabled":true,"metricName":"restaurant-bus-rl"}}],"visibilityConfig":{"sampledRequestsEnabled":"true","cloudWatchMetricsEnabled":"true","metricName":"ec-web-acl"},"lockToken":"xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"},"responseElements":{"nextLockToken":"xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"},"requestID":"xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx","eventID":"xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx","readOnly":"false","eventType":"AwsApiCall","apiVersion":"2019-04-23","managementEvent":"true","recipientAccountId":"166157441623","eventCategory":"Management","sessionCredentialFromConsole":"true","source":"cloudtrail","aws_account_id":"166157441623"}},"location":"Wazuh-AWS"}

If you are using a Wazuh manager with Filebeat you can append this alert to the alerts.json file and this will send it to Elasticsearch.

  • Open the details of the alert in the Security Alerts table
  • Click the magnifier glass icon to filter by the unmapped field

Scenario 1: Unindexed fields can't be filtered
Given: environment with unknown fields in the index pattern
When: the user goes to the Events tab from a module
And the user explores the details of an alert seeing the unindexed field
Then the user can't set the filters using the magnifier glass icon

Scenario 2: Indexed fields can be filtered
Given: environment with unknown fields in the index pattern
When: the user goes to the Events tab from a module
And the user explores the details of an alert seeing the indexed field
Then the user can set the filters using the magnifier glass icon

Scenario 3: No filtrabe fields can't be filtered
Given: environment with unknown fields in the index pattern
When: the user goes to the Events tab from a module
And the user explores the details of an alert seeing the unindexed field
Then the user can't set the filters using the magnifier glass icon

Scenario 4: Unindexed fields can't be filtered
Given: environment with unknown fields in the index pattern
When: the user goes to the Dashboard tab from Security events module
And the user explores the details of an alert in the Alerts Table seeing the unindexed field
Then the user can't set the filters using the magnifier glass icon

Scenario 5: Indexed fields can be filtered
Given: environment with unknown fields in the index pattern
When: the user goes to the Dashboard tab from Security events module
And the user explores the details of an alert in the Alerts Table seeing the indexed field
Then the user can set the filters using the magnifier glass icon

Scenario 6: No filtrabe fields can't be filtered
Given: environment with unknown fields in the index pattern
When: the user goes to the Dashboard tab from Security events module
And the user explores the details of an alert in the Alerts Table seeing the unindexed field
Then the user can't set the filters using the magnifier glass icon

Check List

  • All tests pass
    • yarn test:jest
  • New functionality includes testing.
  • New functionality has been documented.
  • Update CHANGELOG.md
  • Commits are signed per the DCO using --signoff

@asteriscos asteriscos requested a review from a team as a code owner November 28, 2022 18:31
@asteriscos asteriscos self-assigned this Nov 28, 2022
Desvelao
Desvelao previously approved these changes Nov 29, 2022
Copy link
Member

@Desvelao Desvelao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

review:
✔️ code

⚠️ missing entry in the changelog
✔️ tests

Tests

  • Modules/Security events/Dashboard table
    image

  • Modules/Security events/Events tab
    image

Desvelao
Desvelao previously approved these changes Nov 29, 2022
Copy link
Member

@Desvelao Desvelao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

review

LGTM

yenienserrano
yenienserrano previously approved these changes Dec 16, 2022
Copy link
Member

@yenienserrano yenienserrano left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

image

yenienserrano
yenienserrano previously approved these changes Dec 16, 2022
Desvelao
Desvelao previously approved these changes Dec 20, 2022
Copy link
Member

@Desvelao Desvelao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

review

✔️ code
✔️ test

Tests

Scenario 1 Result
1 🟢
2 🟢
3 🟢
4 🟢
5 🟢
6 🟢
Scenario 1 🟢

image

Scenario 2 🟢

image
image

Scenario 3 🟢

After updating the index pattern fields. This field is not filtrable and the filter can not be added.
image

Scenario 4 🟢

image
image

Scenario 5 🟢

image
image

Scenario 6 🟢

After updating the index pattern fields. This field is not filtrable and the filter can not be added.
image

@Desvelao Desvelao added the type/bug Bug issue label Dec 20, 2022
@github-actions
Copy link
Contributor

Code coverage (Jest) % values
Statements 8.74% ( 3234 / 36997 )
Branches 4.57% ( 1313 / 28749 )
Functions 7.59% ( 696 / 9165 )
Lines 8.81% ( 3121 / 35423 )

Copy link
Member

@Machi3mfl Machi3mfl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CR: ✔️
Test: ✔️

AWS Events

Peek.2022-12-22.14-30.mp4

Security Events

Peek.2022-12-22.14-31.mp4

@asteriscos asteriscos merged commit 8bce43f into 4.4-7.10 Dec 22, 2022
@asteriscos asteriscos deleted the fix/disable-unmapped-fields-filters-4429 branch December 22, 2022 17:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type/bug Bug issue
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Unmapped field filter in the Security Events table throws an exception
4 participants