Skip to content

[Rule Tuning] Updated ESQL Rules Based on Validation Results #5151

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 14 commits into from
Sep 30, 2025
Merged

[Rule Tuning] Updated ESQL Rules Based on Validation Results #5151

merged 14 commits into from
Sep 30, 2025
+71 −50

Conversation

eric-forte-elastic
Copy link
Contributor

@eric-forte-elastic eric-forte-elastic commented Sep 25, 2025 •
edited
Loading

Pull Request

Issue link(s):

Summary - What I changed

ESQL Rule Updates based on validation findings from #4955.

In addition to these fixes, there are 8 other rules with issues shown below that still need to be addressed. It would be great @terrancedejesus if you could take a look at these. Thanks!

Note: you can run view-rule with the esql remote validation flag to check any given rule individually using the esql-field-validation branch from #4955.

E.g.

 python -m detection_rules view-rule rules/integrations/okta/credential_access_okta_authentication_for_multiple_users_with_the_same_device_token_hash.toml --esql-remote-validation
3fac01b2-b811-11ef-b25b-f661ea17fbce  FAILURE: <class 'detection_rules.esql_errors.EsqlSchemaError'>: BadRequestError(400, 'verification_exception', 'Found 1 problem\nline 10:9: Unknown column [azure.signinlogs.properties.mfa_detail.auth_method], did you mean any of [azure.signinlogs.properties.risk_detail, azure.signinlogs.properties.app_display_name, azure.signinlogs.properties.correlation_id, azure.signinlogs.properties.is_interactive, azure.signinlogs.properties.risk_state, azure.signinlogs.properties.resource_id, azure.signinlogs.properties.status.error_code, azure.signinlogs.properties.client_app_used, azure.signinlogs.properties.original_request_id, azure.signinlogs.properties.authentication_protocol, azure.signinlogs.properties.user_type, azure.signinlogs.properties.user_principal_name, azure.signinlogs.properties.app_id, azure.signinlogs.properties.conditional_access_status, azure.signinlogs.properties.resource_display_name, azure.signinlogs.properties.authentication_requirement, azure.signinlogs.operation_name, azure.signinlogs.result_description, azure.signinlogs.result_signature]?')
c07f7898-5dc3-11f0-9f27-f661ea17fbcd  FAILURE: <class 'detection_rules.esql_errors.EsqlSchemaError'>: BadRequestError(400, 'verification_exception', 'Found 2 problems\nline 35:66: Unknown column [azure.platformlogs.identity.claim.appid], did you mean any of [azure.platformlogs.identity.claim.upn, azure.platformlogs.result_type]?\nline 36:69: Unknown column [azure.platformlogs.identity.claim.objectid], did you mean any of [azure.platformlogs.identity.claim.upn, azure.platformlogs.result_type]?')
cca64114-fb8b-11ef-86e2-f661ea17fbce  FAILURE: <class 'detection_rules.esql_errors.EsqlSchemaError'>: BadRequestError(400, 'verification_exception', 'Found 1 problem\nline 50:65: Unknown column [azure.signinlogs.properties.session_id], did you mean any of [azure.signinlogs.properties.resource_id, azure.signinlogs.properties.user_id, azure.signinlogs.properties.app_id, azure.signinlogs.properties.risk_state, azure.signinlogs.properties.status.error_code, azure.signinlogs.properties.app_display_name, azure.signinlogs.properties.incoming_token_type, azure.signinlogs.properties.user_principal_name, azure.signinlogs.properties.resource_display_name, azure.signinlogs.properties.device_detail.device_id, azure.signinlogs.properties.device_detail.browser, azure.signinlogs.properties.authentication_requirement, azure.signinlogs.result_description, azure.signinlogs.properties.conditional_access_status, azure.signinlogs.properties.device_detail.operating_system, azure.signinlogs.result_signature, azure.signinlogs.category, azure.signinlogs.result_type]?')
2d6f5332-42ea-11f0-b09a-f661ea17fbcd  FAILURE: <class 'detection_rules.esql_errors.EsqlSchemaError'>: BadRequestError(400, 'verification_exception', 'Found 1 problem\nline 31:65: Unknown column [azure.signinlogs.properties.session_id], did you mean any of [azure.signinlogs.properties.resource_id, azure.signinlogs.properties.user_id, azure.signinlogs.properties.app_id, azure.signinlogs.properties.risk_state, azure.signinlogs.properties.status.error_code, azure.signinlogs.properties.app_display_name, azure.signinlogs.properties.incoming_token_type, azure.signinlogs.properties.user_principal_name, azure.signinlogs.properties.resource_display_name, azure.signinlogs.properties.device_detail.device_id, azure.signinlogs.properties.device_detail.browser, azure.signinlogs.properties.authentication_requirement, azure.signinlogs.result_description, azure.signinlogs.properties.conditional_access_status, azure.signinlogs.properties.device_detail.operating_system, azure.signinlogs.result_signature, azure.signinlogs.category, azure.signinlogs.result_type, Esql.azure_signinlogs_properties_app_display_name_lower, Esql.azure_signinlogs_properties_incoming_token_type_lower]?')
35ab3cfa-6c67-11ef-ab4d-f661ea17fbcc  FAILURE: <class 'detection_rules.esql_errors.EsqlSchemaError'>: BadRequestError(400, 'verification_exception', 'Found 1 problem\nline 54:65: Unknown column [azure.signinlogs.properties.session_id], did you mean any of [azure.signinlogs.properties.resource_id, azure.signinlogs.properties.user_id, azure.signinlogs.properties.app_id, azure.signinlogs.properties.risk_state, azure.signinlogs.properties.status.error_code, azure.signinlogs.properties.app_display_name, azure.signinlogs.properties.incoming_token_type, azure.signinlogs.properties.user_principal_name, azure.signinlogs.properties.resource_display_name, azure.signinlogs.properties.device_detail.device_id, azure.signinlogs.properties.device_detail.browser, azure.signinlogs.properties.authentication_requirement, azure.signinlogs.result_description, azure.signinlogs.properties.conditional_access_status, azure.signinlogs.properties.device_detail.operating_system, azure.signinlogs.result_signature, azure.signinlogs.category, azure.signinlogs.result_type, Esql.azure_signinlogs_properties_app_display_name_lower, Esql.azure_signinlogs_properties_incoming_token_type_lower]?')
0d3d2254-2b4a-11f0-a019-f661ea17fbcc  FAILURE: <class 'detection_rules.esql_errors.EsqlSchemaError'>: BadRequestError(400, 'verification_exception', 'Found 2 problems\nline 5:10: Unknown column [azure.signinlogs.properties.session_id], did you mean any of [azure.signinlogs.properties.user_id, azure.signinlogs.properties.app_id, azure.signinlogs.properties.user_principal_name, azure.graphactivitylogs.properties.app_id, azure.graphactivitylogs.properties.scopes, azure.graphactivitylogs.properties.user_principal_object_id]?\nline 9:10: Unknown column [azure.graphactivitylogs.properties.c_sid], did you mean any of [azure.graphactivitylogs.properties.app_id, azure.graphactivitylogs.properties.scopes, azure.graphactivitylogs.properties.user_principal_object_id, azure.signinlogs.properties.app_id, azure.signinlogs.properties.user_id]?')
375132c6-25d5-11f0-8745-f661ea17fbcd  FAILURE: <class 'detection_rules.esql_errors.EsqlSchemaError'>: BadRequestError(400, 'verification_exception', 'Found 1 problem\nline 14:51: Unknown column [azure.signinlogs.properties.session_id], did you mean any of [azure.signinlogs.properties.resource_id, azure.signinlogs.properties.app_id, azure.signinlogs.properties.user_type, azure.signinlogs.properties.risk_state, azure.signinlogs.properties.is_interactive, azure.signinlogs.properties.user_display_name, azure.signinlogs.properties.app_display_name, azure.signinlogs.properties.incoming_token_type, azure.signinlogs.properties.user_principal_name, azure.signinlogs.properties.resource_display_name, azure.signinlogs.properties.unique_token_identifier, azure.signinlogs.properties.authentication_protocol, azure.signinlogs.properties.device_detail.browser, azure.signinlogs.properties.risk_level_aggregated, azure.signinlogs.properties.authentication_requirement, azure.signinlogs.properties.conditional_access_status, azure.signinlogs.properties.device_detail.operating_system, azure.signinlogs.identity]?')
498e4094-60e7-11f0-8847-f661ea17fbcd  FAILURE: <class 'detection_rules.esql_errors.EsqlSchemaError'>: BadRequestError(400, 'verification_exception', 'Found 1 problem\nline 3:105: Unknown column [azure.auditlogs.properties.target_resources.0.modified_properties.0.new_value]')

How To Test

Checklist

  • Added a label for the type of pr: bug, enhancement, schema, maintenance, Rule: New, Rule: Deprecation, Rule: Tuning, Hunt: New, or Hunt: Tuning so guidelines can be generated
  • Added the meta:rapid-merge label if planning to merge within 24 hours
  • Secret and sensitive material has been managed correctly
  • Automated testing was updated or added to match the most common scenarios
  • Documentation and comments were added for features that require explanation

Contributor checklist

@eric-forte-elastic eric-forte-elastic self-assigned this Sep 25, 2025
@github-actions GitHub Actions
Copy link
Contributor

Enhancement - Guidelines

These guidelines serve as a reminder set of considerations when addressing adding a new schema feature to the code.

Documentation and Context

  • Describe the feature enhancement in detail (alternative solutions, description of the solution, etc.) if not already documented in an issue.
  • Include additional context or screenshots.
  • Ensure the enhancement includes necessary updates to the documentation and versioning.

Code Standards and Practices

  • Code follows established design patterns within the repo and avoids duplication.
  • Ensure that the code is modular and reusable where applicable.

Testing

  • New unit tests have been added to cover the enhancement.
  • Existing unit tests have been updated to reflect the changes.
  • Provide evidence of testing and validating the enhancement (e.g., test logs, screenshots).
  • Validate that any rules affected by the enhancement are correctly updated.
  • Ensure that performance is not negatively impacted by the changes.
  • Verify that any release artifacts are properly generated and tested.
  • Conducted system testing, including fleet, import, and create APIs (e.g., run make test-cli, make test-remote-cli, make test-hunting-cli)

Additional Schema Related Checks

  • Verify that the enhancement works across all relevant environments (e.g., different OS versions).
  • Link to the relevant Kibana PR or issue provided
  • Test export/import flow:
    • Exported detection rule(s) from Kibana to showcase the feature(s)
    • Converted the exported ndjson file(s) to toml in the detection-rules repo
    • Re-exported the toml rule(s) to ndjson and re-imported into Kibana
  • Updated necessary unit tests to accommodate the feature
  • Incorporated a comprehensive test rule in unit tests for full schema coverage
  • Applied min_compat restrictions to limit the feature to a specified minimum stack version
  • Executed all unit tests locally with a test toml rule to confirm passing
  • Included Kibana PR implementer as an optional reviewer for insights on the feature
  • Implemented requisite downgrade functionality
  • Cross-referenced the feature with product documentation for consistency
  • Confirm that the proper version label is applied to the PR patch, minor, major.

@tradebot-elastic
Copy link

tradebot-elastic commented Sep 25, 2025 •
edited
Loading

⛔️ Test failed

Results
  • ❌ AWS Bedrock Guardrails Detected Multiple Violations by a Single User Over a Session (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Unusual High Denied Sensitive Information Policy Blocks Detected (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ AWS S3 Static Site JavaScript File Uploaded (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ AWS Bedrock Detected Multiple Attempts to use Denied Models by a Single User (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Unusual High Denied Topic Blocks Detected (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Unusual High Word Policy Blocks Detected (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Potential Azure OpenAI Model Theft (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Unusual High Confidence Content Filter Blocks Detected (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ AWS S3 Bucket Enumeration or Brute Force (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Potential Denial of Azure OpenAI ML Service (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Potential Abuse of Resources by High Token Count and Large Response Sizes (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ AWS Bedrock Invocations without Guardrails Detected by a Single User Over a Session (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ AWS Bedrock Guardrails Detected Multiple Policy Violations Within a Single Blocked Request (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Azure OpenAI Insecure Output Handling (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta

@tradebot-elastic
Copy link

tradebot-elastic commented Sep 25, 2025 •
edited
Loading

⛔️ Test failed

Results
  • ❌ AWS Bedrock Guardrails Detected Multiple Violations by a Single User Over a Session (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Unusual High Denied Sensitive Information Policy Blocks Detected (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ AWS S3 Static Site JavaScript File Uploaded (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ AWS Bedrock Detected Multiple Attempts to use Denied Models by a Single User (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Unusual High Denied Topic Blocks Detected (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Unusual High Word Policy Blocks Detected (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Potential Azure OpenAI Model Theft (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Unusual High Confidence Content Filter Blocks Detected (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ AWS S3 Bucket Enumeration or Brute Force (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Potential Denial of Azure OpenAI ML Service (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Potential Abuse of Resources by High Token Count and Large Response Sizes (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ AWS Bedrock Invocations without Guardrails Detected by a Single User Over a Session (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ AWS Bedrock Guardrails Detected Multiple Policy Violations Within a Single Blocked Request (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Azure OpenAI Insecure Output Handling (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta

@tradebot-elastic
Copy link

tradebot-elastic commented Sep 25, 2025 •
edited
Loading

⛔️ Test failed

Results
  • ❌ AWS Bedrock Guardrails Detected Multiple Violations by a Single User Over a Session (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Unusual High Denied Sensitive Information Policy Blocks Detected (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ AWS S3 Static Site JavaScript File Uploaded (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ AWS Bedrock Detected Multiple Attempts to use Denied Models by a Single User (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ High Number of Okta Device Token Cookies Generated for Authentication (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Unusual High Denied Topic Blocks Detected (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Okta User Sessions Started from Different Geolocations (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Unusual High Word Policy Blocks Detected (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Potential Azure OpenAI Model Theft (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Unusual High Confidence Content Filter Blocks Detected (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ AWS S3 Bucket Enumeration or Brute Force (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Multiple Okta User Authentication Events with Client Address (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Multiple Okta User Authentication Events with Same Device Token Hash (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Potential Denial of Azure OpenAI ML Service (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Potential Abuse of Resources by High Token Count and Large Response Sizes (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Multiple Device Token Hashes for Single Okta Session (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ AWS Bedrock Invocations without Guardrails Detected by a Single User Over a Session (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ AWS Bedrock Guardrails Detected Multiple Policy Violations Within a Single Blocked Request (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Azure OpenAI Insecure Output Handling (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta

@eric-forte-elastic eric-forte-elastic mentioned this pull request Sep 25, 2025
Open
5 tasks
@tradebot-elastic
Copy link

tradebot-elastic commented Sep 26, 2025 •
edited
Loading

⛔️ Test failed

Results
  • ❌ AWS Bedrock Guardrails Detected Multiple Violations by a Single User Over a Session (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Unusual High Denied Sensitive Information Policy Blocks Detected (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ AWS S3 Static Site JavaScript File Uploaded (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ AWS Bedrock Detected Multiple Attempts to use Denied Models by a Single User (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ High Number of Okta Device Token Cookies Generated for Authentication (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Unusual High Denied Topic Blocks Detected (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Okta User Sessions Started from Different Geolocations (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Unusual High Word Policy Blocks Detected (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Potential Azure OpenAI Model Theft (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Unusual High Confidence Content Filter Blocks Detected (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ AWS S3 Bucket Enumeration or Brute Force (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Multiple Okta User Authentication Events with Client Address (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Multiple Okta User Authentication Events with Same Device Token Hash (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Potential Denial of Azure OpenAI ML Service (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Potential Abuse of Resources by High Token Count and Large Response Sizes (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Multiple Device Token Hashes for Single Okta Session (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ AWS Bedrock Invocations without Guardrails Detected by a Single User Over a Session (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ AWS Bedrock Guardrails Detected Multiple Policy Violations Within a Single Blocked Request (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Azure OpenAI Insecure Output Handling (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta

@eric-forte-elastic eric-forte-elastic marked this pull request as ready for review September 26, 2025 16:25
terrancedejesus
terrancedejesus approved these changes Sep 26, 2025
View reviewed changes
Copy link
Contributor

@terrancedejesus terrancedejesus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Glad to see validation is working!

@botelastic botelastic bot added the Integration: Okta okta related rules label Sep 26, 2025
eric-forte-elastic
eric-forte-elastic commented Sep 26, 2025
View reviewed changes
@terrancedejesus
fixed incorrect field in non-ecs-schema.json; changed logs-azure.sign…
99acbcd 
…inlogs* sightings to logs-azure.signinlogs-*
@botelastic botelastic bot added the Integration: Azure azure related rules label Sep 26, 2025
@tradebot-elastic
Copy link

tradebot-elastic commented Sep 26, 2025 •
edited
Loading

⛔️ Test failed

Results
  • ❌ AWS Bedrock Guardrails Detected Multiple Violations by a Single User Over a Session (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Unusual High Denied Sensitive Information Policy Blocks Detected (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ AWS S3 Static Site JavaScript File Uploaded (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ AWS Bedrock Detected Multiple Attempts to use Denied Models by a Single User (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ High Number of Okta Device Token Cookies Generated for Authentication (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Unusual High Denied Topic Blocks Detected (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Microsoft Entra ID Exccessive Account Lockouts Detected (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Okta User Sessions Started from Different Geolocations (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Unusual High Word Policy Blocks Detected (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Microsoft 365 Brute Force via Entra ID Sign-Ins (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Suspicious Microsoft OAuth Flow via Auth Broker to DRS (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Microsoft Entra ID High Risk Sign-in (kuery)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Microsoft Entra ID MFA TOTP Brute Force Attempts (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Potential Azure OpenAI Model Theft (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Unusual High Confidence Content Filter Blocks Detected (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ AWS S3 Bucket Enumeration or Brute Force (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Multiple Okta User Authentication Events with Client Address (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Multiple Okta User Authentication Events with Same Device Token Hash (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Potential Denial of Azure OpenAI ML Service (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Potential Abuse of Resources by High Token Count and Large Response Sizes (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Multiple Device Token Hashes for Single Okta Session (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Microsoft Entra ID Sign-In Brute Force Activity (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Microsoft Entra ID Concurrent Sign-Ins with Suspicious Properties (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ AWS Bedrock Invocations without Guardrails Detected by a Single User Over a Session (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ AWS Bedrock Guardrails Detected Multiple Policy Violations Within a Single Blocked Request (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Azure OpenAI Insecure Output Handling (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta

Mikaayenson
Mikaayenson approved these changes Sep 26, 2025
View reviewed changes
Copy link
Contributor

@Mikaayenson Mikaayenson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🏅 Great work. Roughly 34% of our esql rules improved.

@tradebot-elastic
Copy link

tradebot-elastic commented Sep 26, 2025 •
edited
Loading

⛔️ Test failed

Results
  • ❌ AWS Bedrock Guardrails Detected Multiple Violations by a Single User Over a Session (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Unusual High Denied Sensitive Information Policy Blocks Detected (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ AWS S3 Static Site JavaScript File Uploaded (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ AWS Bedrock Detected Multiple Attempts to use Denied Models by a Single User (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ High Number of Okta Device Token Cookies Generated for Authentication (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Unusual High Denied Topic Blocks Detected (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Microsoft Entra ID Exccessive Account Lockouts Detected (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Okta User Sessions Started from Different Geolocations (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Unusual High Word Policy Blocks Detected (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Microsoft 365 Brute Force via Entra ID Sign-Ins (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Suspicious Microsoft OAuth Flow via Auth Broker to DRS (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Microsoft Entra ID High Risk Sign-in (kuery)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Microsoft Entra ID MFA TOTP Brute Force Attempts (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Potential Azure OpenAI Model Theft (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Unusual High Confidence Content Filter Blocks Detected (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ AWS S3 Bucket Enumeration or Brute Force (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Multiple Okta User Authentication Events with Client Address (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Multiple Okta User Authentication Events with Same Device Token Hash (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Potential Denial of Azure OpenAI ML Service (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Potential Abuse of Resources by High Token Count and Large Response Sizes (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Multiple Device Token Hashes for Single Okta Session (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Microsoft Entra ID Sign-In Brute Force Activity (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Microsoft Entra ID Concurrent Sign-Ins with Suspicious Properties (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ AWS Bedrock Invocations without Guardrails Detected by a Single User Over a Session (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ AWS Bedrock Guardrails Detected Multiple Policy Violations Within a Single Blocked Request (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Azure OpenAI Insecure Output Handling (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta

@tradebot-elastic
Copy link

tradebot-elastic commented Sep 26, 2025 •
edited
Loading

⛔️ Test failed

Results
  • ❌ AWS Bedrock Guardrails Detected Multiple Violations by a Single User Over a Session (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Unusual High Denied Sensitive Information Policy Blocks Detected (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ AWS S3 Static Site JavaScript File Uploaded (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ AWS Bedrock Detected Multiple Attempts to use Denied Models by a Single User (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ High Number of Okta Device Token Cookies Generated for Authentication (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Unusual High Denied Topic Blocks Detected (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Microsoft Entra ID Exccessive Account Lockouts Detected (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Okta User Sessions Started from Different Geolocations (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Unusual High Word Policy Blocks Detected (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Microsoft 365 Brute Force via Entra ID Sign-Ins (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Suspicious Microsoft OAuth Flow via Auth Broker to DRS (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Microsoft Entra ID High Risk Sign-in (kuery)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Microsoft Entra ID MFA TOTP Brute Force Attempts (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Potential Azure OpenAI Model Theft (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Unusual High Confidence Content Filter Blocks Detected (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ AWS S3 Bucket Enumeration or Brute Force (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Multiple Okta User Authentication Events with Client Address (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Multiple Okta User Authentication Events with Same Device Token Hash (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Potential Denial of Azure OpenAI ML Service (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Potential Abuse of Resources by High Token Count and Large Response Sizes (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Multiple Device Token Hashes for Single Okta Session (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Microsoft Entra ID Sign-In Brute Force Activity (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Microsoft Entra ID Concurrent Sign-Ins with Suspicious Properties (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ AWS Bedrock Invocations without Guardrails Detected by a Single User Over a Session (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ AWS Bedrock Guardrails Detected Multiple Policy Violations Within a Single Blocked Request (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Azure OpenAI Insecure Output Handling (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta

@tradebot-elastic
Copy link

tradebot-elastic commented Sep 26, 2025 •
edited
Loading

⛔️ Test failed

Results
  • ❌ AWS Bedrock Guardrails Detected Multiple Violations by a Single User Over a Session (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Unusual High Denied Sensitive Information Policy Blocks Detected (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ AWS S3 Static Site JavaScript File Uploaded (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ AWS Bedrock Detected Multiple Attempts to use Denied Models by a Single User (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ High Number of Okta Device Token Cookies Generated for Authentication (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Unusual High Denied Topic Blocks Detected (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Microsoft Entra ID Exccessive Account Lockouts Detected (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Okta User Sessions Started from Different Geolocations (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Unusual High Word Policy Blocks Detected (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Microsoft 365 Brute Force via Entra ID Sign-Ins (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Suspicious Microsoft OAuth Flow via Auth Broker to DRS (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Microsoft Entra ID High Risk Sign-in (kuery)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Microsoft Entra ID MFA TOTP Brute Force Attempts (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Potential Azure OpenAI Model Theft (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Unusual High Confidence Content Filter Blocks Detected (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ AWS S3 Bucket Enumeration or Brute Force (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Multiple Okta User Authentication Events with Client Address (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Multiple Okta User Authentication Events with Same Device Token Hash (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Potential Denial of Azure OpenAI ML Service (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Potential Abuse of Resources by High Token Count and Large Response Sizes (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Multiple Device Token Hashes for Single Okta Session (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Microsoft Entra ID Sign-In Brute Force Activity (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Microsoft Entra ID Concurrent Sign-Ins with Suspicious Properties (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ AWS Bedrock Invocations without Guardrails Detected by a Single User Over a Session (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ AWS Bedrock Guardrails Detected Multiple Policy Violations Within a Single Blocked Request (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Azure OpenAI Insecure Output Handling (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta

terrancedejesus
terrancedejesus reviewed Sep 26, 2025
View reviewed changes
terrancedejesus
terrancedejesus reviewed Sep 26, 2025
View reviewed changes
terrancedejesus
terrancedejesus reviewed Sep 26, 2025
View reviewed changes
terrancedejesus
terrancedejesus reviewed Sep 26, 2025
View reviewed changes
@tradebot-elastic
Copy link

tradebot-elastic commented Sep 26, 2025 •
edited
Loading

⛔️ Test failed

Results
  • ❌ AWS Bedrock Guardrails Detected Multiple Violations by a Single User Over a Session (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Unusual High Denied Sensitive Information Policy Blocks Detected (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ AWS S3 Static Site JavaScript File Uploaded (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ AWS Bedrock Detected Multiple Attempts to use Denied Models by a Single User (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ High Number of Okta Device Token Cookies Generated for Authentication (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Unusual High Denied Topic Blocks Detected (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Microsoft Entra ID Exccessive Account Lockouts Detected (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Okta User Sessions Started from Different Geolocations (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Unusual High Word Policy Blocks Detected (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Microsoft 365 Brute Force via Entra ID Sign-Ins (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Suspicious Microsoft OAuth Flow via Auth Broker to DRS (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Microsoft Entra ID High Risk Sign-in (kuery)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Microsoft Entra ID MFA TOTP Brute Force Attempts (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Potential Azure OpenAI Model Theft (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Unusual High Confidence Content Filter Blocks Detected (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ AWS S3 Bucket Enumeration or Brute Force (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Multiple Okta User Authentication Events with Client Address (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Multiple Okta User Authentication Events with Same Device Token Hash (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Potential Denial of Azure OpenAI ML Service (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Potential Abuse of Resources by High Token Count and Large Response Sizes (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Excessive Secret or Key Retrieval from Azure Key Vault (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Multiple Device Token Hashes for Single Okta Session (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Microsoft Entra ID Sign-In Brute Force Activity (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Microsoft Entra ID Concurrent Sign-Ins with Suspicious Properties (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ AWS Bedrock Invocations without Guardrails Detected by a Single User Over a Session (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ AWS Bedrock Guardrails Detected Multiple Policy Violations Within a Single Blocked Request (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Azure OpenAI Insecure Output Handling (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta

@tradebot-elastic
Copy link

tradebot-elastic commented Sep 26, 2025 •
edited
Loading

⛔️ Test failed

Results
  • ❌ AWS Bedrock Guardrails Detected Multiple Violations by a Single User Over a Session (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Unusual High Denied Sensitive Information Policy Blocks Detected (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ AWS S3 Static Site JavaScript File Uploaded (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ AWS Bedrock Detected Multiple Attempts to use Denied Models by a Single User (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ High Number of Okta Device Token Cookies Generated for Authentication (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Unusual High Denied Topic Blocks Detected (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Microsoft Entra ID Exccessive Account Lockouts Detected (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Okta User Sessions Started from Different Geolocations (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Unusual High Word Policy Blocks Detected (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Microsoft 365 Brute Force via Entra ID Sign-Ins (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Suspicious Microsoft OAuth Flow via Auth Broker to DRS (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Microsoft Entra ID High Risk Sign-in (kuery)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Microsoft Entra ID MFA TOTP Brute Force Attempts (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Potential Azure OpenAI Model Theft (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Unusual High Confidence Content Filter Blocks Detected (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ AWS S3 Bucket Enumeration or Brute Force (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Multiple Okta User Authentication Events with Client Address (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Multiple Okta User Authentication Events with Same Device Token Hash (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Potential Denial of Azure OpenAI ML Service (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Potential Abuse of Resources by High Token Count and Large Response Sizes (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Excessive Secret or Key Retrieval from Azure Key Vault (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Multiple Device Token Hashes for Single Okta Session (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Microsoft Entra ID Sign-In Brute Force Activity (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Microsoft Entra ID Concurrent Sign-Ins with Suspicious Properties (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ AWS Bedrock Invocations without Guardrails Detected by a Single User Over a Session (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ AWS Bedrock Guardrails Detected Multiple Policy Violations Within a Single Blocked Request (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Azure OpenAI Insecure Output Handling (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta

@eric-forte-elastic
Copy link
Contributor Author

eric-forte-elastic commented Sep 27, 2025 •
edited
Loading

Note o365.audit.AuthenticationType is present in the schema files and is released as of 2.21.1 soon as it has been added to the integration. However, the current logic of the ES|QL validation PR is to use find_least_compatible_version which will return 2.11.0 even for 9.0.0^ stacks. Wheras this rule depends on package version 2.21.1.

@eric-forte-elastic
Copy link
Contributor Author

Note o365.audit.AuthenticationType is present in the schema files and is released as of 2.21.1 soon as it has been added to the integration. However, the current logic of the ES|QL validation PR is to use find_least_compatible_version which will return 2.11.0 even for 9.0.0^ stacks. Wheras this rule depends on package version 2.21.1.

Addressed in 5ae9937 where we switched to load latest schemas which then pull this field and passes validation with no action needed here.

@tradebot-elastic
Copy link

tradebot-elastic commented Sep 30, 2025 •
edited
Loading

⛔️ Test failed

Results
  • ❌ AWS Bedrock Guardrails Detected Multiple Violations by a Single User Over a Session (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Unusual High Denied Sensitive Information Policy Blocks Detected (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ AWS S3 Static Site JavaScript File Uploaded (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ AWS Bedrock Detected Multiple Attempts to use Denied Models by a Single User (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ High Number of Okta Device Token Cookies Generated for Authentication (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Unusual High Denied Topic Blocks Detected (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Microsoft Entra ID Exccessive Account Lockouts Detected (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Okta User Sessions Started from Different Geolocations (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Unusual High Word Policy Blocks Detected (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Microsoft 365 Brute Force via Entra ID Sign-Ins (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Suspicious Microsoft OAuth Flow via Auth Broker to DRS (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Microsoft Entra ID High Risk Sign-in (kuery)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Microsoft Entra ID MFA TOTP Brute Force Attempts (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Potential Azure OpenAI Model Theft (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Unusual High Confidence Content Filter Blocks Detected (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ AWS S3 Bucket Enumeration or Brute Force (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Multiple Okta User Authentication Events with Client Address (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Multiple Okta User Authentication Events with Same Device Token Hash (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Potential Denial of Azure OpenAI ML Service (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Potential Abuse of Resources by High Token Count and Large Response Sizes (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Excessive Secret or Key Retrieval from Azure Key Vault (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Multiple Device Token Hashes for Single Okta Session (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Microsoft Entra ID Sign-In Brute Force Activity (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Microsoft Entra ID Concurrent Sign-Ins with Suspicious Properties (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ AWS Bedrock Invocations without Guardrails Detected by a Single User Over a Session (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ AWS Bedrock Guardrails Detected Multiple Policy Violations Within a Single Blocked Request (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Azure OpenAI Insecure Output Handling (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta

@tradebot-elastic
Copy link

tradebot-elastic commented Sep 30, 2025 •
edited
Loading

⛔️ Test failed

Results
  • ❌ AWS Bedrock Guardrails Detected Multiple Violations by a Single User Over a Session (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Unusual High Denied Sensitive Information Policy Blocks Detected (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ AWS S3 Static Site JavaScript File Uploaded (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ AWS Bedrock Detected Multiple Attempts to use Denied Models by a Single User (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ High Number of Okta Device Token Cookies Generated for Authentication (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Unusual High Denied Topic Blocks Detected (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Microsoft Entra ID Exccessive Account Lockouts Detected (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Okta User Sessions Started from Different Geolocations (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Unusual High Word Policy Blocks Detected (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Microsoft 365 Brute Force via Entra ID Sign-Ins (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Suspicious Microsoft OAuth Flow via Auth Broker to DRS (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Microsoft Entra ID High Risk Sign-in (kuery)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Microsoft Entra ID MFA TOTP Brute Force Attempts (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Potential Azure OpenAI Model Theft (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Unusual High Confidence Content Filter Blocks Detected (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ AWS S3 Bucket Enumeration or Brute Force (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Multiple Okta User Authentication Events with Client Address (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Multiple Okta User Authentication Events with Same Device Token Hash (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Potential Denial of Azure OpenAI ML Service (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Potential Abuse of Resources by High Token Count and Large Response Sizes (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Excessive Secret or Key Retrieval from Azure Key Vault (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Multiple Device Token Hashes for Single Okta Session (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Microsoft Entra ID Sign-In Brute Force Activity (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Microsoft Entra ID Concurrent Sign-Ins with Suspicious Properties (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ AWS Bedrock Invocations without Guardrails Detected by a Single User Over a Session (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ AWS Bedrock Guardrails Detected Multiple Policy Violations Within a Single Blocked Request (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Azure OpenAI Insecure Output Handling (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta

@eric-forte-elastic
Copy link
Contributor Author

eric-forte-elastic commented Sep 30, 2025 •
edited
Loading

Tested validation against all stack versions via updated code in a368516.

>  git checkout origin/update_esql_rules_from_validation -- detection_rules/etc/non-ecs-schema.json
>  git checkout origin/update_esql_rules_from_validation -- rules

> python -m detection_rules dev test esql-remote-validation --verbosity 1
....
894b7cc9-040b-427c-aca5-36b40d3667bf: Validating 894b7cc9-040b-427c-aca5-36b40d3667bf against 9.2.0 stack
894b7cc9-040b-427c-aca5-36b40d3667bf: Validating 894b7cc9-040b-427c-aca5-36b40d3667bf against 8.18.0 stack
894b7cc9-040b-427c-aca5-36b40d3667bf: Validating 894b7cc9-040b-427c-aca5-36b40d3667bf against 8.19.0 stack
894b7cc9-040b-427c-aca5-36b40d3667bf: Validating 894b7cc9-040b-427c-aca5-36b40d3667bf against 9.0.0 stack
894b7cc9-040b-427c-aca5-36b40d3667bf: Validating 894b7cc9-040b-427c-aca5-36b40d3667bf against 9.1.0 stack
Total rules: 75
Failed rules: 0
Failed rules written to failed_rules.log

No ESQL validation failures 🚀 Good to merge.

@eric-forte-elastic eric-forte-elastic merged commit 7410ec7 into main Sep 30, 2025
15 checks passed
@eric-forte-elastic eric-forte-elastic deleted the update_esql_rules_from_validation branch September 30, 2025 04:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@imays11 imays11 imays11 left review comments

@terrancedejesus terrancedejesus terrancedejesus approved these changes

@Mikaayenson Mikaayenson Mikaayenson approved these changes

@traut traut Awaiting requested review from traut traut is a code owner

Assignees

@eric-forte-elastic eric-forte-elastic

Labels

backport: auto Domain: Cloud Integration: AWS AWS related rules Integration: AWS-Bedrock Integration: Azure azure related rules Integration: Microsoft 365 Integration: Okta okta related rules patch Rule: Tuning tweaking or tuning an existing rule schema

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@eric-forte-elastic @tradebot-elastic @Mikaayenson @imays11 @terrancedejesus