Skip to content

[Rule Tuning] Update PowerShell ES|QL Rules KEEP Condition #5391

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open

[Rule Tuning] Update PowerShell ES|QL Rules KEEP Condition #5391

wants to merge 4 commits into from
+32 −39

Conversation

@w0rk3r
Copy link
Contributor

@w0rk3r w0rk3r commented Dec 1, 2025

Issues

Small tuning included that resolves #5263

Summary

Updates the KEEP condition of the PowerShell ES|QL rules so we start receiving telemetry on the new PowerShell fields.

@w0rk3r w0rk3r self-assigned this Dec 1, 2025
@w0rk3r w0rk3r added Rule: Tuning tweaking or tuning an existing rule OS: Windows windows related rules Domain: Endpoint backport: auto labels Dec 1, 2025
@botelastic botelastic bot added the bbr Building Block Rules label Dec 1, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Dec 1, 2025

Rule: Tuning - Guidelines

These guidelines serve as a reminder set of considerations when tuning an existing rule.

Documentation and Context

  • Detailed description of the suggested changes.
  • Provide example JSON data or screenshots.
  • Provide evidence of reducing benign events mistakenly identified as threats (False Positives).
  • Provide evidence of enhancing detection of true threats that were previously missed (False Negatives).
  • Provide evidence of optimizing resource consumption and execution time of detection rules (Performance).
  • Provide evidence of specific environment factors influencing customized rule tuning (Contextual Tuning).
  • Provide evidence of improvements made by modifying sensitivity by changing alert triggering thresholds (Threshold Adjustments).
  • Provide evidence of refining rules to better detect deviations from typical behavior (Behavioral Tuning).
  • Provide evidence of improvements of adjusting rules based on time-based patterns (Temporal Tuning).
  • Provide reasoning of adjusting priority or severity levels of alerts (Severity Tuning).
  • Provide evidence of improving quality integrity of our data used by detection rules (Data Quality).
  • Ensure the tuning includes necessary updates to the release documentation and versioning.

Rule Metadata Checks

  • updated_date matches the date of tuning PR merged.
  • min_stack_version should support the widest stack versions.
  • name and description should be descriptive and not include typos.
  • query should be inclusive, not overly exclusive. Review to ensure the original intent of the rule is maintained.

Testing and Validation

  • Validate that the tuned rule's performance is satisfactory and does not negatively impact the stack.
  • Ensure that the tuned rule has a low false positive rate.

@tradebot-elastic
Copy link

tradebot-elastic commented Dec 1, 2025 •
edited
Loading

⛔️ Test failed

Results
  • ❌ Potential PowerShell Obfuscation via Concatenated Dynamic Command Invocation (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Potential PowerShell Obfuscation via Invalid Escape Sequences (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Potential PowerShell Obfuscation via Special Character Overuse (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Potential PowerShell Obfuscation via Character Array Reconstruction (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ PowerShell Obfuscation via Negative Index String Reversal (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Dynamic IEX Reconstruction via Method String Access (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Potential Dynamic IEX Reconstruction via Environment Variables (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Potential PowerShell Obfuscation via Backtick-Escaped Variable Expansion (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Potential PowerShell Obfuscation via String Reordering (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Potential PowerShell Obfuscation via Reverse Keywords (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Potential PowerShell Obfuscation via String Concatenation (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Potential PowerShell Obfuscation via High Numeric Character Proportion (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta

@tradebot-elastic
Copy link

tradebot-elastic commented Dec 1, 2025 •
edited
Loading

⛔️ Test failed

Results
  • ❌ Potential PowerShell Obfuscation via Concatenated Dynamic Command Invocation (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Potential PowerShell Obfuscation via Invalid Escape Sequences (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Potential PowerShell Obfuscation via Special Character Overuse (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Potential PowerShell Obfuscation via Character Array Reconstruction (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ PowerShell Obfuscation via Negative Index String Reversal (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Dynamic IEX Reconstruction via Method String Access (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Potential Dynamic IEX Reconstruction via Environment Variables (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Potential PowerShell Obfuscation via Backtick-Escaped Variable Expansion (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Potential PowerShell Obfuscation via String Reordering (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Potential PowerShell Obfuscation via Reverse Keywords (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Potential PowerShell Obfuscation via String Concatenation (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Potential PowerShell Obfuscation via High Numeric Character Proportion (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta

Samirbous
Samirbous reviewed Dec 2, 2025
View reviewed changes
rules/windows/defense_evasion_posh_obfuscation_backtick.toml Outdated
Esql.script_block_tmp,
powershell.file.script_block_text,
powershell.file.script_block_id,
powershell.file.script_block_entropy_bits,
Copy link
Contributor

@Samirbous Samirbous Dec 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what if the customer is using ps logs without the custom ingestion pipeline u added for entropy? the query will error ? if so I would suggest create one dedicated for ps with entropy enrichment to experiment with performance and adoption ?

Copy link
Contributor Author

@w0rk3r w0rk3r Dec 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As long as customers are using the official integration to collect the logs (the one we support in the query), they will have the field, this is like any addition of fields done by integrations, as for performance, it is done in the ingest time, so there is no difference for the rules.

We can also do powershell.file.*, so we are sure to not fail the field verification

Copy link
Contributor Author

@w0rk3r w0rk3r Dec 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(It is available since Windows Integration v3.1.3, published a month ago)

Copy link
Contributor Author

@w0rk3r w0rk3r Dec 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

16db378

@tradebot-elastic
Copy link

tradebot-elastic commented Dec 2, 2025 •
edited
Loading

⛔️ Test failed

Results
  • ❌ Potential PowerShell Obfuscation via Concatenated Dynamic Command Invocation (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Potential PowerShell Obfuscation via Invalid Escape Sequences (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Potential PowerShell Obfuscation via Special Character Overuse (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Potential PowerShell Obfuscation via Character Array Reconstruction (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ PowerShell Obfuscation via Negative Index String Reversal (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Dynamic IEX Reconstruction via Method String Access (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Potential Dynamic IEX Reconstruction via Environment Variables (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Potential PowerShell Obfuscation via Backtick-Escaped Variable Expansion (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Potential PowerShell Obfuscation via String Reordering (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Potential PowerShell Obfuscation via Reverse Keywords (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Potential PowerShell Obfuscation via String Concatenation (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Potential PowerShell Obfuscation via High Numeric Character Proportion (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta

@w0rk3r w0rk3r requested a review from Samirbous December 2, 2025 12:07
@tradebot-elastic
Copy link

tradebot-elastic commented Dec 2, 2025 •
edited
Loading

⛔️ Test failed

Results
  • ❌ Potential PowerShell Obfuscation via Concatenated Dynamic Command Invocation (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Potential PowerShell Obfuscation via Invalid Escape Sequences (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Potential PowerShell Obfuscation via Special Character Overuse (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Potential PowerShell Obfuscation via Character Array Reconstruction (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ PowerShell Obfuscation via Negative Index String Reversal (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Dynamic IEX Reconstruction via Method String Access (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Potential Dynamic IEX Reconstruction via Environment Variables (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Potential PowerShell Obfuscation via Backtick-Escaped Variable Expansion (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Potential PowerShell Obfuscation via String Reordering (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Potential PowerShell Obfuscation via Reverse Keywords (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Potential PowerShell Obfuscation via String Concatenation (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta
  • ❌ Potential PowerShell Obfuscation via High Numeric Character Proportion (esql)
    • coverage_issue: no_rta
    • stack_validation_failed: no_rta

@imays11
Copy link
Contributor

imays11 commented Dec 4, 2025

since you're doing a mass update on ESQL rule keep fields, could you also add data_stream.namespace since this field has been requested by customer for ESQL rules.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@shashank-elastic shashank-elastic shashank-elastic approved these changes

@Mikaayenson Mikaayenson Awaiting requested review from Mikaayenson

@Aegrah Aegrah Awaiting requested review from Aegrah

@terrancedejesus terrancedejesus Awaiting requested review from terrancedejesus

@Samirbous Samirbous Awaiting requested review from Samirbous

At least 2 approving reviews are required to merge this pull request.

Assignees

@w0rk3r w0rk3r

Labels

backport: auto bbr Building Block Rules Domain: Endpoint OS: Windows windows related rules Rule: Tuning tweaking or tuning an existing rule

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[Rule Tuning] Potential PowerShell Obfuscation via High Special Character Proportion

6 participants

@w0rk3r @tradebot-elastic @imays11 @Samirbous @shashank-elastic