[DE] - Detection Engine backlog overview

[yctercero]

Code/Ownership

Beta Give feedback

1. [Security Solution] Domain-based architecture of the Detection Engine #138600
   Team: SecuritySolution Team:Detection Engine Team:Detection Rule Management Team:Detections and Resp discuss epic technical debt +7

Detection Alerts

Consider next

Beta Give feedback

1. [Security Solution][Alerts] Warn users when rule interval is larger than time range searched #154963
   Feature:Detection Alerts Team:Detection Engine consider-next enhancement sdh-linked v8.8.0 +6
2. [Security Solution] Truncated error messages during rule execution #147918
   Feature:Detection Alerts Feature:Rule Monitoring Team: SecuritySolution Team:Detection Engine Team:Detection Rule Management Team:Detections and Resp bug consider-next impact:medium +9
3. [Security Solution][Alerts] Decouple gap detection from additional lookback #138933
   Feature:Detection Alerts Team: SecuritySolution Team:Detection Engine consider-next discuss enhancement sdh-linked technical debt +8
   
4. [Security Solution] Threshold rules incorrectly populate kibana.alert.original_time when the rule groups by @timestamp, effecting Timeline investigations #144467
   Feature:Threshold Rule Team: SecuritySolution Team:Detection Engine Team:Detections and Resp bug consider-next impact:medium sdh-linked +8
   
5. [Security Solution] When investigating Threshold alerts with a timestamp override, the timeline's from date should be based on the value of the override #144473
   Feature:Threshold Rule Team: SecuritySolution Team:Detection Engine Team:Detections and Resp bug impact:medium sdh-linked +7
   

Enhancement - Auditing

Beta Give feedback

1. Security APP, see who change the signal.status #95621
   8.11 candidate Feature:Detection Alerts Team: SecuritySolution Team:Detection Engine Team:Detections and Resp Theme: TBD Theme: rac needs design +8
   
2. Assign SIEM Signals to a user #76627
   8.11 candidate Feature:Detection Alerts Team: SecuritySolution Team:Detection Engine Team:Detections and Resp Team:SIEM Theme: TBD enhancement needs design +9
3. https://github.com/elastic/security-team/issues/2504
4. [Security Feature][Feature Request] Add the active user to the signal event when marking an alert as closed, open, or acknowledged #123444
   8.11 candidate Feature:Detection Alerts Team: SecuritySolution Team:Detection Engine Team:Detections and Resp Team:ResponseOps Theme: rac enhancement +8
   

Enhancement - alerts index

Beta Give feedback

1. [Security Solution][Alerts] Detection alerts indices are missing data_stream ECS field mappings #129946
   Feature:Detection Alerts Team:ResponseOps bug consider-next impact:high sdh-linked +6
2. [Security Solution][Alerts] Threshold rule alert history should be invalidated if rule is updated #129825
   Feature:Threshold Rule Team: SecuritySolution Team:Detection Engine Team:Detections and Resp enhancement good first issue impact:medium +7
   
3. [Security Solution] data_stream.namespace field mapping missing from .internal.alerts-security.alerts-default indices #156060
   Team: SecuritySolution Team:Detection Engine Team:Detections and Resp bug +4
4. [Security Solution][Detection Alerts] Create common type for dot notation fields #122405
   Team: SecuritySolution Team:Detection Engine Team:Detections and Resp technical debt +4
   
5. [Security Solution][Alerts] Use dynamic mappings to create smaller templates & mappings #100884
   Team: SecuritySolution Team:Detection Engine Team:Detections and Resp Team:SIEM Theme: rac discuss +6

Enahancement

Beta Give feedback

1. Personalization of the Highlighted fields in an Alert Rule #131778
   Feature:Detection Alerts Team: SecuritySolution Team:Detection Engine Team:Detection Rule Management Team:Detections and Resp enhancement +6
   
2. [Security Solution][Detections] Supporting alerts-on-alerts Pre-packaged Rules #124756
   Feature:Detection Rules Team: SecuritySolution Team:Detection Engine Team:Detections and Resp Theme: simp_prot_mgmt enhancement +6
   
3. [RAC][Discuss] Support for user-defined field mappings in .alerts indices #103777
   Feature:Detection Alerts Team: SecuritySolution Team:Detection Engine Team:Detections and Resp Team:ResponseOps Theme: alert_triage Theme: rac consider-next sdh-linked +9
4. [Security Solution] Elastic or Custom Jobs filters are not displaying under ML job dropdown #145221
   Feature:Detection Alerts Team: SecuritySolution Team:Detection Engine Team:Detections and Resp enhancement +5
5. [Security Solutions] Detection Alerts - Want To Only See that Alert #86615
   Team: SecuritySolution Team:Detection Engine Team:Detections and Resp +3
6. [Security Solution][Detections] Format Custom Machine Learning Job Field to Time Format in Alert View #90564
   Feature:Detection Alerts Feature:Detection Rules Feature:ML Rule Team:Detection Engine Team:Detections and Resp enhancement sec-specialists +7
7. [Security Solution][Detections]First-class treatment of ML Detection Alert fields #90877
   Feature:Detection Alerts Feature:ML Rule Team: SecuritySolution Team:Detection Engine Team:Detections and Resp enhancement +6
8. SIEM Signals Index: Unique "incident id" or "signal id" per signal fired instead of just _id for doc record. #75129
   Feature:Detection Rules Team: SecuritySolution Team:Detection Engine Team:Detections and Resp Team:SIEM enhancement +6
9. Customized Security Alerts dashboard #134623
   Team:Detection Engine +1
10. [SecuritySolution][Detections] Provide finer grained feature controls between editing rules and modifying alerts #86235
    Feature:Detection Alerts Feature:Detection Alerts/Rules RBAC Feature:Detection Rules Team: SecuritySolution Team:Detection Engine Team:Detections and Resp enhancement +7
11. Scripted Fields are not supported by SIEM Detections Rules #97778
    Feature:Detection Alerts Team: SecuritySolution Team:Detection Engine Team:Detections and Resp Team:SIEM Team:Security Solution Platform enhancement impact:medium +8

Bugs

Beta Give feedback

1. [Security Solution] Detections fail when an index is missing a primary shard #101990
   Feature:Detection Alerts Team: SecuritySolution Team:Detection Engine Team:Detections and Resp bug enhancement impact:medium +7
2. [Security Solution][Sourcerer] Delay in Alerts Security Data View initialization after Alerts index is created #131427
   8.4 candidate Feature:Data Views Feature:Detection Alerts Team: SecuritySolution Team:Detection Engine Team:Detections and Resp bug +7
3. [Security Solution][Detections] Inconsistent handling of gap detection and max signals #100181
   8.15 candidate Feature:Detection Alerts Team: CTI Team: SecuritySolution Team:Detection Engine Team:Detections and Resp bug impact:low +8
4. [Security Solution] Error displayed when apply the sorting on threat.enrichments.matched.* fields under alerts table. #130951
   Feature:Detection Alerts Team: CTI Team: SecuritySolution Team:Detection Engine Team:Detections and Resp bug impact:medium +7

7.x Issues

Beta Give feedback

1. [Security Solution][Alerts] Detection alerts indices are missing data_stream ECS field mappings #129946
   Feature:Detection Alerts Team:ResponseOps bug consider-next impact:high sdh-linked +6
2. [Security Solution]Incorrect Date showing under trends data over mouser hover on seting last year timestamp #102628
   Feature:Detection Alerts Team: SecuritySolution Team:Detection Engine Team:Detections and Resp Theme: rac bug good first issue impact:low +8
   
3. [Security Solution][Alerts] Add tests for changes in the schema #110798
   Team: SecuritySolution Team:Detection Engine Team:Detections and Resp Theme: rac technical debt +5
4. [Security Solution][Detection Alerts] Refresh isn't synced with workflow status updates in Timeline #112011
   Feature:Detection Alerts Team: SecuritySolution Team:Detection Engine Team:Detections and Resp bug impact:low +6
5. [Security Solution][Detections] Alerts mappings conflict with experimental ECS threat fields #100510
   Feature:Indicator Match Rule Feature:ecs Team: SecuritySolution Team:Detection Engine Team:Detections and Resp bug +6

Detection Alerts

Detection Engine

Consider Next

Beta Give feedback

1. [Security Solution] Failed rule execution shows 0 for index duration even though alerts were written #155672
   Feature:Rule Monitoring Team: SecuritySolution Team:Detection Engine Team:Detection Rule Management Team:Detections and Resp bug consider-next impact:medium +8
2. Custom rule in Elastic Security throws RuleDataWriteDisabledError error when run #148460
   Team: SecuritySolution Team:Detection Engine Team:Detections and Resp bug consider-next impact:high sdh-linked +7
3. [Security Solution][Alerts] Clean up searchAfterBulkCreate function #140142
   Team:Detection Engine consider-next technical debt +3
4. [Security Solution] Make Rule Execution Errors more user friendly and actionable #128340
   Feature:Detection Rules Feature:Rule Monitoring Team: SecuritySolution Team:Detection Engine Team:Detection Rule Management Team:Detections and Resp consider-next enhancement +8
5. [Security Solution] [Detections] [Platform] Expose _tier in the Security Solution #130517
   8.10 candidate 8.15 candidate Feature:Detection Rules Feature:Hosts Feature:Network Feature:SecurityOverview Feature:Timeline Team: SecuritySolution Team:Detection Engine consider-next enhancement +11
6. [Security Solution] Give user the ability to cancel a long running rule execution #93740
   Feature:Rule Management Team: SecuritySolution Team:Detection Engine Team:Detections and Resp Theme: simp_prot_mgmt consider-next dependencies enhancement needs design sdh-linked +10
7. [SecuritySolution] DELETE index endpoint's logic has implicitly changed #158308
   8.8 candidate Team: SecuritySolution Team:Detection Engine needs_docs +4

Bugs

Beta Give feedback

1. [Security Solution] Detection Rules' max_signals parameter is not respected beyond 1000 #157313
   Team: SecuritySolution Team:Detection Engine Team:Detections and Resp bug consider-next v8.8.0 +6
   
2. [Security Solution] ML rule can miss anomaly documents if its interval/lookback is too short #158152
   Feature:ML Rule Team: SecuritySolution Team:Detection Engine Team:Detections and Resp bug impact:medium sdh-linked +7
3. [Security Solution] Rule incorrectly reports gap when modifying interval while disabled and then re-enabling #155671
   8.15 candidate Feature:Gap Detection/Remediation Team: SecuritySolution Team:Detection Engine Team:Detections and Resp bug impact:low +7
4. [Security Solution] Incorrect validation message is displaying for the Fields #137153
   Team: SecuritySolution Team:Detection Engine Team:Detections and Resp bug impact:low +5
5. [Security Solution][Detections] Change in set-value behavior affecting Security ML Detection Rule #113645
   Feature:ML Rule Team: SecuritySolution Team:Detection Engine Team:Detections and Resp bug impact:medium sdh-linked +7
   

Tech Debt

Beta Give feedback

1. [Detections][Discuss] Detection engine tech debt/refactor opportunities #80792
   Feature:Detection Rules Team: SecuritySolution Team:Detection Engine Team:Detections and Resp Team:SIEM technical debt +6
2. [Security Solution][Alerts] Refactor security rules to use more similar coding patterns #140145
   Team:Detection Engine technical debt +2
3. [Security Solution][Alerts] Add TS types for remaining alert types #128937
   Team: SecuritySolution Team:Detection Engine Team:Detections and Resp technical debt +4
4. [Security Solution][Alerts] Integrate security rule schemas with alert schemas #128950
   Team: SecuritySolution Team:Detection Engine Team:Detections and Resp technical debt +4
5. [Security Solution] update generateId function to use _seq_no and _primary_term #102395
   Team: SecuritySolution Team:Detection Engine Team:Detections and Resp auto-backport release_note:skip technical debt +6
   

Enhancement

Beta Give feedback

1. https://github.com/elastic/enhancements/issues/18981
2. [Security Solution][Detection Alerts] Rule status only displays the last error/warning from a rule execution #115133
   Team: SecuritySolution Team:Detection Engine Team:Detections and Resp Theme: simp_prot_mgmt impact:medium +5
3. [RAM][Security Solution] Include the EQL building block alerts in the alerts context so that the action-per-alert context for EQL sequences has the entire sequence #155748
   Near Future Work Team:Detection Engine enhancement +3
4. [Security Solution][Alerts] Log actual queries executed by detection engine #141463
   Team: SecuritySolution Team:Detection Engine enhancement +3
5. [Security Solution][Alerts] Investigate ways to apply legacy alerts aliases without user action #126556
   Team: SecuritySolution Team:Detection Engine Team:Detections and Resp enhancement technical debt +5
6. [Security Solution][Alerts] Warn if rules are querying indices without efficient range configuration #120687
   Feature:Detection Rules Team: SecuritySolution Team:Detection Engine Team:Detections and Resp +4
7. Feature: Processing of threshold detection events #88632
   Feature:Threshold Rule Team: SecuritySolution Team:Detection Engine Team:Detections and Resp Team:SIEM enhancement triage_needed +7
8. [Security Solution] [Detections] Provide UI setting to disable rule's pre-execution privilege checks #90958
   Feature:Detection Rules Team: SecuritySolution Team:Detection Engine Team:Detections and Resp Theme: simp_prot_mgmt discuss enhancement +7

7.x Issues

Beta Give feedback

1. [Security Solution] Rules don't always write a status when finished executing #116358
   Feature:Rule Monitoring Team: SecuritySolution Team:Detection Engine Team:Detection Rule Management Team:Detections and Resp enhancement +6
2. [Security Solution] Strange Built-In Rule Behavior #125171
   Question Team: SecuritySolution Team:Detection Engine Team:Detections and Resp v7.17.0 +5