[SIEM][Detection Engine] Append to Query in Detections Rules

[dhurley14]

Meta ticket tracking this
#59715

⚠️POC Ticket (Proof of Concept) ⚠️

As a user I want to augment or append the immutable query with additional filter / query capabilities or add an augmentation to an existing rule. This will be space-specific for now and this will result in a new version of the rule when it is edited.

• Modify update rule route with optional query
• Update rules schema to include this as a first class citizen
• Use io.ts for the output validation and leave anything else alone until other work is finished.
• Use a feature flag for env var / kibana.yml to prevent this from being exposed in 7.7 🙅‍♂
• Add tests to ensure feature flag is not set

Unknowns

This should be combinable with the global lists somehow. Once this ticket is complete we can figure out how to combine them, but these are not dependent on each other.

Will the filter type get passed down from component?

Knowns

How will this impact versioning? - increment rule version number -> prepackaged rules will not be able to use this.

Example

When updating a rule there will be a new field where the user can augment this rule and create a new version. This will be space specific for now.

{
  "name": "Query with a rule id",
  "description": "Query with a rule_id that acts like an external id",
  "rule_id": "query-rule-id",
  "risk_score": 1,
  "severity": "high",
  "type": "query",
  "query": "user.name: root or user.name: admin",
  "user_query": "host.name: google", // Lucene or kuery and then for ML it would have to be KQL
  "user_filter": [
    {
      "query": {
        "match_phrase": {
          "host.name": "some-computer"
        }
      }
    },
    {
      "exists": {
        "field": "host.hostname"
      }
    }
  ],
}

[elasticmachine]

Pinging @elastic/siem (Team:SIEM)

[FrankHassanabad]

"Reviewed by Frank Hassanabad on 7/29/2020", This looks complete and will be shippable at this point. Going to close this and associated tickets. Any new bugs/features should be new tickets. Note this work is superseded by exception lists being a join data type.