[Timeline] Add Indicator Match Timeline Template

[peasead]

Summary

Currently, there are generic timelines for network, endpoint, and processes.

I would like to add a generic indicator match timeline for the Indicator Match detection rule.

This includes the threat.indicator.matched.atomic, threat.indicator.matched.field, and threat.indicator.matched.type fields.

Resolves #95565

Checklist

Delete any items that are not applicable to this PR.

• Documentation was added for features that require explanation or tutorials [Timeline] Update Timeline Template Documentation security-docs#600
• Unit or functional tests were updated or added to match the most common scenarios

Using the documentation here, I was able to successfully run the add_prepackaged_timelines.sh and ./timelines/find_timeline_by_filter.sh scripts to load and verify the Timeline templates in 7.12.0.

sh ./timelines/add_prepackaged_timelines.sh
{
  "success": false,
  "success_count": 1,
  "errors": [
    {
      "id": "(template_timeline_id) db366523-f1c6-4c1f-8731-6ce5ed9e5717",
      "error": {
        "status_code": 409,
        "message": "Timeline template version conflict. The provided templateTimelineVersion does not match the current template."
      }
    },
    {
      "id": "(template_timeline_id) 91832785-286d-4ebe-b884-1a208d111a70",
      "error": {
        "status_code": 409,
        "message": "Timeline template version conflict. The provided templateTimelineVersion does not match the current template."
      }
    },
    {
      "id": "(template_timeline_id) 76e52245-7519-4251-91ab-262fb1a1728c",
      "error": {
        "status_code": 409,
        "message": "Timeline template version conflict. The provided templateTimelineVersion does not match the current template."
      }
    }
  ],
  "timelines_installed": 1,
  "timelines_updated": 0

sh ./timelines/find_timeline_by_filter.sh immutable template elastic
{
  "data": {
    "getAllTimeline": {
      "totalCount": 4,
      "defaultTimelineCount": 0,
      "templateTimelineCount": 4,
      "elasticTemplateTimelineCount": 4,
      "customTemplateTimelineCount": 0,
      "favoriteCount": 0,
      "timeline": [
        {
          "savedObjectId": "13a24c90-918a-11eb-8c0e-81118a850125",
          "description": "This Timeline template is for alerts generated by Indicator Match detection rules.",
          "favorite": [],
          "eventIdToNoteIds": [],
          "notes": [],
          "noteIds": [],
          "pinnedEventIds": [],
          "status": "immutable",
          "title": "Generic Indicator Match Timeline",
          "timelineType": "template",
          "templateTimelineId": "495ad7a7-316e-4544-8a0f-9c098daee76e",
          "templateTimelineVersion": 1,
          "created": 1617130778584,
          "createdBy": "Elastic",
          "updated": 1617130778584,
          "updatedBy": "Elastic",
          "version": "WzIwMywxXQ==",
          "__typename": "TimelineResult"
        },
        {
          "savedObjectId": "1b29d5a0-9180-11eb-b3d2-7d2797f79400",
          "description": "",
          "favorite": [],
          "eventIdToNoteIds": [],
          "notes": [],
          "noteIds": [],
          "pinnedEventIds": [],
          "status": "immutable",
          "title": "Generic Process Timeline",
          "timelineType": "template",
          "templateTimelineId": "76e52245-7519-4251-91ab-262fb1a1728c",
          "templateTimelineVersion": 2,
          "created": 1617126496250,
          "createdBy": "Elastic",
          "updated": 1617126496250,
          "updatedBy": "Elastic",
          "version": "WzE0MCwxXQ==",
          "__typename": "TimelineResult"
        },
        {
          "savedObjectId": "1b29ae90-9180-11eb-b3d2-7d2797f79400",
          "description": "",
          "favorite": [],
          "eventIdToNoteIds": [],
          "notes": [],
          "noteIds": [],
          "pinnedEventIds": [],
          "status": "immutable",
          "title": "Generic Network Timeline",
          "timelineType": "template",
          "templateTimelineId": "91832785-286d-4ebe-b884-1a208d111a70",
          "templateTimelineVersion": 2,
          "created": 1617126496249,
          "createdBy": "Elastic",
          "updated": 1617126496249,
          "updatedBy": "Elastic",
          "version": "WzEzOSwxXQ==",
          "__typename": "TimelineResult"
        },
        {
          "savedObjectId": "1b298780-9180-11eb-b3d2-7d2797f79400",
          "description": "",
          "favorite": [],
          "eventIdToNoteIds": [],
          "notes": [],
          "noteIds": [],
          "pinnedEventIds": [],
          "status": "immutable",
          "title": "Generic Endpoint Timeline",
          "timelineType": "template",
          "templateTimelineId": "db366523-f1c6-4c1f-8731-6ce5ed9e5717",
          "templateTimelineVersion": 2,
          "created": 1617126496248,
          "createdBy": "Elastic",
          "updated": 1617126496248,
          "updatedBy": "Elastic",
          "version": "WzEzOCwxXQ==",
          "__typename": "TimelineResult"
        }
      ],
      "__typename": "ResponseTimelines"
    }
  }
}

For maintainers

• This was checked for breaking API changes and was labeled appropriately

[elasticmachine]

Pinging @elastic/security-solution (Team: SecuritySolution)

[MikePaquette]

@peasead this is awesome! Timeline templates are powerful, and I'm psyched to see this new one being added!

Question: What would you think about calling this a generic threat match timeline template?

Two reasons why I ask:

1. Forward looking: I understand that currently all indicator match rules write enrichment results to the threat.indicator.matched.* fields in the signal document, but our plans for this rule type are that it could be used more generically in the future, where the enrichment results might be stored in another prefix/object.

2. Consistency: Existing generic timeline templates are process, network, and endpoint which are more about the subject of timeline, rather than the rule type. threat match would seem to be more in line with the other generic timeline templates.

[peasead]

Great feedback, thanks @MikePaquette

Absolutely makes sense to change the name. I can do that and update the PR.

[peasead]

Can anyone help me with these checks?

[rylnd]

@elasticmachine merge upstream

[rylnd]

@elasticmachine merge upstream

[peasead]

Thanks @rylnd

What're the next steps? Anything I need to do?

[ecezalp]

was able to verify the changes with the following steps

1. set the following env vars

export ELASTICSEARCH_USERNAME=${user}
export ELASTICSEARCH_PASSWORD=${password}
export ELASTICSEARCH_URL=https://${ip}:9200
export KIBANA_URL=http://localhost:5601
export TASK_MANAGER_INDEX=.kibana-task-manager-${your user id}
export KIBANA_INDEX=.kibana-${your user id}

and source ~/.bash_profile

2. add xpack.securitySolution.signalsIndex: .siem-signals-${username} to config/kibana.dev.yml

3. run the following

brew update && brew install curl && brew install jq
cd x-pack/plugins/security_solution/server/lib/detection_engine/scripts
sh ./timelines/add_prepackaged_timelines.sh

4. go to http://localhost:5601/app/security#/detections/rules and click on Install Elastic prebuild rules

after following the steps above, I was able to find and interact with the threat match template. I was able to find my previously generated alerts in the timeline view.

LGTM!

[ecezalp]

@elasticmachine merge upstream

[kibanamachine]

💚 Build Succeeded

• continuous-integration/kibana-ci/pull-request
• Commit: 9b28812
• Storybooks Preview

Metrics [docs]

✅ unchanged

History

• 💚 Build #118097 succeeded 94abf20
• 💔 Build #117921 failed 3fdfff6
• 💔 Build #116558 failed f4e726b
• 💔 Build #116332 failed a4639ba

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

[rylnd]

We need to backport this to 7.x to get it into the 7.x releases. Doing that now!