[RAC] RuleDataClient writer should handle inserting required fields into new alert documents

[marshallmain]

Right now the RuleDataWriter is a thin wrapper around the ElasticsearchClient. It expects callers to provide a request that it can provide directly to the ElasticsearchClient.bulk method. The RuleDataWriter does not make any changes to the documents before sending them off to ES, which means callers are responsible for adding the required fields (at least space IDs, consumer, producer, rule type ID, event.kind, and @timestamp). Since these fields are absolutely required for all alerts-as-data documents in order for RBAC and filtering to work correctly, inserting them should not be left up to individual rule types.

Rather than taking a fully formed request ready to pass into ElasticsearchClient.bulk, the RuleDataWriter.bulk method could take an array of alert documents that each contain the rule specific fields that need to be included in the alert. The RuleDataWriter would then augment each document with the additional required fields, format the documents in the request to ElasticsearchClient.bulk, and index the docs.

This could be done as a refactor for 7.16 rather than rushing to get it in 7.15, as it's not absolutely required for alerts as data to work correctly. However, this change will make it easier to develop rules on top of the rule registry - reducing the likelihood of bugs in required fields and reducing the knowledge devs writing rules need to have about the internals of the rule registry - so it is medium-high priority.

[elasticmachine]

Pinging @elastic/security-detections-response (Team:Detections and Resp)

[elasticmachine]

Pinging @elastic/security-solution (Team: SecuritySolution)

[jasonrhodes]

I think this is a good idea, thanks for filing 👍

[weltenwort]

ℹ️ A lightweight variant of this on the lifecycle/persistent-executor-level is in progress in #108679.

[marshallmain]

Addressed by #108679