[Security Solution] Introduce an endpoint to use for the detection rules table filters/data

[xcrzx]

Epic: https://github.com/elastic/security-team/issues/1974 (internal)

Summary

Currently we use the detection_engine/rules/prepackages/_status endpoint to populate the Elastic rules and Custom rules counters on the rules management page:

We re-fetch those counters once in 5 minutes and on every rule state mutation. However, the detection_engine/rules/prepackages/_status was designed for different purposes. Using it for re-fetching the rules table counters is computationally expensive and will become even more expensive with advances in rule customization epic.

We need to extract the logic that calculates the number of custom and prebuilt rules to a separate endpoint and use it to populate the rules table values. Consider introducing an endpoint that returns all data required for the table filters: the number of custom and prebuilt rules, rule tags, etc.

[elasticmachine]

Pinging @elastic/security-detections-response (Team:Detections and Resp)

[elasticmachine]

Pinging @elastic/security-solution (Team: SecuritySolution)

[banderror]

Once #125659 is addressed we will need to get back to the filters endpoint and optimize its performance by using aggregation queries.

[banderror]

Re: #137428 (comment)

Aggregations are already used for fetching aggregated rule tags. What we can do is to use aggs for fetching the rest of the data needed for filters. We can do it as part of a separate ticket: #166364