[FR] Add details and export options for dev package-stats command #2020
Description
Is your feature request related to a problem? Please describe.
This feature request is not related to a problem, it is more of a small enhancement to current capabilities. Within the dev side of the detection rules as a utility, we have the capability to quickly identify new, modified and deprecated rules for each Elastic Stack release cycle.
Describe the solution you'd like
While this is great, the option to enabled details which show all rules and their respective status, along with export options for JSON and CSV would be beneficial for end-to-end (E2E) rule scoping and testing for each cycle.
This will use the Pandas library to convert the collected rules into a dataframe, thus allowing aggregations and export options to JSON or CSV. Additionally, printing the dataframe as a markdown table to the console for engineers who do not want it exported but may use it in Github.
Dataframe to Markdown and printed to console
Exported to CSV
Describe alternatives you've considered
I have considered just printing to console from a loop, iterating over each object of new, changed and modified rules. While this would work, leveraging pandas will unlock some other potentials for aggregations, easy filtering of results as well as export options.
Additional context
This code was added during E2E testing to scope out which rules to be verified and worked. Code has already been written and tested but adding this issue for discussion purposes.
New Commands:
python -m detection_rules dev package-stats -t TOKEN_ID --details
python -m detection_rules dev package-stats -t TOKEN_ID -e "json/csv"