[FEATURE] Enhancements to stats table

[pinnapuvijay]

Is your feature request related to a problem? Please describe.
During our testing/adoption,

1. we feel like a framework level flag to determine how the stats should be stored(relational vs nested constructs)
2. Storing the stats also for the rules that pass would be helpful
3. Capturing the actual value of the rule output( especially aggregations) so they can be referenced to determine if a rule is passed/failed in the subsequent executions of the rule. Basically, looking at historical metric(last 7 days avg, etc) to determine the status of the rule.
4. Not all rules get executed even if one of the rules marked as FAIL fails. A framework level flag to exit or continue with the other rules when a rule fails would.
5. stats for the rule that is marked as FAIL need to be logged as well when the rule fails.

Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
May be a stats_detail that supplements the stats table will also be a good alternative. This detail table will be in relational format and would also contain granular data that is requested above

Additional context
Add any other context or screenshots about the feature request here.

[asingamaneni]

@pinnapuvijay Thanks for creating this issue. Except for the first point I agree with all the new feature requests.
We don't need to have stat's table as relational and nested constructs can easily be unwrapped, as well as if teams try to interchange the settings there will be problems with table schema.

Can you please show how you would like to take the implementation for the remaining and flags you would introduce as part of this pr.

[pinnapuvijay]

I would assume teams who choose one or the other to begin with will remain to continue to stick to it, unless it is really not working for them. That's why, i am proposing to have a framework level flag with enough documentation on how either of the options benefits them so teams can make an educated choice to begin their journey with Spark Expectations. Even if they choose to switch down the road, i understand there would be schema changes but i would be interested to know how it cause confusion or complexities with switching.

[asingamaneni]

I would assume teams who choose one or the other to begin with will remain to continue to stick to it, unless it is really not working for them. That's why, i am proposing to have a framework level flag with enough documentation on how either of the options benefits them so teams can make an educated choice to begin their journey with Spark Expectations. Even if they choose to switch down the road, i understand there would be schema changes but i would be interested to know how it cause confusion or complexities with switching.

Can you please illustrate how the data model is going to be for the stats table. Want to review before proceeding.