Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

core: add metric savings to audit result #14997

Merged
merged 7 commits into from May 10, 2023
Merged

core: add metric savings to audit result #14997

merged 7 commits into from May 10, 2023

Conversation

adamraine
Copy link
Member

This PR allows audits to indicate the savings the audit has on each metric in the unit of the respective metric. Originally I imagined we would compute the potential 0-1 score increase for each metric, but I think it's better this way for a few reasons:

  • It wouldn't be that hard to recompute the log normal score in the report renderer, we would just need to add computeLogNormalScore and the default metric scoring options to the report renderer.
  • We will need the metric savings if we decide to compute impact against field metrics in the report renderer.
  • We don't have a system for audits to depend on one another, so computing the potential 0-1 score increase would require us to wire all the metric computation data artifacts into every performance audit.

@adamraine adamraine requested a review from a team as a code owner April 19, 2023 23:13
@adamraine adamraine requested review from brendankenny and removed request for a team April 19, 2023 23:13
@adamraine adamraine merged commit 3a0c62d into main May 10, 2023
33 checks passed
@adamraine adamraine deleted the metric-savings branch May 10, 2023 23:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants