Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Include interpretations of metric values in reports #666

Open
wholmgren opened this issue Apr 1, 2021 · 0 comments
Open

Include interpretations of metric values in reports #666

wholmgren opened this issue Apr 1, 2021 · 0 comments
Labels
enhancement New feature or request reports Issue pertains to reporting

Comments

@wholmgren
Copy link
Member

From @dplarson and @awig

A common question is regarding how to interpret the results in “plain language”. That’s maybe not necessary for the more standard metrics (MAE, MBE, RMSE), but it does seem beneficial to make that clearer to report readers. One option is revising the metrics page (as needed) so each metric has an interpretation statement and then linking to the metrics page in the report. Another option would be adding a hover box or maybe just bullet points that list the interpretations per metric. E.g., MBE measures the bias of a forecast; MBE < 0 means the forecast tends to under-predict, MBE > 0 means the forecast tends to over-predict, and MBE = 0 means the forecast has no bias overall.

We should definitely add links from the report back to the metrics page. I like the idea of an interpretation statement, but worry that it would take a lot of time to do right. Hover boxes might also be helpful if we can keep them short and sweet.

Same applies to summary statistics.

Part of my interest in proposing this project came from struggling with the RMSE decomposition for solar forecasts. I've always wanted the reports to explore those kinds of issues. One challenge is that the metrics are user-selected and not always present in the report. I'll have to think harder about how to handle that. There has to be a better answer than just put it on the static metrics page - otherwise that'll turn into a text book that no one reads.

@wholmgren wholmgren added enhancement New feature or request reports Issue pertains to reporting labels Oct 1, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request reports Issue pertains to reporting
Projects
None yet
Development

No branches or pull requests

1 participant