Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.Sign up
Metrics for assessing model fit for a portfolio of hourly-modeled buildings #99
#85 At a previous meeting we saw OpenEE's work on weighted portfolio CV(RMSE), and I wanted to jot down some LBNL thinking on that. If RMSE is squared before adding up the values, then the output is a statistically valid metric, an aggregated CV(RMSE). Two cautions though, when interpreting the results:
We're happy to chat more on this (though I'm not the right person to get deeper in the weeds myself)
Thanks @eliotcrowe. I think we're on the same page - here are some of the conclusions that were buried in the 50 or so threads in #71. in general, we think the aggregate CVRMSE might be a decent metric for planning and trend analysis, but we definitely can't determine the actual values of appropriate thresholds without more testing, because they would vary depending on many different factors.