Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Avoid mixing RMSE and MAPE as performance indicators for model selection #24

Closed
antoinecarme opened this issue Jan 5, 2017 · 2 comments
Closed

Avoid mixing RMSE and MAPE as performance indicators for model selection #24

antoinecarme opened this issue Jan 5, 2017 · 2 comments
Assignees
@antoinecarme
Labels
class:bug priority:high topic:modeling_quality

Comments

@antoinecarme
Copy link
Owner

The code as it is today, uses RMSE sometimes and MAPE sometimes.

There is an option,

self.mModelSelection_Criterion = "L2";

That shoud be used for model selection everywhere. By the way it should be set to "MAPE" be default.

This is a very serious bug.
@antoinecarme antoinecarme self-assigned this Jan 5, 2017
@antoinecarme
Copy link
Owner Author

Try to quantify the impact and run the tests before committing a fix.

antoinecarme added a commit that referenced this issue Jan 5, 2017
@antoinecarme
Avoid mixing RMSE and MAPE as performance indicators for model select…
9ea32d8 
…ion #24

Use MAPE by default.
antoinecarme added a commit that referenced this issue Jan 5, 2017
@antoinecarme
Avoid mixing RMSE and MAPE as performance indicators for model select…
5041a60 
…ion #24

Fixed!
antoinecarme added a commit that referenced this issue Jan 5, 2017
@antoinecarme
Avoid mixing RMSE and MAPE as performance indicators for model select…
bf395df 
…ion #24

quantify the impact and run the tests before committing

update these logs.
@antoinecarme
Copy link
Owner Author

FIXED!!!

antoinecarme added a commit that referenced this issue Jan 9, 2017
@antoinecarme
Avoid mixing RMSE and MAPE as performance indicators for model select…
c0166fd 
…ion #24

Artificial dataset test failure - Warning about singular matrix #22

Artificial dataset test failure #21

Updated these references.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@antoinecarme antoinecarme

Labels
class:bug priority:high topic:modeling_quality
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@antoinecarme