Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Address issue that Ludwig often reports root_mean_squared_error_percentage being infinity #1281

Closed
amholler opened this issue Sep 2, 2021 · 1 comment · Fixed by #2139
Closed
Assignees
Labels
help wanted Help from the community is welcome

Comments

@amholler
Copy link
Collaborator

amholler commented Sep 2, 2021

This seems to be happening when target value is 0, due to divide by 0. One example is trial_9b499_00018
in Sarcos/Hband/Tabconcat.

Piero and I discussed this; his comments:

Regarding RMSPE. It's a nasty issue and there's no consesus, see here:
https://stats.stackexchange.com/questions/86708/how-to-calculate-relative-error-when-the-true-value-is-zero
That said, I belive hasing a max(epsilon, abs(y)) as the denominator instead of y is a very reasonable choice.

Also, AFAICT we aren't multiplying the overall result by 100.0 to convert to percent FWIW.

@w4nderlust
Copy link
Collaborator

RMSPE numerical stability t's a nasty issue and there's no consesus, see here: https://stats.stackexchange.com/questions/86708/how-to-calculate-relative-error-when-the-true-value-is-zero
That said, I believe hasing a max(epsilon, abs(y)) as the denominator instead of y is a very reasonable choice.

@w4nderlust w4nderlust added easy help wanted Help from the community is welcome labels Sep 24, 2021
jppgks added a commit that referenced this issue Jun 14, 2022
@jppgks jppgks self-assigned this Jun 14, 2022
justinxzhao added a commit that referenced this issue Jun 21, 2022
* rmspe: add epsilon to avoid division by zero

Fixes #1281

* use test convention

* Update ludwig/utils/loss_utils.py

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

Co-authored-by: Justin <justinxzhao@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
help wanted Help from the community is welcome
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants