Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clear Metrics Description #57

Closed
2448845600 opened this issue Jun 21, 2023 · 2 comments
Closed

Clear Metrics Description #57

2448845600 opened this issue Jun 21, 2023 · 2 comments

Comments

@2448845600
Copy link

With CFG.NULL_VAL = 0.0, three metrics (MAE, RMSE, MAPE) are defaulted as masked ones. Maybe you should use Masked XXX instead of XXX in experimental results?

@zezhishao
Copy link
Collaborator

zezhishao commented Jun 21, 2023

Hi thanks for you suggestions.
The “0” is the abnormal value for traffic datasets. Thus, the “MAE” we wanted is indeed the masked ones, since calculating errors on abnormal values and forcing models to fit these abnormal data does not make any sense. Almost all the previous works followed this setting and reported these metrics without the “masked”.
Moreover, the default null value of these metrics is nan, instead of 0.0, since in many datasets the “0” is not abnormal values.

@2448845600
Copy link
Author

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants