Mean Absolute Percentage Error (MAPE)

[aalbayrak394]

🚀 Feature

I'd like to implement the mean absolute percentage error (MAPE) in ignite/metrics.

It is a commonly used metric for regression problems and it would be really convenient to be able to use it directly with ignite evaluators.

For that, I would write a custom Metric class in a new file mean_absolute_percentage_error.py inheriting from the base Metric class in ignite/metrics/metric.py.

[vfdev-5]

@aalbayrak394 thanks for asking, we have this metric in ignite contrib metrics: https://pytorch.org/ignite/v0.4.13/generated/ignite.contrib.metrics.regression.MedianAbsolutePercentageError.html

EDIT: sorry, misread mean vs median.

yes, let's add it into contrib/regression metrics

OK, MAPE is also MARE: https://pytorch.org/ignite/v0.4.13/generated/ignite.contrib.metrics.regression.MeanAbsoluteRelativeError.html#ignite.contrib.metrics.regression.MeanAbsoluteRelativeError

Let's make a shortcut and improve docs for better discovery.
What do you think ?

[aalbayrak394]

Sure, that would make more sense. Maybe adding ", also known as Mean Absolute Percentage Error" to the first line of the doc would do it?

[vfdev-5]

Yeah, and also add "(MAPE)" and "(MARE)" like that:

Calculate Mean Absolute Relative Error (MARE), also known as Mean Absolute Percentage Error (MAPE).

[aalbayrak394]

I created a PR #3199

[aalbayrak394]

@vfdev-5 How comes that some CI tests are failing when I only changed a line in the docstring?

[vfdev-5]

@aalbayrak394 some CI tests are flaky and some of them are broken right now.

For example, https://github.com/pytorch/ignite/actions/workflows/triage.yml, where I upgraded actions/labeler to v5 but they had a breaking change between v4 and v5. If you would like to help with it, you are very welcome!