Issue with MRR

[dhairyadalal]

Attempting to the MRR with your example and am getting a Typing error.

System:

• python 3.8.2

from rank_eval import ndcg, mrr
import numpy as np

# Note that y_true does not need to be ordered
# Integers are documents IDs, while floats are the true relevance scores
y_true = np.array([[[12, 0.5], [25, 0.3]], [[11, 0.4], [2, 0.6]]])
y_pred = np.array(
    [
        [[12, 0.9], [234, 0.8], [25, 0.7], [36, 0.6], [32, 0.5], [35, 0.4]],
        [[12, 0.9], [11, 0.8], [25, 0.7], [36, 0.6], [2, 0.5], [35, 0.4]],
    ]
)
k = 5

mrr(y_true, y_pred, k, threads=1)

Error message

---------------------------------------------------------------------------
TypingError                               Traceback (most recent call last)
<ipython-input-24-7d69c81d4a8f> in <module>
     13 k = 5
     14 
---> 15 mrr(y_true, y_pred, k, threads=1)

~/anaconda3/envs/deeplearning/lib/python3.8/site-packages/rank_eval/metrics.py in mrr(y_true, y_pred, k, return_mean, sort, threads)
    482     """
    483 
--> 484     return _choose_optimal_function(
    485         y_true=y_true,
    486         y_pred=y_pred,

~/anaconda3/envs/deeplearning/lib/python3.8/site-packages/rank_eval/metrics.py in _choose_optimal_function(y_true, y_pred, f_name, f_single, f_parallel, f_additional_args, return_mean, sort, threads)
    234         if sort:
    235             y_pred = _descending_sort_parallel(y_pred)
--> 236         scores = f_parallel(y_true, y_pred, **f_additional_args)
    237         if return_mean:
    238             return np.mean(scores)

~/anaconda3/envs/deeplearning/lib/python3.8/site-packages/numba/core/dispatcher.py in _compile_for_args(self, *args, **kws)
    399                 e.patch_message(msg)
    400 
--> 401             error_rewrite(e, 'typing')
    402         except errors.UnsupportedError as e:
    403             # Something unsupported is present in the user code, add help info

~/anaconda3/envs/deeplearning/lib/python3.8/site-packages/numba/core/dispatcher.py in error_rewrite(e, issue_type)
    342                 raise e
    343             else:
--> 344                 reraise(type(e), e, None)
    345 
    346         argtypes = []

~/anaconda3/envs/deeplearning/lib/python3.8/site-packages/numba/core/utils.py in reraise(tp, value, tb)
     77         value = tp()
     78     if value.__traceback__ is not tb:
---> 79         raise value.with_traceback(tb)
     80     raise value
     81 

TypingError: Failed in nopython mode pipeline (step: nopython frontend)
Failed in nopython mode pipeline (step: nopython frontend)
Failed in nopython mode pipeline (step: nopython frontend)
Invalid use of Function(<built-in function contains>) with argument(s) of type(s): (array(float64, 1d, A), float64)
 * parameterized
In definition 0:
    All templates rejected with literals.
In definition 1:
    All templates rejected without literals.
In definition 2:
    All templates rejected with literals.
In definition 3:
    All templates rejected without literals.
In definition 4:
    All templates rejected with literals.
In definition 5:
    All templates rejected without literals.
In definition 6:
    All templates rejected with literals.
In definition 7:
    All templates rejected without literals.
In definition 8:
    All templates rejected with literals.
In definition 9:
    All templates rejected without literals.
In definition 10:
    All templates rejected with literals.
In definition 11:
    All templates rejected without literals.
In definition 12:
    All templates rejected with literals.
In definition 13:
    All templates rejected without literals.
This error is usually caused by passing an argument of a type that is unsupported by the named function.
[1] During: typing of intrinsic-call at /home/doc/anaconda3/envs/deeplearning/lib/python3.8/site-packages/rank_eval/metrics.py (78)

File "../../../anaconda3/envs/deeplearning/lib/python3.8/site-packages/rank_eval/metrics.py", line 78:
def _reciprocal_rank(y_true, y_pred, k):
    <source elided>
    for i in range(k):
        if y_pred[i, 0] in y_true[:, 0]:
        ^

[1] During: resolving callee type: type(CPUDispatcher(<function _reciprocal_rank at 0x7f6697ed9dc0>))
[2] During: typing of call at /home/doc/anaconda3/envs/deeplearning/lib/python3.8/site-packages/rank_eval/metrics.py (11)

[3] During: resolving callee type: type(CPUDispatcher(<function _reciprocal_rank at 0x7f6697ed9dc0>))
[4] During: typing of call at /home/doc/anaconda3/envs/deeplearning/lib/python3.8/site-packages/rank_eval/metrics.py (11)


File "../../../anaconda3/envs/deeplearning/lib/python3.8/site-packages/rank_eval/metrics.py", line 11:
def _parallelize(f, y_true, y_pred, k):
    <source elided>
    for i in prange(len(y_true)):
        scores[i] = f(y_true[i], y_pred[i], k)
        ^

[1] During: resolving callee type: type(CPUDispatcher(<function _parallelize at 0x7f6697ed4e50>))
[2] During: typing of call at /home/doc/anaconda3/envs/deeplearning/lib/python3.8/site-packages/rank_eval/metrics.py (85)

[3] During: resolving callee type: type(CPUDispatcher(<function _parallelize at 0x7f6697ed4e50>))
[4] During: typing of call at /home/doc/anaconda3/envs/deeplearning/lib/python3.8/site-packages/rank_eval/metrics.py (85)


File "../../../anaconda3/envs/deeplearning/lib/python3.8/site-packages/rank_eval/metrics.py", line 85:
def _mrr(y_true, y_pred, k):
    return _parallelize(_reciprocal_rank, y_true, y_pred, k)
    ^

[AmenRa]

It works on my end.

Do the other metrics raise the same error?

[AmenRa]

Closing for inactivity. Feel free to reopen the issue.