P4 metric

[davidechicco]

Hola Adrian
Thanks for having released the metrica R package, it is very useful.

I have a request: can you please add the P4 metric to the list of available metrics?

The formula is the following one: P4 = (4 * TN * TP) / (4 * TN * TP + (TP + TN)*(FP + FN))

Thanks!

-- Davide

[adriancorrendo]

Hola Davide! Gracias.

P4 is a nice suggestion!
Sure, I'll add it to metrica. Just give some time to prepare the formula and documentation.
It should be ready in a week or so!

Thanks,

ADRIAN

[adriancorrendo]

Hi Davide,

Please, could you check that the new function p4() works as expected??

You may install the development version contained in the branch named 'update_P4'

install.packages("devtools")

devtools::install_github("adriancorrendo/metrica", ref = "update_P4")

If you can confirm it works for you I will proceed to make the update to main branch and submit to CRAN.

Thanks,

Adrian

[davidechicco]

Hi Adrian
Thanks for having implemented the P4 function.

I tried to run one of your examples but unfortunately it did not work:

binary_landCover %>% p4(data = ., obs = actual, pred = predicted, tidy = TRUE)
Error in p4(data = ., obs = actual, pred = predicted, tidy = TRUE) : 
  could not find function "p4"

[adriancorrendo]

Thanks, Davide!

Did you check that the library is loaded?

The issue looks like because it does find the function. This is because the package is either not loaded or didn't install correctly. If you installed the version from the branch "update_P4" (with the code install_github(..., ref = "update_P4"), p4 should pop up when you type metrica::p4(...)

Did you have any issue when installing from this specific branch?

Thanks,

Adrian

[davidechicco]

Hola Adrian
Now it works, I obtained p4 = 0.9531223 for that land cover example.

Best,

-- Davide

[adriancorrendo]

Awesome!
I will then proceed to close this issue.

You may expect to see the new version 2.0.4 on CRAN during the next few days.

Cheers,

ADRIAN

[adriancorrendo]

Davide,

you may see that for MULTICLASS cases, the p4 is not working.
So now it just generated a "NaN". I've tried the generalized formula that uses precision, recall, specificity, and npv, but I was getting weird values.

For multinomial cases, I normally apply an argument called "atom = TRUE/FALSE", which will allow to estimate the value for each class (atom = TRUE) or as a "general/overall" estimate (atom = FALSE). However, I couldn't make it work.

Do you think you could help me with a generalization formula of the p4 for multiclass cases?

You may check my code attempt with some #hidden lines at this file from lines 105-141.

Thank you in advance!

Best,

ADRIAN

[adriancorrendo]

I meant, it works, but giving weird values (> 1, sometimes negatives).
In theory, TN, TP, etc could be obtained without major issues from the confusion matrix, but in this case it didn't work well.
That's why I just decided to throw a Warning message and generate NaN when the user try it for multinomial cases.

Best,

Adrian

[davidechicco]

Hi Adrian,
I'm sorry but I don't know how to help you here. If you have a line of code I could run on my laptop, just let me know.