coercion to matrix of textstat_simil return zeros the diagonals

[kbenoit]

Our method for as.matrix.dist() is from base, but we have our own quanteda::as.matrix.simil(). I suggest we add proxy to the Imports and use its as.matrix.dist() and as.matrix.simil() functions. The main difference is how it treats the diagonals. From ?proxy::as.matrix.simil:

diag: In the context of as.matrix the value to use on the diagonal representing self-proximities. In case of similarities, this defaults to NA since a priori there are no upper bounds, so the maximum similarity needs to be specified by the user.

set.seed(10)
x <- matrix(rnorm(16), ncol = 4)

## should be the same: OK
proxy::simil(x)
#             1           2           3
# 2 -0.13028526                        
# 3  0.03843595  0.37908734            
# 4  0.90753240  0.19031358  0.41521589
textstat_simil(as.dfm(x))
#             text1       text2       text3
# text2 -0.13028526                        
# text3  0.03843595  0.37908734            
# text4  0.90753240  0.19031358  0.41521589

## should be the same: FAIL
proxy::simil(x) %>% quanteda:::as.matrix.simil()
#             1          2          3         4
# 1  1.00000000 -0.1302853 0.03843595 0.9075324
# 2 -0.13028526  1.0000000 0.37908734 0.1903136
# 3  0.03843595  0.3790873 1.00000000 0.4152159
# 4  0.90753240  0.1903136 0.41521589 1.0000000
proxy::simil(x) %>% proxy:::as.matrix.simil()
#             1          2          3         4
# 1          NA -0.1302853 0.03843595 0.9075324
# 2 -0.13028526         NA 0.37908734 0.1903136
# 3  0.03843595  0.3790873         NA 0.4152159
# 4  0.90753240  0.1903136 0.41521589        NA
## need to add diag argument to as.matrix.simil
proxy::simil(x) %>% proxy:::as.matrix.simil(diag = 1.0)
#             1          2          3         4
# 1  1.00000000 -0.1302853 0.03843595 0.9075324
# 2 -0.13028526  1.0000000 0.37908734 0.1903136
# 3  0.03843595  0.3790873 1.00000000 0.4152159
# 4  0.90753240  0.1903136 0.41521589 1.0000000

## should be the same:  FAIL
textstat_simil(as.dfm(x)) %>% quanteda:::as.matrix.simil()
#             text1      text2      text3     text4
# text1  1.00000000 -0.1302853 0.03843595 0.9075324
# text2 -0.13028526  1.0000000 0.37908734 0.1903136
# text3  0.03843595  0.3790873 1.00000000 0.4152159
# text4  0.90753240  0.1903136 0.41521589 1.0000000
textstat_simil(as.dfm(x)) %>% proxy:::as.matrix.simil()
#             text1      text2      text3     text4
# text1          NA -0.1302853 0.03843595 0.9075324
# text2 -0.13028526         NA 0.37908734 0.1903136
# text3  0.03843595  0.3790873         NA 0.4152159
# text4  0.90753240  0.1903136 0.41521589        NA
# should require explicit diag = 1.0 argument
textstat_simil(as.dfm(x)) %>% proxy:::as.matrix.simil(diag = 1.0)
#             text1      text2      text3     text4
# text1  1.00000000 -0.1302853 0.03843595 0.9075324
# text2 -0.13028526  1.0000000 0.37908734 0.1903136
# text3  0.03843595  0.3790873 1.00000000 0.4152159
# text4  0.90753240  0.1903136 0.41521589 1.0000000


proxy::dist(x)
#          1        2        3
# 2 2.991541                  
# 3 2.381054 2.297994         
# 4 0.730720 2.608110 1.917761
textstat_dist(as.dfm(x))
#          text1    text2    text3
# text2 2.991541                  
# text3 2.381054 2.297994         
# text4 0.730720 2.608110 1.917761

proxy::dist(x) %>% proxy:::as.matrix.dist()
#          1        2        3        4
# 1 0.000000 2.991541 2.381054 0.730720
# 2 2.991541 0.000000 2.297994 2.608110
# 3 2.381054 2.297994 0.000000 1.917761
# 4 0.730720 2.608110 1.917761 0.000000
textstat_dist(as.dfm(x)) %>% base::as.matrix() # calls base::as.matrix.dist
#          text1    text2    text3    text4
# text1 0.000000 2.991541 2.381054 0.730720
# text2 2.991541 0.000000 2.297994 2.608110
# text3 2.381054 2.297994 0.000000 1.917761
# text4 0.730720 2.608110 1.917761 0.000000
## same except proxy::as.matrix.dist() has a diag = 0 argument
textstat_dist(as.dfm(x)) %>% proxy:::as.matrix.dist()
#          text1    text2    text3    text4
# text1 0.000000 2.991541 2.381054 0.730720
# text2 2.991541 0.000000 2.297994 2.608110
# text3 2.381054 2.297994 0.000000 1.917761
# text4 0.730720 2.608110 1.917761 0.000000

[kbenoit]

Thinking about this more, let's forget it. The default diagonal should be 1.0 for the similarity of an object with itself, for the measures we support.

[kbenoit]

We still have a problem noted here that the diagonal becomes zero in the coercion of a textstat_simil() object to a matrix. This should not happen. proxy makes it NA, but we should make it 1.0.