dfm_lookup is not robust enough

[koheiw]

# Error when nothing matches
mx <- dfm(data_char_ukimmig2010)
dict <- dictionary(list('en'=list('foreign policy' = 'aaaaaa', 'domestic politics' = 'bbbbb')))
dfm_lookup(mx, dict)

# Error in x[, unlist(entries_id, use.names = FALSE)] : 
#     invalid or not-yet-implemented 'Matrix' subsetting 

# This is OK
dict <- dictionary(list('en'=list('fp' = 'foreign', 'dp' = 'domestic')))
dfm_lookup(mx, dict)

# but I cannot use space in keys
dict <- dictionary(list('en'=list('foreign policy' = 'foreign', 'domestic politics' = 'domestic')))
dfm_lookup(mx, dict)

# Error in intI(j, n = x@Dim[2], dn[[2]], give.dn = FALSE) : 
#     invalid character indexing 
# In addition: Warning message:
#     In features2vector(features) :
#     dfm_lookup(mx, dict) does not support multi-word features

[kbenoit]

Interesting: These did not cause a problem for me:

> require(quanteda)
Loading required package: quanteda
quanteda version 0.9.9.50
Using 7 of 8 cores for parallel computing

Attaching package: ‘quanteda’

The following object is masked from ‘package:utils’:

    View

> mx <- dfm(data_char_ukimmig2010)
> dict <- dictionary(list('en'=list('foreign policy' = 'aaaaaa', 'domestic politics' = 'bbbbb')))
> dfm_lookup(mx, dict)
Document-feature matrix of: 9 documents, 2 features (0% sparse).
9 x 2 sparse Matrix of class "dfmSparse"
              features
docs           en.foreign policy en.domestic politics
  BNP                          0                    0
  Coalition                    0                    0
  Conservative                 0                    0
  Greens                       0                    0
  Labour                       0                    0
  LibDem                       0                    0
  PC                           0                    0
  SNP                          0                    0
  UKIP                         0                    0

> dict <- dictionary(list('en'=list('fp' = 'foreign', 'dp' = 'domestic')))
> dfm_lookup(mx, dict)
Document-feature matrix of: 9 documents, 2 features (0% sparse).
9 x 2 sparse Matrix of class "dfmSparse"
              features
docs           en.fp en.dp
  BNP              3     0
  Coalition        0     0
  Conservative     3     0
  Greens           0     0
  Labour           0     0
  LibDem           0     0
  PC               0     0
  SNP              0     0
  UKIP             1     0
> 
> # but I cannot use space in keys
> dict <- dictionary(list('en'=list('foreign policy' = 'foreign', 'domestic politics' = 'domestic')))
> dfm_lookup(mx, dict)
Document-feature matrix of: 9 documents, 2 features (0% sparse).
9 x 2 sparse Matrix of class "dfmSparse"
              features
docs           en.foreign policy en.domestic politics
  BNP                          3                    0
  Coalition                    0                    0
  Conservative                 3                    0
  Greens                       0                    0
  Labour                       0                    0
  LibDem                       0                    0
  PC                           0                    0
  SNP                          0                    0
  UKIP                         1                    0