removeFeatures returns empty list with glob/regex

[kazarmy]

Given (note #citizens in wash1):

txt <- c(wash1 <- "Fellow #citizens, I am again called upon by the voice of my country to 
                   execute the functions of its Chief Magistrate.",
         wash2 <- "When the occasion proper for it shall arrive, I shall endeavor to express
                   the high sense I entertain of this distinguished honor.")

both

removeFeatures(tokenize(txt, removePunct = TRUE), "#*", valuetype="glob")

and

removeFeatures(tokenize(txt, removePunct = TRUE), "#(.*)", valuetype="regex")

return

tokenizedText object from 2 documents.
Component 1 :
 [1] "Fellow"     "I"          "am"         "again"      "called"     "upon"       "by"         "the"        "voice"     
[10] "of"         "my"         "country"    "to"         "execute"    "the"        "functions"  "of"         "its"       
[19] "Chief"      "Magistrate"

Component 2 :
character(0)

when they should return

tokenizedText object from 2 documents.
Component 1 :
 [1] "Fellow"     "I"          "am"         "again"      "called"     "upon"       "by"         "the"        "voice"     
[10] "of"         "my"         "country"    "to"         "execute"    "the"        "functions"  "of"         "its"       
[19] "Chief"      "Magistrate"

Component 2 :
 [1] "when"          "the"           "occasion"      "proper"        "for"           "it"            "shall"        
 [8] "arrive"        "i"             "shall"         "endeavor"      "to"            "express"       "the"          
[15] "high"          "sense"         "i"             "entertain"     "of"            "this"          "distinguished"
[22] "honor"

I'm using quanteda 0.9.4 with R 3.2.3 (64-bit) on Windows 10.
A workaround is to do the following:

tokens <- tokenize(txt, removePunct = TRUE)
attr.tokens <- attributes(tokens)
tokens <- lapply(tokens, function(doc) doc[!stringi::stri_startswith_fixed(doc, "#")])
attributes(tokens) <- attr.tokens

[kbenoit]

Note: Always better to use the quanteda methods rather than simple lapply, since lapply will strip some attributes that are meaningful to quanteda. Thanks for finding the bug.