tokens.tokens(x, remove_hyphens = TRUE) does not split hyphenated word components

[kbenoit]

Describe the bug

tokens.tokens(x, remove_hyphens = TRUE) does not split the hyphenated words.

> txt <- "Auto-immune system."
> tokens(txt, remove_hyphens = TRUE)
tokens from 1 document.
text1 :
[1] "Auto"   "-"      "immune" "system" "."     

> tokens(txt, remove_hyphens = FALSE)
tokens from 1 document.
text1 :
[1] "Auto-immune" "system"      "."          

> tokens(txt, remove_hyphens = FALSE) %>% tokens(remove_hyphens = TRUE)
tokens from 1 document.
text1 :
[1] "Auto immune" "system"      "."  

Expected behavior

> tokens(txt, remove_hyphens = FALSE) %>% tokens(remove_hyphens = TRUE)
tokens from 1 document.
text1 :
[1] "Auto"   "-"      "immune" "system" "."

## System information

> packageVersion("quanteda")
[1] ‘1.3.14’

[kbenoit]

We should close this issue since this behaviour is now resolved. It's inefficient, true, but we can address that in #1500.