You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
It is possible to keep unigrams in tokens_ngrams() by setting n = 1 but it is not in tokens_compound(). Keeping unigrams makes it easy to apply pattern matching in the downstream. How about adding an option like keep_unigram?
It is possible to keep unigrams in
tokens_ngrams()by settingn = 1but it is not intokens_compound(). Keeping unigrams makes it easy to apply pattern matching in the downstream. How about adding an option likekeep_unigram?