how to select top most frequent features from a dfm

[Monduiz]

I am looking through the documentation trying to do the same as max_features from tfidfvectorizer in scikit.

max_features is very clearly defined in tfidfvectorizer:

If not None, build a vocabulary that only consider the top max_features ordered by term frequency across the corpus.

Like say I I use tokens(x, what = "character") and I would like to adjust with
dfm(x, max_features =50000) .

It seems logical to me that this would be a dfm operation. I am missing what the operation is and when it appears in the workflow if its already there. Basically, the goal is to reduce the dfm matrix to the 50000 first term by frequency. max_count in dfm_trim would not accomplish that.

What is not clear to me in the documentation:

If dfm(x[1:50000]; does this selects the first 50000 by top word frequency?
If dfm(x, max_docfreq = 50000; does this selects the first 50000 by top word frequency?

Do I have to add another step with %>% dfm_sort(margin = "features") but then not sure how to select by features and frequency.

Thanks for the guidance!

[kbenoit]

If you want to keep the top 50,000 most frequent documents, you can do so using:

x <- dfm_select(x, names(topfeatures(x, n = 50000)))

This also works:

x <- dfm_sort(x)[, max(c(nfeat(x), 50000)]

Also, we are working on this - see #1254, which as currently proposed would be:

dfm_trim(x, min_count = 50000, count_fun = "invrank")

[Monduiz]

Thank you @kbenoit! I like the solution proposed!