Starting point for TidierText.jl

[kdpsingh]

If we decide to make a TidierText.jl package, here are two great resources as a starting point:

• https://gist.github.com/atantos/8b306b800a5a23fb9b9ebfecde6219d2

• https://youtu.be/CwQLa2_ljpI

[drizk1]

Over the last week or so, I have been building on the above work to get this draft of TidierText.jl. I thought i would share it here, any/all are welcome to join in/share thoughts.

bind_tf_idf, most of the unnest_*, and anti_join are implemented with some optional arguments and basic macros to work with TidierData.jl as below:

@chain sentences begin
    #@unnest_tokens(word, text, to_lower = true)
    @unnest_regex(word, text, "\\.? ")
    @group_by(doc_id, word)
    @summarize(n = n())
    @ungroup()
    @arrange(doc_id)
    @antijoin(get_stopwords())
    @bind_tf_idf(word, doc_id, n)
end

[kdpsingh]

This looks amazing, @drizk1! Do you want to transfer this repo into TidierOrg? Then we can work together to get this in shape for release.

[drizk1]

Sounds good! I'll get it transferred over shortly

[kdpsingh]

Now that TidierText.jl is up and off to the races, will close this issue.