Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Text processing #10

jeroen opened this issue Oct 3, 2019 · 0 comments


Copy link

commented Oct 3, 2019

Text Analysis in R

Text analysis is a scientific subspace that is not served well by general tidyverse / datascience tools. It is a focus area of rOpenSci and we have interesting tools and expertise around this topic.

  • hunspell: High-Performance Stemmer, Tokenizer, and Spell Checker
  • spelling: spell checker based on hunspell
  • tokenizers (Lincoln Mullen): high level tokenizing toolkit
  • textreuse: measuring similarity among documents
  • pdftools: extract text or data from pdf files
  • tabulizer: extract tables from pdf files
  • rtika: extract text or metadata from any file format
  • tesseract: ocr engine (text from images)
  • antiword, unrtf: extract text from word/rtf files.
  • cld2, cld3: language detection

Besides our own packages, we also have a lot of experts in our network doing interesting things. The tidytext package was conceived at unconf 2016 and Kenneth and Patrick organized the ropensci textworkshop 2017 and 2018.

Suggested speakers and topics

The call could be an intro/overview of text tools, or alternatively an expert could give an in-depth talk about a specific tool. We should not restrict this to official "ropensci packages", but reach out to our network about developments in the space that are interesting to our community.

Some cool recent work is done by the quanteda group from Kenneth Benoit. They are also building out a commercial product based on the quanteda R package. It would be interesting to hear what is new with quanteda, and their experience with building a product on top of R.

@jeroen jeroen added the 0/idea label Oct 3, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
None yet
1 participant
You can’t perform that action at this time.