Kill tokenized texts

[koheiw]

No description provided.

[kbenoit]

Nice. RIP tokenizedTexts.

Can you run revdep_check on this? Ignore the issues I’ve already fixed in #1024.

[codecov]

Codecov Report

Merging #1034 into master will increase coverage by 0.8%.
The diff coverage is 100%.

@@            Coverage Diff            @@
##           master    #1034     +/-   ##
=========================================
+ Coverage   89.16%   89.97%   +0.8%     
=========================================
  Files          90       88      -2     
  Lines        6961     6631    -330     
=========================================
- Hits         6207     5966    -241     
+ Misses        754      665     -89

[koheiw]

Checking 10 packages: clustRcompaR, corpustools, gofastr, phrasemachine, preText, readtext, stm, textstem, tidytext, word.alignment
Package library: /tmp/RtmpoURqdx/revdep3840f6364d5, /tmp/RtmpoURqdx/R-lib, /usr/local/lib/R/site-library, /usr/lib/R/site-library, /usr/lib/R/library
Checked clustRcompaR  : 0 errors | 0 warnings | 0 notes
Checked corpustools   : 0 errors | 0 warnings | 0 notes
Checked gofastr       : 0 errors | 0 warnings | 0 notes
Checked phrasemachine : 0 errors | 0 warnings | 0 notes
Checked preText       : 0 errors | 0 warnings | 1 note 
Checked readtext      : 0 errors | 0 warnings | 0 notes
Checked stm           : 0 errors | 1 warning  | 0 notes
Checked textstem      : 1 error  | 0 warnings | 1 note 
Checked tidytext      : 1 error  | 0 warnings | 0 notes
Checked word.alignment: 0 errors | 0 warnings | 1 note 
Checked 10/10. Elapsed 00:11. Remaining ~00:00

[codecov-io]

Codecov Report

Merging #1034 into master will increase coverage by 0.8%.
The diff coverage is 100%.

@@            Coverage Diff            @@
##           master    #1034     +/-   ##
=========================================
+ Coverage   89.16%   89.97%   +0.8%     
=========================================
  Files          90       88      -2     
  Lines        6961     6631    -330     
=========================================
- Hits         6207     5966    -241     
+ Misses        754      665     -89

[kbenoit]

On the revdep checks:

• I just issued a PR for textstem (Use quanteda 1.0-compatible tokens() call trinker/textstem#6) to fix the error.
• My earlier PR for tidytext (Ensure compatibility w/quanteda v1.0 juliasilge/tidytext#87) passes fine with this branch.
• On stm, I cannot check the warning, since I am unable to build that package. (It issues a long stream of compiler warnings and then crashed out.)

[kbenoit]

Awesome work.

I'm going to approve and let you have the honour of merging!