Fix tests for corrected tokenize_skip_ngrams()

[lmullen]

This pull request fixes two failing tests caused by an imminent release of v0.2 of the tokenizers package. The version 0.1.4 of the tokenizers package currently on CRAN has an incorrect implementation of skip n-grams. (See this issue for details.) The master branch of tokenizers has a fix. As a result the indices used in the tests are now incorrect and I have updated them.

Note that tokenize_skip_ngrams() now returns vastly more tokens, which is how it is supposed to work, but which might come as a surprise.

[lmullen]

The CI checks failed because they are using the CRAN version of tokenizers.

[juliasilge]

Thank you so much, Lincoln! 🙌

[juliasilge]

Wowwwwwwwww, that is a lot more skip grams. 😯

[lmullen]

@juliasilge Thanks for the quick response to the CRAN maintainers, and for your flexibility in having to create a new release because of tokenizers changes.

[juliasilge]

@lmullen Do you think the CRAN maintainers intend for me to submit it right away (when my Travis builds and win-builder are still failing because binaries aren't built yet) or to wait until I can demonstrate that everything is passing?

[lmullen]

I think they probably want it right away. They will run their own tests on the current CRAN, I'd guess.

[github-actions]

This pull request has been automatically locked. If you believe you have found a related problem, please file a new issue (with a reprex: https://reprex.tidyverse.org) and link to this issue.