#5123 Text.tokenize

[GregoryTravis]

Pull Request Description

Implement Text.tokenize.

Important Notes

Checklist

Please ensure that the following checklist has been satisfied before submitting the PR:

• The documentation has been updated, if necessary.
• Screenshots/screencasts have been attached, if there are any visual changes. For interactive or animated visual changes, a screencast is preferred.
• All code follows the
  Scala,
  Java,
  and
  Rust
  style guides. In case you are using a language not listed above, follow the Rust style guide.
• All code has been tested:
  • Unit tests have been written where possible.
  • If GUI codebase was changed, the GUI was tested when built using ./run ide build.

[jdunkerley]

Looks good to me. Can we test for one where a sub-group is conditional - something like ((ab)|(baa)).

[radeusgd]

Looks good