feat: lex 0s 1s 0's 1's etc as words

[hippietrail]

Issues

This addresses #774

Description

In sentences like "Computers work with 1s and 0s", 1 and 0 were lexed as numbers, cutting off the plural suffixes. This left s and to be a detected as a potential error for sand.

This change specifically lexes single digits followed by s or 's as words, leaving the rest up to the linters.

I've also added 0s and 1s to the curated dictionary. 0's and `1's1 are omitted as those should be flagged and suggested to change to the non-apostrophe versions.

Other digits and longer numbers are omitted from the dictionary although they are still lexed as words. They can be added in if we find evidence of them causing similar problems.

One unexpected side-effect is that even with 0s in the dictionary and 0's flagged as an error, Harper does not manage to suggest 0s as a correction for 0's.

Demo

Before:

After, spelled correctly, without apostrophes:

After, spelled incorrectly, with apostrophes:

How Has This Been Tested?

I added new tests, including for edge cases I identified, such as more letters or digits following directly after the s.

Checklist

• I have performed a self-review of my own code
• I have added tests to cover my changes

[elijah-potter]

One unexpected side-effect is that even with 0s in the dictionary and 0's flagged as an error, Harper does not manage to suggest 0s as a correction for 0's.

This is a flaw in our spell-checker, which is my next target for improvement.

On the whole, this is great work. I'm a little hesitant to add such a special case, but I can see no downside. Onward!

[hippietrail]

One unexpected side-effect is that even with 0s in the dictionary and 0's flagged as an error, Harper does not manage to suggest 0s as a correction for 0's.

This is a flaw in our spell-checker, which is my next target for improvement.

On the whole, this is great work. I'm a little hesitant to add such a special case, but I can see no downside. Onward!

The more dogfooding I do the more cases of numbers followed by various letter suffixes I find. I also noticed the "wordlikes" concept which I wasn't aware of when I made this or the "decades" stuff.

I think these all needed to be treated as words to prevent the suffixes being separated by the lexer and then taking part in various linters with what comes after the next space, which so far always feels wrong.

I think it's fine to have special-case lexers that catch either words or wordlikes that are hard to catch in monolithic lexers but I think I made the wrong choice with the decade lexer being its own new token type instead of either word or wordlike.

I'm not sure about actual words vs wordlikes yet. I'll dig into that soon I think.