Fix term prefix/postfix regular expressions

[JaredReisinger]

If compromise gets bundled/packed/minified/transpiled into an aggregate Javascript file, it's possible for the term prefix/postfix regular expressions to change semantically, because they aren't strictly correct.

In particular these two lines include the escapes \& and \•, which aren't necessary. The first passes through just fine, but \• gets converted (in my case) to \\u2022. Since this is in the context of a charset ([]), it now means that the prefix/postfix will include \, u, 2, and 0. This is not what is expected. The minimal fix is simply to remove the unneeded \ non-escapes; this seems to be enough in my case.

Just for reference, I noticed this with nlp('today at 2:30pm').text('normal'). In development, this returns today at 2:30pm as expected. In production, I was getting today at 30pm because the 2: was treated as a starting prefix.

---

An open question is whether the expressions are what they should be, but that's a more difficult question. The code indicates that the list of punctuation comes from Wikipedia, but I don't feel that's a normative reference for punctuation. I think a more robust expression might be something like:

/(\s|\p{General_Category=Punctuation}|\p{General_Category=Symbol}|º|ª)+/u

... which is "all whitespace, all punctuation (as per Unicode), and all symbols (as per Unicode), plus the male and female ordinal characters (which are categorized as letters, sadly)". This seems to cover all of the characters from the original expressions, plus several more. It also avoids having different-but-hard-to-notice differences between the startings and endings expressions (once they are anchored as needed). If you really want different starting/ending punctuation, the open-/close-specific unicode categories can be explicitly listed in the appropriate expression.

Just a thought for a future fix/improvement. :-)

[spencermountain]

whoa! thanks Jared - I always forget what needs escaping in and out of the [] sections.
I didn't know that babel touches regex, either.

ya- what a mess those regexes are. If I can remember, there were some subtle differences between what is allowed in a starting and an ending. I'd love for some help making this stronger and altogether more sensible. Please give it a spin! change is welcome.
thanks!