Replace noqa directive parser with an actual lexer

[charliermarsh]

In #5567 and related PRs, I moved away from regular expressions and exact matches to a more token-based approach to noqa parsing. There are two codepaths here, one for # noqa: F401 (inline exemptions) and another for # ruff: noqa: F401 (whole-file exemptions). We should replace both of these with an actual lexer to lex directives, and then match against the resulting token stream. That would likely simplify the code a lot and allow us to unify the implementations.

[charliermarsh]

Low-priority. We could in theory though use this same lexer for (e.g.) # isort: on directives. We could even use this approach to support autoformatting of directives.