Architecture for sharing processed data across rules

[mpacer]

If we need to use more computationally intense analyses in multiple rules (e.g., nltk & syntax parsing to identify whether •while• is being used as a conjunction or a adverb) it would make more sense to memoize the output so that it can be accessed by other rules rather than rerun.

This should be a fairly general system that automatically builds the data structures so that they can be shared across individual checks, possibly with some kind of a require type statement being present in more than one rule?

[suchow]

Commit 3ae3d83 introduces a framework for memoization, allowing you to define shared processing functions like this:

from memoize import memoize


@memoize
def reverse(text):
    return text[::-1]

and then call them from within the check function of a particular linting rule:

from proselint.reverse import reverse


def check(text):

    reversed_text = reverse(text)

    error_code = "PL000"
    msg = "First line always has an error."

    return [(1, 1, error_code, msg)]

One of the nice things about this system is that each linter rule includes all the data transformations it requires (e.g., maybe someday there will be parse_tree = parse(text)), but because the same text is passed to all the linters, the parsing is run only the very first time that it's requested by a linter rule.

I think this closes this issue because it implements a minimal version of what's requested.

Does that work for you?