Teach the Ghostwriter about @staticmethod and @classmethod methods

[Zac-HD]

The Hypothesis Ghostwriter currently only writes tests for functions, because they're a natural starting point for property-based tests and the primary design goal is to make it incredibly easy to get started.

It's also usually unclear what properties you could guess for classes or instances, but in conversation at PyCon today we worked out that there are a few special cases in which we can test something after all:

• a @staticmethod is just like a function (albeit glued onto a class), so we could treat it just like we do any other function, replacing the dot with triple-underscore for name-based analyses.
• @classmethods are similar in that they don't require a pre-build instance. In addition to the "fuzz" tests which just check for internal errors, we can also:
  • assert that from_* classmethods return an instance of the class (by strong convention)
  • and assert that the matching to_* method (if one exists) inverts this.

[Cheukting]

This looks really interesting. Do you mind if I dip my toe into it? (I don't know if it will be a good issue for me but I am happy to spend time on it, tell me if I should go something else instead)

[Zac-HD]

Go for it! Just expect to also need a while to read through the module, there's A Lot Of Code. You might also find it helpful to read "callers to callees" rather than "top to bottom", starting with the equivalent function? Also remember that you don't need to understand the whole thing to make useful changes to it! (yay tests 😁)

[Cheukting]

quick report of what I found by testing the equivalent function. It actually does not gives an error when using a staticmethod or classmethod directly:

from hypothesis.extra import ghostwriter

def just_func(variable: int):
    return varibale

class A:

    @staticmethod
    def stat_func(variable: int):
        return varibale

    @classmethod
    def class_func(cls, variable: int):
        return varibale

print(ghostwriter.equivalent(A.class_func, just_func))
print("================")
print(ghostwriter.equivalent(A.class_func, just_func))

Will gives:

# This test code was written by the `hypothesis.extra.ghostwriter` module
# and is provided under the Creative Commons Zero public domain dedication.

from hypothesis import given, strategies as st


@given(variable=st.integers())
def test_equivalent_A_class_func_just_func(variable):
    result_class_func = A.class_func(variable=variable)
    result_just_func = just_func(variable=variable)
    assert result_class_func == result_just_func, (result_class_func, result_just_func)

================
# This test code was written by the `hypothesis.extra.ghostwriter` module
# and is provided under the Creative Commons Zero public domain dedication.

from hypothesis import given, strategies as st


@given(variable=st.integers())
def test_equivalent_A_class_func_just_func(variable):
    result_class_func = A.class_func(variable=variable)
    result_just_func = just_func(variable=variable)
    assert result_class_func == result_just_func, (result_class_func, result_just_func)

So I guess for the 1st task it just need to rename the test to test_equivalent_A___class_func_just_func? (I don't quite get where we need to replace the dot)
and the 2nd task to add in the extra checks?

[Cheukting]

It got more complicated when using with CLI as . is always interperted as module. I guess the triple underscore you mean here? so someone should type A___class_func instead of A.class_func?

[Zac-HD]

• triple-underscore is for this part of the magic() ghostwriter, where it's trying to detect round-trips based on the name.
  • Actually, we don't need to replace the dot, we just need to make sure that we include the class name as well as the method name. (otherwise if we had two classes with a from_dict() classmethod in the same module, we'd only test one of them!)
  • Similarly, we'll need to improve the 'magic discovery' for a module, so that it checks classes for testable methods
  • Based on a quick glance, I think the current system seems fine for test names.
• Yep, the CLI seems a bit tricky; we can revisit that in a future PR.
• FYI, here's how to syntax-highlight code blocks on GitHub; I tend to use python and python-traceback a lot (and they don't have to be fully valid to highlight well, either, so the latter works for pytest outputs)

[Cheukting]

Ah sorry, I actually know the syntax highlighting but am too lazy to use it. My bad

[Cheukting]

For the CLI stuff actually I was working on it so don't mind including it