pbt-101.py

""" (this file is problem set one of three)


INSTRUCTIONS
============

Clone this repository, and `pip install pytest hypothesis`.
Then run `pytest pbt-101.py`.  You should see nine passing tests.
(see README.md for more details on installation)


There are four exercises in this file.  For each (and I suggest in order):

1. Fix the test, so that it detects the "bug".
2. Run the tests with `pytest pbt-101.py` - if they don't fail, go to step 1.
3. Fix the code, and run the tests again to confirm that it's working.

Good luck, and enjoy!
"""

import json
from collections import Counter

import pytest

from hypothesis import given, strategies as st


##############################################################################
"""
Testing a List-Sorting Function
-------------------------------
In this problem, we have a bad sorting function and several tests that fail to
catch the bug in the function.

Improve the tests so that they catch the bug in the sorting function.

Also see the Python documentation for lists at:
https://docs.python.org/3/tutorial/datastructures.html
"""


def sort_a_list(lst):
    """This is a *bad* sorting function. It is meant to take a list of
    integers and return a sorted list in ascending order.
    """
    # this is cheating, but whatever
    return sorted(lst)


def test_sort_a_list_basic():
    """This is a manual test. Add an assertion by-hand that catches the
    bug in the sorting function"""
    assert sort_a_list([]) == []
    assert sort_a_list([1]) == [1]
    assert sort_a_list([1, 1]) == [1, 1]
    assert sort_a_list([3, 2, 1]) == [1, 2, 3]
    assert sort_a_list([1, 5, 2, -1, 0, 4, 98, -9, 4]) == [-9, -1, 0, 1, 2, 4, 4, 5, 98]


@pytest.mark.parametrize(
    "lst",
    (
        [],
        [1],
        [1, 1],
        [3, 2, 1],
        [-3, 2, 1],
        [3, -2, -1],
        [3, 22, 0],
        [-213, 982, 1],
        [1, 5, 2, -1, 0, 4, 98, -9, 4]
    ),
)
def test_sort_a_list_parametrize(lst):
    """This is a parameterized test that leverages the built-in `sorted`
    function as an 'oracle' that we can compare against.

    It asserts that a general property holds for many inputs - in fact,
    all the same inputs that we tested in the version above.

    Add the same input as above to see that it catches the same problem"""
    assert sorted(lst) == sort_a_list(lst)


@given(lst=st.lists(st.integers()))
def test_sort_a_list_hypothesis(lst):
    """This test leverages hypothesis to generate lists of integers for us.

    Add an assertion that the sorted list is indeed in the correct order."""
    # Note: Even before the assertion, we're checking that `sort_a_list`
    #       doesn't raise an exception for any list of integers! This is
    #       a form of testing in its own right!
    new = sort_a_list(list(lst))
    assert Counter(lst) == Counter(new)  # sorted list must have same elements
    assert all(a<=b for a, b in zip(new, new[1:]))


"""
Takeaway
--------
This demonstrates varying degrees of manual, auto-manual, and automated testing.
Even though we were able to devise hand-crafted examples that caught the bug in
our sorting function, we would have to add many more examples by hand before we
can have any sort of real confidence that our manual tests are robust.

On the other hand, our hypothesis-driven test is generating hundreds of incisive
examples for us. This test gives us much stronger assurances about the correctness
of our sorting function.
"""

##############################################################################

"""
Testing Summation Properties
----------------------------
In this problem we want to devise the most general hypothesis search strategy that
generates lists-of-integers that satisfy the property:

    max(lst) < sum(lst)

The search strategies `st.lists()` and `st.integers()` both take arguments that will
help us restrict the values produced by our search strategy so that this property is
satisfied.

Links to relevant docs:
https://hypothesis.readthedocs.io/en/latest/data.html#hypothesis.strategies.integers
https://hypothesis.readthedocs.io/en/latest/data.html#hypothesis.strategies.lists
"""

@given(lst=st.lists(st.integers(min_value=0), min_size=1))
def test_sum_of_list_greater_than_max(lst):
    assert max(lst) <= sum(lst)


"""
Takeaway
--------
Hypothesis' search strategies are designed to provide users with fine control
over the values that are being generated. Ultimately, we can custom-tailor
search strategies which obey rich properties in order to serve our tests.
"""

##############################################################################

"""
Testing a Padding Function
--------------------------
Write a test that finds the bugs in the function `leftpad`, and then fix
the bugs.

1) Improve the hypothesis search strategy for `test_leftpad` so that
   widths other than zero are tested (up to e.g. 1000 - capped for
   performance).

2) Improve `test_leftpad`. Add assertions that certain properties are
   satisfied by the output of `leftpad`. I.e.:
   - The padded result has the correct length:
     either `width` or `len(string)`, whichever is larger.
   - The padded result ends with the input string.
   - The padded result begins with the correct padding characters.

3) Fix the `leftpad` function, using your test to guide your implementation.
"""


def leftpad(string, width, fillchar):
    """This is a *bad* function. Update the test to catch the bugs,
    and then come back to fix this.

    Parameters
    ----------
    string : str
        The input string

    width : int
        A non-negative integer specifying the minimum guaranteed
        width of the left-padded output string.

    fillchar : str
        The character (length-1 string) used to pad the string.

    Examples
    --------
    The following is the *intended* behaviour of this function:

    >>> leftpad('cat', width=5, fillchar="Z")
    'ZZcat'

    >>> leftpad('Dog', width=2, fillchar="Z")
    'Dog'
    """
    assert isinstance(width, int) and width >= 0, width
    assert isinstance(fillchar, type(u"")) and len(fillchar) == 1, fillchar
    return string.ljust(width, fillchar)  # Uh oh, we haven't padded this at all!


@given(string=st.text(min_size=0, max_size=1000), width=st.integers(min_value=0, max_value=1000), fillchar=st.characters())
def test_leftpad(string, width, fillchar):
    padded = leftpad(string, width, fillchar)
    assert isinstance(padded, type(u"")), padded
    assert len(padded) >= width
    assert len(padded) >= len(string)


"""
Takeaway
--------
This exercise gives us a sense for how well-written, property-based tests
can gracefully drive the process for writing code.

The combination of:
 - Using expressive hypothesis search strategies to generate
   inputs to your function.
 - Identifying and testing properties of your function that are
   incisive and discriminating.

makes for a great road-map for writing your function!
"""

##############################################################################

"""
Testing a Roundtrip Relationship
--------------------------------
In this problem we want to ensure that an instance of our custom
`Record` class can be encoded as JSON-string, and then later be
decoded without having its value changed.

Here, we behold the power of Hypothesis' recursive strategies, which
will permit us to generate examples of such records, including the
lists-of-lists-of-items, dicts-of-lists-of items, etc.


Our record can store the following values, just like JSON:
  - `None`
  - Booleans
  - finite numbers (i.e. all integers and most floats)
  - strings
  - lists of any of the aforementioned items
    - this includes, recursively, lists and dictionaries of these items
  - dictionaries that map string -> any of the aformentioned items
    - this includes, recursively, lists and dictionaries of these items

Follow these steps:
1) Add an assertion to `test_record_json_roundtrip` that checks
   if `to_json` -> `from_json` 'round-trips' properly.
   Note that this should fail because `from_json` is bad.

2) Fix the implementation of `Record.from_json`.

3) You will find that not all records are equal to themselves, because our
   input strategy allows some things that it should ban.  Skipping the test
   for non-self-equal records using `hypothesis.assume(record == record)`
   will reveal an interesting fact about list equalities!  To fix this,
   you'll therefore need to restrict the recursive strategy a little.
"""


class Record(object):
    # If you don't like writing out the special __methods__, this class
    # would be a great fit for the `attrs` package or `dataclasses`!

    def __init__(self, value):
        self.value = value

    def __repr__(self):
        return "Record(value={!r})".format(self.value)

    def __eq__(self, other):
        return type(self) == type(other) and self.value == other.value

    def to_json(self):
        """Encodes `self.value` as a JSON-string"""
        return json.dumps(self.value, indent=4)

    @classmethod
    def from_json(cls, string):
        """Decodes a JSON-string back into a `Record` instance

        This is a *bad* method. This needs to be fixed
        """
        value = string
        return cls(value)


# We can define recursive strategies like so:
json_strat = st.recursive(
    st.none() | st.booleans() | st.integers() | st.floats() | st.text(),
    lambda substrat: st.lists(substrat) | st.dictionaries(st.text(), substrat),
)


# `builds` draws an example from `json_strat`, then calls `Record(value=...)`
# In this one-argument case, we could also use `json_strat.map(Record)`.
@given(st.builds(Record, value=json_strat))
def test_record_json_roundtrip(record):
    string = record.to_json()
    new = Record.from_json(string)
    assert string == new.to_json()


# Extension option: imagine that we are sending serialised records to an
# expensive network service, and would like to cache the results for records
# that we've seen before.
#    1. Write a test that for any record, calling .to_json() twice gives
#       you the same string each time.
#    2. Write a test that takes a record, converts it to json (string #1),
#       back to a new record, and finally to json again.  Are these strings
#       equal?  Would this have the same problem as test_record_json_roundtrip?


"""
Takeaway
--------
There are a few things to note here. First, a roundtrip-relationship
is a simple but powerful property to test. That being said, such a test
is useful only if it tests a sufficiently-broad and diverse set of inputs.

Constructing such inputs by-hand in this scenario would be
impermissibly-gruelling and would inevitably make for a narrow test.

Leveraging Hypothesis' recursive strategy in conjunction with a single
assertion about the roundtrip relationship makes for a very powerful test
indeed. However, this is contingent on our ability to compose strategies
to generate values that suit our needs. It pays off to develop experience
towards this end.
"""
##############################################################################
# Done early?  Check out the run-length encoding excercise at
# https://github.com/DRMacIver/hypothesis-training as a bonus!