Add a Hypothesis plugin (#2097)

* Configure Hypothesis

* Hypothesis plugin docs

* Add Hypothesis plugin

Co-authored-by: Samuel Colvin <s@muelcolvin.com>

.gitignore

@@ -16,6 +16,7 @@ dist/
 .mypy_cache/
 test.py
 .coverage
+.hypothesis
 /htmlcov/
 /benchmarks/*.json
 /docs/.changelog.md

changes/2097-Zac-HD.md

@@ -0,0 +1 @@
+Add a [Hypothesis](https://hypothesis.readthedocs.io/) plugin for easier [property-based testing](https://increment.com/testing/in-praise-of-property-based-testing/) with Pydantic's custom types - [usage details here](https://pydantic-docs.helpmanual.io/hypothesis_plugin/)

docs/examples/hypothesis_property_based_test.py

@@ -0,0 +1,24 @@
+import typing
+from hypothesis import given, strategies as st
+from pydantic import BaseModel, EmailStr, PaymentCardNumber, PositiveFloat
+
+
+class Model(BaseModel):
+    card: PaymentCardNumber
+    price: PositiveFloat
+    users: typing.List[EmailStr]
+
+
+@given(st.builds(Model))
+def test_property(instance):
+    # Hypothesis calls this test function many times with varied Models,
+    # so you can write a test that should pass given *any* instance.
+    assert 0 < instance.price
+    assert all('@' in email for email in instance.users)
+
+
+@given(st.builds(Model, price=st.floats(100, 200)))
+def test_with_discount(instance):
+    # This test shows how you can override specific fields,
+    # and let Hypothesis fill in any you don't care about.
+    assert 100 <= instance.price <= 200

docs/hypothesis_plugin.md

@@ -0,0 +1,28 @@
+[Hypothesis](https://hypothesis.readthedocs.io/) is the Python library for
+[property-based testing](https://increment.com/testing/in-praise-of-property-based-testing/).
+Hypothesis can infer how to construct type-annotated classes, and supports builtin types,
+many standard library types, and generic types from the
+[`typing`](https://docs.python.org/3/library/typing.html) and
+[`typing_extensions`](https://pypi.org/project/typing-extensions/) modules by default.
+
+From Pydantic v1.8 and [Hypothesis v5.29.0](https://hypothesis.readthedocs.io/en/latest/changes.html#v5-29-0),
+Hypothesis will automatically load support for [custom types](usage/types.md) like
+`PaymentCardNumber` and `PositiveFloat`, so that the
+[`st.builds()`](https://hypothesis.readthedocs.io/en/latest/data.html#hypothesis.strategies.builds)
+and [`st.from_type()`](https://hypothesis.readthedocs.io/en/latest/data.html#hypothesis.strategies.from_type)
+strategies support them without any user configuration.
+
+
+### Example tests
+```py
+{!.tmp_examples/hypothesis_property_based_test.py!}
+```
+_(This script is complete, it should run "as is")_
+
+
+### Use with JSON Schemas
+To test client-side code, you can use [`Model.schema()`](usage/models.md) with the
+[`hypothesis-jsonschema` package](https://pypi.org/project/hypothesis-jsonschema/)
+to generate arbitrary JSON instances matching the schema.
+For web API testing, [Schemathesis](https://schemathesis.readthedocs.io) provides
+a higher-level wrapper and can detect both errors and security vulnerabilities.

docs/requirements.txt

@@ -1,6 +1,7 @@
 ansi2html==1.6.0
 flake8==3.8.4
 flake8-quotes==3.2.0
+hypothesis==5.44.0
 mkdocs==1.1.2
 mkdocs-exclude==1.0.2
 mkdocs-material==6.2.8

mkdocs.yml

@@ -46,6 +46,7 @@ nav:
 - benchmarks.md
 - 'Mypy plugin': mypy_plugin.md
 - 'PyCharm plugin': pycharm_plugin.md
+- 'Hypothesis plugin': hypothesis_plugin.md
 - 'Code Generation': datamodel_code_generator.md
 - changelog.md