Skip to content

Commit

Permalink
Update README
Browse files Browse the repository at this point in the history
  • Loading branch information
Dmitry Dygalo committed Sep 11, 2019
1 parent 7c8aa60 commit 06058bc
Showing 1 changed file with 70 additions and 40 deletions.
110 changes: 70 additions & 40 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -3,76 +3,103 @@ Schemathesis

|Version| |Python versions| |License|

Schemathesis is a tool that generates test cases for your
Open API / Swagger schemas.
Schemathesis is a tool for testing your web applications built with Open API / Swagger specifications.

The main goal is to bring property-based testing to web applications and
verify if all values allowed by the schema are processed correctly
by the application.
It reads the application schema and generates test cases which will ensure that your application is compliant with its schema.

Empowered by ``Hypothesis``, ``hypothesis_jsonschema`` and ``pytest``.
The application under test could be written in any language, the only thing you need is a valid API schema in a supported format.

Supported specification versions:
**Supported specification versions**:

- Swagger 2.0
- Open API 3.0.x

**NOTE**: The library is WIP, the API is a subject to change.
More API specifications will be added in the future.

Built with:

- `hypothesis`_

- `hypothesis_jsonschema`_

- `pytest`_

Installation
------------

To install Schemathesis via ``pip`` run the following command:

.. code:: bash
pip install schemathesis
Optionally you could install ``requests`` for convenient HTTP calls.

Usage
-----

To generate test cases for your schema you need:
To examine your application with Schemathesis you need to:

- Setup & run your application, so it is accessible via the network;
- Write a couple of tests in Python;
- Run the tests via ``pytest``.

- Create a parametrizer;
- Wrap a test with ``Parametrizer.parametrize`` method
- Provide a client and url of a running application instance
Suppose you have your application running on ``http://0.0.0.0:8080`` and its
schema is available at ``http://0.0.0.0:8080/swagger.json``.

A basic test, that will verify that any data, that fit into the schema will not cause any internal server error could
look like this:

.. code:: python
import pytest
import requests
from schemathesis import Parametrizer
schema = Parametrizer.from_path("path/to/schema.yaml")
@pytest.fixture(scope="session")
def client():
with requests.Session() as session:
yield session
BASE_URL = "http://0.0.0.0:8080"
schema = Parametrizer.from_uri(f"{BASE_URL}/swagger.json")
@schema.parametrize()
def test_users_endpoint(client, case):
url = "http://0.0.0.0:8080" + case.formatted_path
response = client.request(
def test_no_server_errors(case):
response = requests.request(
case.method,
url,
f"{BASE_URL}case.formatted_path",
headers=case.headers,
params=case.query,
json=case.body
)
assert response.status_code == 200
assert response.status_code < 500
It consists of four main parts:

1. Schema preparation; ``Parametrizer`` should be initialized with a valid schema location.

2. Test parametrization; ``@schema.parametrize()`` generates separate tests for all endpoint/method combination available in the schema.

3. A network call to the running application; ``requests`` will do the job, for example.

4. Verifying a property you'd like to test; In the example, we verify that any app response will not indicate a server-side error (HTTP codes 5xx).

**Other properties that could be tested**:

Each wrapped test will have the ``case`` fixture, that represents a
hypothesis test case.
- Any call will be processed in <50 ms - you can verify the app performance;
- Any unauthorized access will end with 401 HTTP response code;

Case consists of:
Each test function should have the ``case`` fixture, that represents a single test case.

- ``method``
- ``formatted_path``
- ``headers``
- ``query``
- ``body``
Important ``Case`` attributes:

For each ``schemathesis`` will create ``hypothesis`` strategies which will
generate bunch of random inputs acceptable by schema.
This data could be used to verify that your application works in the way
as described in the schema or that schema describes expected behaviour.
- ``method`` - HTTP method
- ``formatted_path`` - full endpoint path
- ``headers`` - HTTP headers
- ``query`` - query parameters
- ``body`` - request body

For example the data could be send against running app container via
``requests`` and response is checked for an expected status code or error
message.
For each test, Schemathesis will generate a bunch of random inputs acceptable by the schema.
This data could be used to verify that your application works in the way as described in the schema or that schema describes expected behavior.

By default, there will be 100 test cases per endpoint/method combination.
To limit the number of examples you could use ``hypothesis.settings`` decorator on your test functions:

.. code:: python
Expand All @@ -86,7 +113,7 @@ To limit the number of examples you could use ``hypothesis.settings`` decorator
Documentation
-------------

For full documentation, please see https://schemathesis.readthedocs.io/en/latest/
For the full documentation, please see https://schemathesis.readthedocs.io/en/latest/ (WIP)

Or you can look at the ``docs/`` directory in the repository.

Expand All @@ -109,4 +136,7 @@ will be licensed under its MIT license.
.. |License| image:: https://img.shields.io/pypi/l/schemathesis.svg
:target: https://opensource.org/licenses/MIT

.. _hypothesis: https://hypothesis.works/
.. _hypothesis_jsonschema: https://github.com/Zac-HD/hypothesis-jsonschema
.. _pytest: http://pytest.org/en/latest/
.. _MIT license: https://opensource.org/licenses/MIT

0 comments on commit 06058bc

Please sign in to comment.