Skip to content


Switch branches/tags

Name already in use

A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch?

Latest commit


Git stats


Failed to load latest commit information.
Latest commit message
Commit time

Hultnér Technologies

Hultnér Technologies AB | @ahultner | Blog | Slides | Nordisk Python Community | GitHub

⠠⠵ Schema-based-API-Testing

Automatically generate test-cases based on your API-schemas.

The goal with this talk is to introduce the audience to property-based testing for API’s using schemas to automatically generate test scenarios, enabling them to write more powerful tests faster.

The talk will focus on a subset of the field of property-based testing where we focus on testing by automatically generating properties and test strategies from the API Schemas that we often already have. These tests ensure that our APIs conform to their specified schema and enables us to write a much larger amount of tests in less time.

I will focus on the schemathesis library which leverages the strong hypothesis library as well as the hypothesis-JSONSchema extension strategies, and will in the future also support GraphQL via the hypothesis-graphql strategies. I’m a contributor to schemathesis and currently working on the future GraphQL support with schemathesis creator, Dmitry Dygalo.

I will also compare it with its predecessor “swagger-conformance”, pure property-based testing through hypothesis, schema strategies with hypothesis-graphql and hypothesis-jsonschema, and discuss their advantages and disadvantages. I will also briefly talk about “QuickREST: Property-based Test Generation of OpenAPI-Described RESTful APIs”, a research paper that’s inspired part of the functionality in these tools.

By focusing on property-based test generation using schemas we already have, I will show that a field like property-based testing, which can seem quite daunting at first, actually can have a low barrier to entry while yielding large amounts of value in return and is useful for most common web projects today.

The talk will show how formal schemas for APIs can and will continue to provide additional value outside the scope of documentation.

⠠⠵ Quick reference


$ schemathesis run
================== Schemathesis test session starts ==================
platform …
rootdir: /~/dev/hultner_technologies/Schema-based-API-Testing/code
hypothesis profile 'default' -> database=Directory…
Schema location:
Base URL:
Specification version: Swagger 2.0
Workers: 1
collected endpoints: 5

GET /todos/ .                                                  [ 20%]
POST /todos/ .                                                 [ 40%]
DELETE /todos/{id} .                                           [ 60%]
GET /todos/{id} .                                              [ 80%]
PUT /todos/{id} .                                              [100%]

============================== SUMMARY ===============================

Performed checks:
    not_a_server_error                    500 / 500 passed          PASSED

========================= 5 passed in 5.32s ==========================

Via pytest

# Minimal usage example
import requests
import schemathesis

schema = schemathesis.from_uri("")

def test_no_server_errors(case):
    # `requests` will make an appropriate call under the hood
    response =  # use `call_wsgi` if you used `schemathesis.from_wsgi`
    # You could use built-in checks
    # Or assert the response manually
    assert response.status_code < 500


⠠⠵ FlaskCon, 2020 July 5

⠠⠵ EuroPython, 2020 July 23

⠠⠵ PiterPy, 2020 August 4

  • Type: Talk (40 mins)
  • Python level: Intermediate
  • Domain level: Intermediate
  • Schedule

⠠⠵ FAQ, Schemathesis

Stateful testing?
Yes, using links.

In-progress, check link for latest progress.

OpenAPI, Swagger?
Yes, Schemathesis supports both the older Swagger 2.0 standard and the newer OpenAPI 3 standard

Incompabilities? Can be addressed using fixups, a built in fixup for FastAPI is included.
Activate in CLI with: --fixups=all

Do you have to use the network? No, Schemathesis can import your app directly and hit it without using the network if you use either WSGI or ASGI.

I have a question not covered here, where can I ask it?
I'm @ahultner on twitter, otherwise you can also email me (address in slides).

⠠⠵ Links


Talk: Automatically generate test-cases based on your API-schemas.







No releases published


No packages published