[BUG] Examples for request body ignored during testcase generation

[hoog1511]

Describe the bug

Yesterday I was testing an OAS which had the following pattern ^(?=[[:ascii:]]+$)[^\/\\\s$]{1,36}$ and for some reason this caused schemathesis to ignore the examples I provided for my request body.

To Reproduce
Steps to reproduce the behavior:

1. Add the pattern to a header parameter.
2. Add an example request body to one of your requests which uses said header parameter.
3. Run schemathesis (I used CLI version)

Expected behavior

Examples being included with my testcases.

Environment (please complete the following information):

• OS: [Windows]
• Python version: [3.11.0]
• Schemathesis version: [3.19.5]
• Spec version: [3.0.3]

[Stranger6667]

The reason is that the pattern contains parts that don't work as one might expect in Python, specifically :ascii:. This character class is specific to PCRE, but Python's regex engine treats [[:ascii:]] as matching a character in the [:asci list, similarly ECMAScript treats it the same.

We use Python's regex with some pre-processing so it supports some parts from ECMAScript, and in general, it is not possible to unambiguously detect which regex style is intended, because some patterns, such as the one you provided are valid in multiple regex engines but mean different things. The Open API spec implies ECMAScript, so Schemathesis supports it on a best-effort basis. The proper solution is possible with the use of greenery and transforming regular expressions to the target style. Before this happens, there will be counter-examples that won't work as expected.

So, from the Schemathesis POV, some schemas with regex are hard to generate because in many scenarios we fall back to the filtering-based approach and Hypothesis raises Unsatisfiable. In this scenario, we skip example generation at all as it is likely that the test itself will fail with Unsatisfiable (not always, but with a decent probability).

Recently, Hypothesis became better at generating strings from regexes, so there are fewer cases when we fall back into this.

However, it is still confusing that e.g. with --hypothesis-phase=explicit some endpoints with examples will be skipped without a note.

So, in this specific case, I think rewriting the pattern to the equivalent one in the ECMAScript / Python syntax should solve the issue, otherwise we can raise an error / warning to inform the user about it.