2.0 release (#659)

user facing changes: - Move anything that isn't supposed to be public into a private folder, a la pytest - Move helpers from tavern.testutil.helpers to tavern.helpers - Make --tavern-merge-ext-function-values default - Move 'clear cookies' to a top-level request item - Change force_format_include to force_original_structure - Change to using jsonschema instead of pykwalify for tavern schema - change test_block_config to a class - Drop explicit support for pypy3 - Allow multiple MQTT responses to a single request internal changes: - Make mqtt tests run in parallel to avoid them taking a really long time - Try to use dataclasses instead of attrs where possible - More type hints
taverntesting · Jan 12, 2023 · 7c9d78f · 7c9d78f
1 parent 3974c80
commit 7c9d78f
Show file tree

Hide file tree

Showing 100 changed files with 3,275 additions and 1,394 deletions.
diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
@@ -35,9 +35,6 @@ jobs:
 
         - TOXENV: py3-generic
           TOXCFG: tox-integration.ini
-        - TOXENV: pypy3-noextra
-          python: pypy-3.7-v7.x
-          TOXCFG: tox-integration.ini
         - TOXENV: py3-mqtt
           TOXCFG: tox-integration.ini
         - TOXENV: py3-hooks

diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
@@ -1,19 +1,22 @@
 repos:
   - repo: https://github.com/ambv/black
-    rev: stable
+    rev: 22.12.0
     hooks:
       - id: black
-        language_version: python3.7
-  - repo: https://github.com/pre-commit/pre-commit-hooks
-    rev: v2.3.0
+        files: '(tavern|tests)'
+  - repo: https://github.com/PyCQA/flake8
+    rev: 6.0.0
     hooks:
-      - id: pyflakes
-        language_version: python3.7
+      - id: flake8
+        additional_dependencies: ['flake8-pyproject']
+        files: tavern/
+        exclude: tests/
   - repo: https://github.com/pre-commit/mirrors-isort
-    rev: v4.3.21
+    rev: v5.10.1
     hooks:
       - id: isort
         args:
           - --settings-path=pyproject.toml
+        files: '(tavern|tests)'
 
-exclude: (docs/|example/|tests/)
+exclude: (docs/|example/)
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -388,3 +388,47 @@ This locks pytest to <=7.2 to avoid having to fix imports every time a new versi
 This is technically not a operational change but I'm adding a new tag so it can br reverted in future
 
 ##  1.25.1          Remove tbump from dependencies so it can actually be uploaded to pypi (2022-12-13)
+
+##  1.25.2          Only patch pyyaml when a test is actually being loaded to avoid side effect from Tavern just being in the python path (2022-12-15)
+
+##  2.0.0alpha3     Bump version: 2.0.0alpha2 → 2.0.0alpha3 (2021-10-17)
+
+##  2.0.0alpha4     Bump version: 2.0.0alpha3 → 2.0.0alpha4 (2021-11-06)
+
+##  2.0.0alpha5     Bump version: 2.0.0alpha4 → 2.0.0alpha5 (2021-12-12)
+
+##  2.0.0alpha6     Bump version: 2.0.0alpha5 → 2.0.0alpha6 (2021-12-12)
+
+##  2.0.0alpha7     Bump version: 2.0.0alpha6 → 2.0.0alpha7 (2021-12-12)
+
+##  2.0.0alpha8     Bump version: 2.0.0alpha7 → 2.0.0alpha8 (2022-01-09)
+
+##  2.0.0alpha9     Allow MQTT responses to come in out of order (2022-06-12)
+
+#  2.0.0alpha10    Allow 'unexpected' mqtt messages (2022-06-26)
+
+##  2.0.0alpha11    Update 'faker' (2022-07-30)
+
+##  2.0.0alpha12    Bind all MQTT Client callbacks (2022-08-29)
+
+##  2.0.0alpha13    Support ssl_context attributes which expose alpn (2022-08-29)
+
+##  2.0.0alpha14    Allow specifying an ext function to return all parametrized vals, not just sub-vals (2022-10-01)
+
+##  2.0.0beta1      Bump version: 2.0.0alpha14 → 2.0.0beta1 (2022-10-23)
+
+##  2.0.0alpha15    Bump version: 2.0.0alpha14 → 2.0.0alpha15 (2022-10-23)
+
+##  2.0.0alpha16    Bump version: 2.0.0alpha15 → 2.0.0alpha16 (2022-10-23)
+
+##  2.0.0alpha17    Fix out of order MQTT messages and ext functions in mqtt responses (2022-11-08)
+
+##  2.0.0alpha18    Spin up separate message queue for each topic to avoid spinlocking (2022-12-09)
+
+##  2.0.0a19        2.0.0a19 (2022-12-15)
+
+#  2.0.0a20        Drop pypy3 tests (2022-12-15)
+
+##  2.0.0a21        2.0.0a21 (2022-12-19)
+
+##  2.0.0a22        Fix 'attr' issue (2022-12-19)
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
@@ -4,7 +4,8 @@ All configuration for the project should be put into `pyproject.toml`.
 
 ## Working locally
 
-1. Create a virtualenv using whatever method you like (eg, [virtualenvwrapper](https://virtualenvwrapper.readthedocs.io/))
+1. Create a virtualenv using whatever method you like (
+   eg, [virtualenvwrapper](https://virtualenvwrapper.readthedocs.io/))
 
 1. Install dependencies from requirements.txt
 
@@ -22,12 +23,36 @@ If on Windows, you should be able to just run the 'tox' commands in that file.
 
 1. Update requirements files (BOTH of them)
 
-       pip-compile --output-file - --all-extras --resolver=backtracking pyproject.toml --reuse-hashes --generate-hashes > requirements.txt
-       pip-compile --output-file - --all-extras --resolver=backtracking pyproject.toml --strip-extras > constraints.txt
+       pip-compile --all-extras --resolver=backtracking pyproject.toml --output-file requirements.txt --reuse-hashes --generate-hashes
+       pip-compile --all-extras --resolver=backtracking pyproject.toml --output-file constraints.txt --strip-extras
 
 1. Run tests as above
 
 ## Fixing formatting issue
 
     black tavern/ tests/
     isort --profile black tavern/ tests/
+
+## Pre-commit
+
+After installing dependencies, Run
+
+    # check it works
+    pre-commit run --all-files 
+    pre-commit install
+
+Run every so often to update the pre-commit hooks
+
+    pre-commit autoupdate
+
+## Creating a new release
+
+1. Setup `~/.pypirc`
+
+1. Install the correct version of tbump
+
+       pip install tbump@https://github.com/michaelboulton/tbump/archive/714ba8957a3c84b625608ceca39811ebe56229dc.zip
+
+1. Tag and push to git with `tbump <new-tag> --tag-message "<tag-message>"`
+
+1. Upload to pypi with `flit publish`
diff --git a/MANIFEST.in b/MANIFEST.in
@@ -1,3 +1,3 @@
-include tavern/schemas/tests.schema.yaml
-include tavern/_plugins/mqtt/schema.yaml
+include tavern/_core/schema/tests.jsonschema.yaml
+include tavern/_plugins/mqtt/jsonschema.yaml
 include LICENSE
diff --git a/constraints.txt b/constraints.txt
@@ -2,20 +2,19 @@
 # This file is autogenerated by pip-compile with Python 3.10
 # by the following command:
 #
-#    pip-compile --all-extras --output-file=- --resolver=backtracking --strip-extras pyproject.toml
+#    pip-compile --all-extras --output-file=constraints.txt --resolver=backtracking --strip-extras pyproject.toml
 #
 allure-pytest==2.12.0
     # via tavern (pyproject.toml)
 allure-python-commons==2.12.0
     # via allure-pytest
 astroid==2.12.13
     # via pylint
-attrs==21.4.0
+attrs==22.1.0
     # via
     #   allure-python-commons
     #   jsonschema
     #   pytest
-    #   tavern (pyproject.toml)
 bcrypt==4.0.1
     # via paramiko
 black==22.12.0
@@ -45,7 +44,7 @@ colorlog==6.7.0
     # via tavern (pyproject.toml)
 commonmark==0.9.1
     # via rich
-coverage==6.5.0
+coverage==7.0.0
     # via
     #   pytest-cov
     #   tavern (pyproject.toml)
@@ -97,17 +96,17 @@ flit-core==3.8.0
     # via flit
 fluent-logger==0.10.0
     # via tavern (pyproject.toml)
-identify==2.5.9
+identify==2.5.10
     # via pre-commit
 idna==3.4
     # via requests
-importlib-metadata==5.1.0
+importlib-metadata==5.2.0
     # via
     #   keyring
     #   twine
 iniconfig==1.1.1
     # via pytest
-isort==5.11.1
+isort==5.11.3
     # via
     #   pylint
     #   tavern (pyproject.toml)
@@ -129,7 +128,7 @@ jsonschema==3.2.0
     # via
     #   docker-compose
     #   tavern (pyproject.toml)
-keyring==23.11.0
+keyring==23.13.1
     # via twine
 lazy-object-proxy==1.8.0
     # via astroid
@@ -170,7 +169,7 @@ pbr==5.11.0
     # via stevedore
 pep517==0.13.0
     # via build
-pip-tools==6.11.0
+pip-tools==6.12.1
     # via tavern (pyproject.toml)
 pkginfo==1.9.2
     # via twine
@@ -205,7 +204,7 @@ pyjwt==2.6.0
     # via tavern (pyproject.toml)
 pykwalify==1.8.0
     # via tavern (pyproject.toml)
-pylint==2.15.8
+pylint==2.15.9
     # via tavern (pyproject.toml)
 pynacl==1.5.0
     # via paramiko
@@ -254,7 +253,7 @@ ruamel-yaml==0.17.21
     # via pykwalify
 ruamel-yaml-clib==0.2.7
     # via ruamel-yaml
-ruff==0.0.178
+ruff==0.0.187
     # via tavern (pyproject.toml)
 secretstorage==3.3.3
     # via keyring
@@ -290,7 +289,7 @@ tomli-w==1.0.0
     # via flit
 tomlkit==0.11.6
     # via pylint
-tox==3.27.1
+tox==3.28.0
     # via
     #   tavern (pyproject.toml)
     #   tox-travis

diff --git a/docs/source/basics.md b/docs/source/basics.md
@@ -337,7 +337,7 @@ changed to use a different library to avoid this issue.**
 # valid jwt which is signed by the given key.
 response:
   verify_response_with:
-    function: tavern.testutils.helpers:validate_jwt
+    function: tavern.helpers:validate_jwt
     extra_kwargs:
       jwt_key: "token"
       key: CGQgaG7GYvTcpaQZqosLy4
@@ -355,7 +355,7 @@ body of the response against it.
 # which has to contain a user name and may contain a user number.
 response:
   verify_response_with:
-    function: tavern.testutils.helpers:validate_pykwalify
+    function: tavern.helpers:validate_pykwalify
     extra_kwargs:
       schema:
         type: seq
@@ -415,6 +415,7 @@ This can be used as so:
   request:
     url: http://server.com/login
     headers:
+      x-my-header: abc123
       $ext:
         function: utils:generate_bearer_token
     json:
@@ -517,7 +518,7 @@ For example, if our server saves the user ID in the 'sub' field of the JWT:
     status_code: 200
     verify_response_with:
       # Make sure a token exists
-      function: tavern.testutils.helpers:validate_jwt
+      function: tavern.helpers:validate_jwt
       extra_kwargs:
         jwt_key: "token"
         options:
@@ -527,7 +528,7 @@ For example, if our server saves the user ID in the 'sub' field of the JWT:
       # in the test configuration for use in future tests
       # Note the use of $ext again
       $ext:
-        function: tavern.testutils.helpers:validate_jwt
+        function: tavern.helpers:validate_jwt
         extra_kwargs:
           jwt_key: "token"
           options:
@@ -949,7 +950,7 @@ stages:
         json:
           test_user_login_token: token
       verify_response_with:
-        function: tavern.testutils.helpers:validate_jwt
+        function: tavern.helpers:validate_jwt
         extra_kwargs:
           jwt_key: "token"
           options:
@@ -1503,7 +1504,7 @@ third block must start with 4 and the third block must start with 8, 9, "A", or
 ```
 
 This is using the `!re_fullmatch` variant of the tag - this calls
-[`re.fullmatch`](https://docs.python.org/3.7/library/re.html#re.fullmatch) under
+[`re.fullmatch`](https://docs.python.org/3.8/library/re.html#re.fullmatch) under
 the hood, which means that the regex given needs to match the _entire_ part of
 the response that is being checked for it to pass. There is also `!re_search`
 which will pass if it matches _part_ of the thing being checked, or `!re_match`
@@ -1532,7 +1533,7 @@ stages:
         hash: 456
     save:
       $ext:
-        function: tavern.testutils.helpers:validate_regex
+        function: tavern.helpers:validate_regex
         extra_kwargs:
           expression: "v(?P<version>[\d\.]+)-[\w\d]+"
           in_jmespath: "meta.version"
@@ -1599,7 +1600,7 @@ request:
 
 Sometimes there are situations where you need to directly include a block of
 JSON, such as a list, rather than just one value. To do this, there is a
-`!force_format_include` tag which will include whatever variable is being
+`!force_original_structure` tag which will include whatever variable is being
 referenced in the format block rather than coercing it to a string.
 
 For example, if we have an API that will return a list of users on a GET and
@@ -1631,7 +1632,7 @@ could be done by
       url: "{host}/users"
       method: DELETE
       # 'all_users' list will be sent in the request as a list, not a string
-      json: !force_format_include "{all_users}"
+      json: !force_original_structure "{all_users}"
     response:
       status_code: 204