Best practices for flask-testing setup

[Birne94]

I currently consider using tavern for testing our backend services and would appreciate some of your thoughts.

Is there any recommended way of setting up a (local) testing environment with flask? Flask itself provides a test client which can be used in unit tests.

Since spinning up a real test server running on localhost brings other issues with it (clean shutdown, port may already be used etc.), I would like to fall back to this test client.

As far as I can tell, tavern currently only supports requests using the requests (+ mqtt, which is not relevant to flask) module. I suppose one could statically add flask test-client support in the taverns.plugins module, but this seems a little bit hacky.

• Does anybody have experience in using tavern with a local flask setup, preferrably without spinning up a local server?
• Are there any plans and/or development ongoing regarding the plugin system?
• If no, is there any set of requirements or specification regarding a plugin system? If we decide to use tavern ourselves, contributing here might be of interest.

[michaelboulton]

Just to clarify about testing flask with the test client - do you basically want to run flask tests that would be something like

test_client = app.test_client()
response = test_client.get("/scan", headers={"authorization": "bearer abc123"})
assert response.status_code == 200

but written in YAML

request:
  url: www.example.com/scan
  method: GET
  headers:
    authorization: bearer abc123
response:
  status_code: 200

Like that?

[Birne94]

Yes. I want to be able to omit the protocol://host:port part of request.url and would like to pass it to the test client instead.

[Birne94]

Some more thoughts about how the test schema might look:

test_name: test
test_request_class:
  class: tavern.plugins.flask.FlaskTestRequest
  extra_kwargs:
    flask_app: myapp.factory.app

stages:
  - name: test
    request:
      url: /users/1/
      method: GET
    response:
      status_code: 200

FlaskTestRequest takes a Flask app instance or an absolute import path as parameter and instantiates it. All requests are then performed using this class.

test_request_class.class would default to the current RestRequest class, which of course might need some changes to be compatible. MQTT support could be adapted in a similar way.

One might decide to allow for full urls (including protocol and host) to be performed through requests, but this might be an implementation detail.

[michaelboulton]

I wrote up a (rough) spec of what the plugin system is likely to look like here: #41 . Not sure when this will get implemented though.

The relevant parts with this is that the 'session' is probably just a context manager that yields the flask test client, the 'request' block calls test_client.post etc., and then most of the RestResponse can probably be reused for the 'verifier'. a (possible) example:

# flask_tavern_plugin.py
import contextlib

@contextlib.contextmanager
def get_session(app_module):
    from tavern import import_ext_function
    app = import_ext_function(app_module)
    yield app

# etc...

---
test_name: test local flask server works

plugins:
  flask:
    app_module: mymodule.server:app

stages:
  - name: post something
    flask_request:
      url: www.example.com/scan
      method: GET
      headers:
        authorization: bearer abc123
    response:
      status_code: 200

[WhileLoop]

@Birne94

I managed to hack together a live server method that also produces code coverage reports here. https://github.com/WhileLoop/flask-tavern-coverage

The only catch is that you have to run a non tavern test first for the fixture to start and the server can't be in debug mode. Let us know if you come up with anything better.

[michaelboulton]

I've run into that before, fixtures in pytest are only triggered when it starts a python test, not any 'special' kinds of tests (like the YamlItem that Tavern returns). In the pytest source code it only seems to use fixtures when pytest-style (ie, not unittest) python tests are run. This quick hack does make autouse fixtures work at least

diff --git a/tavern/testutils/pytesthook.py b/tavern/testutils/pytesthook.py
index eef1c5e..e5638ba 100644
--- a/tavern/testutils/pytesthook.py
+++ b/tavern/testutils/pytesthook.py
@@ -54,6 +54,7 @@ class YamlFile(pytest.File):
 
             yield YamlItem(test_spec["test_name"], self, test_spec, self.fspath)
 
+from _pytest import fixtures
 
 class YamlItem(pytest.Item):
 
@@ -73,6 +74,18 @@ class YamlItem(pytest.Item):
         self.path = path
         self.spec = spec
 
+        self.cls = None
+        self.obj = object()
+        self.funcargs = []
+        self._fixtureinfo = self.session._fixturemanager.getfixtureinfo(
+            self.parent, self.obj, self.cls,
+            funcargs=None)
+        self._request = fixtures.FixtureRequest(self)
+
+    def setup(self):
+        super(YamlItem, self).setup()
+        fixtures.fillfixtures(self)
+
     def runtest(self):
         verify_tests(self.spec)

[michaelboulton]

@Birne94 If you're still interested in this I have done a quick implementation of a tavern flask plugin. You can use the feature/plugins branch of this repo along with https://github.com/taverntesting/tavern-flask, which has an example/ folder

[Birne94]

@michaelboulton thanks for the heads up. I have been following this project's updates and changes for the past weeks, but haven't gotten around trying the new features out.

It looks promising so far, I will play around with the plugin a bit in the next days.

[shuttle1987]

Currently I'm using Flask and I'd like to use Tavern for some tests because it seems great. Is it possible to use the flask test client now?

In my conftest.py I have something like this:

@pytest.fixture
def client(tmpdir):
    """Create a test client to send requests to"""
    flask_app.config['BASE_UPLOAD_DIRECTORY'] = os.path.join(str(tmpdir), 'test_uploads')

    from api.upload_config import configure_uploads
    configure_uploads(flask_app)
    with flask_app.test_client() as c:
        yield c

Is there some supported way of using this with Tavern?