Recursive dependency involving fixture 'celery_config' if Flask-CeleryExt is not installed

[egabancho]

Disclaimer: I might be the source of this issue fc7ee03#diff-2eeaed663bd0d25b7e608891384b7298L47

When Flask-CeleryExt is not installed by the module where pytest-invenio is used, this error appears:

file .../src/inveniosoftware/invenio-sequencegenerator/tests/test_models.py, line 167
  def test_exceptions(db):
file .../lib/python3.5/site-packages/pytest_invenio/fixtures.py, line 353
  @pytest.fixture(scope='module')
  def database(appctx):
file .../lib/python3.5/site-packages/pytest_invenio/fixtures.py, line 256
  @pytest.fixture(scope='module')
  def appctx(base_app):
file .../lib/python3.5/site-packages/pytest_invenio/fixtures.py, line 177
  @pytest.fixture(scope='module')
  def base_app(create_app, app_config, request, default_handler):
file .../lib/python3.5/site-packages/pytest_invenio/fixtures.py, line 124
  @pytest.fixture(scope='module')
  def app_config(db_uri, broker_uri, celery_config):
file .../lib/python3.5/site-packages/pytest_invenio/fixtures.py, line 95
  @pytest.fixture(scope='module')
  def celery_config(celery_config):
E       recursive dependency involving fixture 'celery_config' detected
>       available fixtures: _configure_application, _monkeypatch_response_class, _push_request_context, accept_any, accept_json, accept_jsonp, accept_mimetype, app, app_config, appctx, base_app, base_client, broker_uri, browser, cache, capfd, capfdbinary, caplog, capsys, capsysbinary, celery_config, cli_runner, client, client_class, config, cov, create_app, database, db, db_uri, default_handler, doctest_namespace, es, es_clear, instance_path, live_server, mailbox, monkeypatch, no_cover, pytestconfig, record_property, record_xml_attribute, record_xml_property, recwarn, request_ctx, script_info, tmp_path, tmp_path_factory, tmpdir, tmpdir_factory
>       use 'pytest --fixtures [testpath]' for help on them.

This happens basically because celery_config fixture doesn't exists and therefore references itself here.

One solution might be to add back flask-celeryext to the requirements, not only the test requirements, but this would make modules, which don't need celery, install it for the tests.

From the top of my head I can't find an elegant solution, perhaps someone better versed on pytest might know something I don't. All I could come up is something along this lines:

try:
    pkg_resources.get_distribution('flask-celeryext')
    @pytest.fixture(scope='module')
    def celery_config(celery_config):
        .....
except pkg_resources.DistributionNotFound:
    @pytest.fixture(scope='module')
    def celery_config():
        ....

[ntarocco]

In the setup.py I see that 'flask-celeryext>=0.3.1 is installed, so we could require it (but then, there could be issue with versions...)

If we don't want to require it, maybe better to mock it and provide a stub instead of changing the method?

try:
    pkg_resources.get_distribution('flask-celeryext')
except pkg_resources.DistributionNotFound:
    def celery_config(): 
        return {}

[egabancho]

Where would you put this code?
If I put it on fixtures.py it will get overwritten by the fixture definition and the error is the same, that's why I proposed to do it in both cases.

[egabancho]

This is the "cleanest" I could get so far :'(

def _celery_config():
    default_config = dict(
        CELERY_ALWAYS_EAGER=True,
        CELERY_CACHE_BACKEND='memory',
        CELERY_EAGER_PROPAGATES_EXCEPTIONS=True,
        CELERY_RESULT_BACKEND='cache',)

    try:
        pkg_resources.get_distribution('flask-celeryext')
        def inner(celery_config):
            celery_config.update(default_config)
            return celery_config
    except pkg_resources.DistributionNotFound:
        def inner():
            return default_config

    return inner

celery_config = pytest.fixture(
    scope='module', name='celery_config')(_celery_config())