Q: overriding settings

[farcepest]

Is there a recommended way to override settings for the live server? Normally I'd use @override_settings(DEBUG=True) so I can see tracebacks while debugging, but had to settle for this:

def before_scenario(context, scenario):
    if "DEBUG" in scenario.tags:
        from django.conf import settings
        settings.DEBUG = True

(and corresponding after_scenario())

It works, but is there a better way?

[bittner]

Not built-in into behave-django's code base, as far as I know. But your code looks just like something that wants to be turned into a decorator. Would you mind preparing a pull request?

(Don't shoot me for guessing!)

Have you tried Django's @override_settings? I'm really just guessing that it could work, because we run a LiveServerTestCase. I even think I blindly tried using it with pytest-django and I remember it may have worked, not sure though. Their project has that feature on the wish list too, btw. This is definitely something we all need for testing Django applications seriously.

EDIT: Oh, you said it didn't work. -- What was the problem then?

[farcepest]

I'm not sure how @override_settings would get applied to LiveServerTestCase in a behave test though. The environment seems like the place to do it, but usually you'd apply it to your class or a method, and in the behave-django case, we've got an instance on context.test, and wrapping that like context.test = override_settings(DEBUG=True)(context.test) doesn't seem to do it. Maybe I'm overlooking some way of applying decorators here, but it seems like you'd have to do this before you create the behave_django.testcase.BehaviorDrivenTestCase instance, and the live server starts in the setupClass() anyway, IIRC.

[graingert]

@farcepest are you still alive?

[froddd]

Has there been any further guidance on that? Seems like it would be a handy thing to have, current solution is hacky at best!

[bittner]

Reading @farcepest's #13 (comment) above I feel it should be easy to write a decorator that does exactly what the 3 lines of code above do. Kind of:

from behave_django.decorators import override_settings

@override_settings(DEBUG=True)
def before_scenario(context, scenario):
    pass

That could be a handy addition. The implementation details of that decorator can then be improved incrementally as soon as someone has time to think of "a better solution". If the decorator worked both in the environment and the actual test implementations that would be cool (bonus points!). 🥇

Is anyone of you guys willing to contribute such a decorator in a PR?

[froddd]

That would be fine for overriding settings for all scenarios, but I think the primary need is to override them per scenario.

I've been struggling to find a way of writing a step that would achieve that -- this having to do with the state of the app, it would make sense to be able to use a 'Given' step.

[farcepest]

It's not easy. If you're using unittest, you can use @override.settings(DEBUG=True) on a test method, and then it applies to just that test method. Or you can apply it to the class. Applying it to before_scenario is not going to do anything useful at all, because when it exits, the original settings are restored. See my previous comment.

[bittner]

Alright, so if that's the problem we have to reset the settings value in after_scenario. It's not too hard to do with such a decorator:

@override_settings(DEBUG=True)
def before_scenario(context, scenario):
    pass

@override_settings(DEBUG=False)
def after_scenario(context, scenario):
    pass

If you don't like this, actually explicit, approach let's find a more elegant way to get this implemented. Am I missing something?

And yes, the Given step implementation would be the natural entry point to set up such a scenario-only behavior, I feel. Let's make the decorator work in test implementations, and we're done. Does that sound too optimistic?

@farcepest I'm referring to your "It works" from above. You say, it works. So, why not implement this as a first attempt and then move on to improve the implementation incrementally. My point of view is, the code needs to be nice to read and self-explanatory. Because readability counts. A decorator is already better than the current situation, and it can hide the implementation details.

Sorry if I'm being too pragmatic. Let's find a better way!

[farcepest]

It's a decorator... It temporarily changes the settings, runs the decorated function, then restores the settings. That just doesn't work on before_scenario, because your scenario still runs with the original settings. It is not at all obvious how to implement this with behave. context.text is an instance of the LiveServerTestCase, so it is too late to apply a decorator to that.

"It works" applied to this:

def before_scenario(context, scenario):
    if "DEBUG" in scenario.tags:
        from django.conf import settings
        settings.DEBUG = True

and then you have to have similar code in after_scenario to undo what you did in before_scenario. It's not pretty, but it works. There doesn't seem to currently be a better solution for this currently.

[farcepest]

and I want to emphasize: @override_settings(DEBUG=True) does not work in a useful way. Sure, the settings are overridden, for the duration of before_scenario, which is not at all helpful.

[bittner]

No offence, please. Decorators are no magic, it's a wrapper function. If I do dirty stuff in its implementation the result will stay dirty, won't it? It may be meant to clean up after itself, but this is not automatic, right?

I've not tried it (nor will I), so I'm really just trying to help us think: If overriding certain settings "the dirty way" won't have an effect, because the LiveServer instance is already running then we'll have to adapt the management command. In the worst case we may have to launch a separate LiveServer instance for each scenario (I hope that's not needed).

Nothing is impossible, it may be hard though. Anyone willing to give it a shot?

[farcepest]

override_settings is specifically intended to temporarily change the settings for the duration of the wrapped class/method. Internally it's a context manager. https://docs.djangoproject.com/en/1.10/topics/testing/tools/#django.test.override_settings

[bittner]

We should agree that we are talking about behave-django. I'm not referring to django.test since #13 (comment) onwards. Technically, it's certainly possible to make things more elegant than today.

[mixxorz]

So right now, this is how I would do it.

# some.feature
Feature: Change the settings for specific scenarios

  @debug
  Scenario: Change some settings here
    Given I want to change DEBUG to True
    Then DEBUG should be True

# environment.py
from django.conf import settings

def before_tag(context, tag):
    if tag == 'debug':
        settings.DEBUG = True

def after_tag(context, tag):
    if tag == 'debug':
        settings.DEBUG = False

This already isn't too bad, unless you need a bunch of different unique configurations.

This is @bittner's proposal:

@override_settings(DEBUG=True)
def before_scenario(context, scenario):
    pass

@override_settings(DEBUG=False)
def after_scenario(context, scenario):
    pass

I don't think this would work though because you want to override the settings on a per Scenario basis. To make this better, maybe we'll go for:

@override_settings('debug', DEBUG=True)
@override_settings('foo', FOO='some value')
def before_tag(context, tag):
    pass

@restore_settings
def after_tag(context, tag):
    pass

First value to override_settings will be which tag the settings would take effect on. The rest of the kwargs will be settings you want to override. The decorator also stores data in a "private" variable in context so restore_settings can restore the previous settings.

Though at this point, I'm liking the first approach more because it's less hacky, more explicit, and less magic, but that's just me. What do you guys think?

[farcepest]

First approach is the only viable one currently, IMHO; @override_settings() only lasts for the duration of the function call, so the net effect on your test is zero, since before_tag() is called before your test and after_tag() is called after your test, and while your test runs, @override_settings() is not in effect.

[mixxorz]

In the example I gave, @override_settings is a custom decorator included in behave-django, not to be confused with the one built-in to django.

[belugame]

Has behave-django seen changes in regards to this issue since?
I would especially be interested what would be the best approach to @patch a method for the time of a scenario or if that is not possible for the time of all behave-tests.

[pydolan]

It makes sense to me for settings to work similar to how behave-django allows fixture loading, where behave-django would support both of the following options:

Interface:

Option 1: in before_scenario

In this example, we override settings for a particular scenario. Since a new test class is instantiated for each scenario, teardown is automatic:

def before_scenario(context, scenario):
    if scenario.name == 'User login with valid credentials':
        context.override_settings = {'DEBUG': True, 'LOGIN_URL': '/login/here'}

Option 2: via decorator on step functions

In this example, we define a Given Step that uses the behave_django.decorators.override_settings decorator, which behave-django would use to override django's settings for the entire scenario:

• Step function:

  @override_settings(DEBUG=True, LOGIN_URL='/login/sso')
  @given('I enable debugging and login with SSO')
  def step_impl(context):
      pass

• Gherkin file:

  Scenario: User login with valid credentials
      Given I enable debugging and login with SSO
      When I login with "test" and "test"
      Then I see content "Welcome"

Implementation:

I'm assuming that in the code where behave-django adds fixtures to the TestCase class, the django settings can also be overwritten. This can possibly be done with django.test.utils.override_settings, either using the class as a context, or mimicking what the decorate_class method is doing. Thoughts?