1.3.0 - change in how variable scope is treated in invoked feature files

[anupamck]

Was there a change to how variable scope is handled in the recent upgrade?

We test a massive API with several endpoints, which we reuse in several tests. Therefore, we have written down our endpoints and their requests in feature files that are separate from the tests that invoke them.

E.g. An endpoint would be documented as shown below, in a separate feature file:

endpoints/titles.feature

Feature: titles

    Background:
        Given path 'titles'

    @get
    Scenario: Get titles
        When method get

This endpoint feature file is invoked by another test feature file, which actually tests this endpoint.

tests/titles.feature

Feature: titles

    Background:
        Given url baseURLOCCAPI

    Scenario: GET standard response
        When def getResponse = call read('classpath:endpoints/titles.feature@get')
        Then match getResponse.responseStatus == 200
        And match each getResponse.response.titles == { code: '#string', name: '#string' }

This separation works beautifully, since data specific to the test (request body, params, user login, assertions) lives in the test feature file, while the endpoint can be reused across several tests (even negative tests). This separation works in v1.2.0. Once I run the test, variables such as the url in the test file are passed to the invoked feature file just as they are. In fact, this behaviour is as described in the documentation here.

When I ran my tests in 1.3.0, I was shocked to see all of them fail. The reason was because the url parameter defined as part of the background here wasn't passed to the invoked request. The same applies to auth tokens and other request parameters - the scope of the parent feature file seems unavailable to the invoked one.

@ptrthomas - Am I missing something here? I was also surprised to not read about this in the release notes. Also, I'd love for you to comment on the design-pattern (or anti-pattern) of separating enpoints and requests from tests.

[ptrthomas]

@anupamck possible duplicate of #2054 or to reopen, we insist that you follow this process: https://github.com/karatelabs/karate/wiki/How-to-Submit-an-Issue

please note that we spent MONTHS releasing RC versions and asking for feedback.

[anupamck]

Thank you for redirecting me to that issue. On reading up, I realized that I had actually used unintended behaviour on 1.2.0 to structure our tests. I also understand that you recommend keeping your tests in one main flow, and that making test code DRY is often a pitfall. I will refactor my tests accordingly.

My only suggestion is to update the release notes of 1.3.0 to include this fix. IMHO, this is a breaking change, since it broke _all _ our tests, which worked perfectly in the previous release.

I am relatively new to Karate, but love the tool already. I will be sure to try out the RCs and provide quicker feedback for upcoming releases. Thank you for everything you do!

[ptrthomas]

@anupamck fine, we edited the release notes. It is a very rare case that applies only to people who started with 1.2.0.

That said, I now wonder if we should NOT reset the url when we call features. You can of course optionally reset this by setting it explicitly. The principle we stick to is explained here for path: https://github.com/karatelabs/karate#path - screenshot below

If that makes sense, let me know, this original intent may have been lost in the refactoring of Karate to handle dynamic scenario outlines and since no one even provided a clear way to replicate. I'll point other threads to this right away.

[snow23man]

imo, based on our implementation of karate, url should never change. That is because it never changes for us, it comes from a config file. The way we have implemented karate, url is the first part, so "http://www.google.ca" and path is everything after, so "/search?=". url being constant and path being dynamic. url and path aren't variables. You can use variables to feed into url and path, at the feature file level, in the background and in the scenarios

[anupamck]

I agree with @snow23man. I expect my base url to never change. Path and parameters change with every request.

[ptrthomas]

thanks all for the comments. opened a new issue to be clear and we will make this change for 1.3.1 which should be out soon: #2209

[ptrthomas]

do comment on the new issue, I just have one more question to clarify - should be url be reset in "isolated (local) scope" or not