Testing is Done Right

[3LOK]

Hi,

My comment is about: https://redux-saga.github.io/redux-saga/docs/advanced/Testing.html

I think the way you see testing sagas is wrong and puts emphasis on exact 'implementation', and not 'result'.

The way you do tests: Check the call for each 'yield', and muck a return value to that yield. In this case if I change the implementation of my saga (in such a way that doesn't affect the results), I need to change me tests. Any refactoring of code will necessarily mean changing the tests.

My view is that your tests check that you implemented your saga in a very very specific way and not that the saga is doing what it needs to be doing.

The way we write test for our sagas is:

1. Set up a test store with a specific scenario. Test store includes a reducer that interacts with actions that are fired by the saga.
2. Run the saga using the standard redux-saga framework.
3. See the side effects (actions that were sent).

I think this way of testing puts the emphasis where it needs to be. If we refactor the saga, and it still does exactly what it needs to do, the tests won't need to change (making the tests helpful for refactoring).

WDYT?

[Andarist]

I kinda feel the same. In my tests I wouldnt like to focus on underlaying implementation, just like in the 'normal' code. For that reason I'm not even testing my sagas right now. Dont want to have my tests broken if I rearrange 2 lines which dont even change the overall behaviour

[slorber]

It's like other tests: do you want to unit-test the details of your code, or do you want to do integration tests of the behavior of the saga and your store, without having to deal implementation details.

I've always felt doing blackbox testing is a superior idea, as testing implementation details means that changing implementation details will likely break tests.

For example, do you want to test your classes that are used to build a REST API, or do you want to test the API directly? Personnally I think it's better to test the API directly, and don't care that much about unit tests, except rare cases where it's worth it because the function to test is likely complex and not going to change (like date/math utilities...)

I guess the subject is also quite related to new debates we've seen on twitter that say it's not really worth it to test React component outputs because when changing React component it will always break the test. That's probably why Jest's snapshot testing feature is gaining interest of the community

If redux-saga gives you good testability for yielded effects, it does not mean you have to test all your code in this fashion.

[3LOK]

I'll make sure to open-source our redux-saga testing infrastructure soon.

Basically what we wrote is a component that you construct with full initial "state", an optional list of reducers (for more complex tests), and the sagas you need (including the tested one). Then our tests look like:

const tester = new SagaIntegrationTester({state, reducers, sagas}).
tester.put(someAction());
return tester.waitForAction(types.FINAL.ACTION).then(action => {
   expect(action.params).to.deep.equal(...);
   expect(tester.state).to.deep.equal(finalState);
})

[ms88privat]

@3LOK sounds awesome, please keep us informed!

[rpmonteiro]

@3LOK Patiently awaiting to hear back from you. I'm not pleased with the current way I have to test my sagas... :(

[3LOK]

Hi all,

Here we go: https://www.npmjs.com/package/redux-saga-tester

Let me know what you think.

[neurosnap]

I agree with @slorber, redux saga is providing you a means to unit test a saga whereas you want to test how the saga integrates with your actions, reducers, etc. At that point you might as well go for a full end to end test because that's effectively the goal you are attempting to accomplish.

[3LOK]

@neurosnap I disagree - 'testing that you wrote specific lines of codes' is not unit-test. (by the way, I think that's also what @slorber meant, @slorber correct me if I'm mistaken).

Here's an example: Lets say I have a function:

function foo(params) {
    var a = params.a;
    var b = params.b;
    return a + b;
}

Now I have excellent unit test for this function. One day, I decide to change the order of the two 'var' lines - you'll agree with me that the unit test should not break - the function still does exactly what you'd expect it to do, right?

But if we see this as a saga:

var a = yield select(aSelector);
var b = yield select(bSelector);
yield put(actions.Result(a+b));

If I were to unit test your way, and now change the order of the two vars, the unit test will fail although the unit still does exactly what it's supposed to be doing (!). That's a clear sign of a bad unit test. This means that the unit test is specific for my specific implementation (implementation = line of codes) - that's not a good unit test. In your case - you might as well just write the same code twice, and use 'diff' - it is effectively the goal you are attempting to accomplish.

For redux-saga, the store and the actions are the inputs and outputs of the saga. To unit-test the saga is to give it different inputs (different stores), and to see its outputs (different actions). It's not the end-to-end test at all (eg. I'm not giving it a real store, or real reducers - just the ones that simulate the test).

My type of unit test allow me to refactor the code and still know it does exactly what it's supposed to do, without needing to rewrite the tests.

[neurosnap]

You make a good point, @3LOK the order of selectors shouldn't change the outcome of the test. It seems more accurate to say that the conventional approach to unit testing sagas is actually to unit test the saga effects, rather each iteration of the generator. Basically with your library, you prepare a mock state, mock reducers, and dispatching a series of actions to determine if the saga dispatches the correct actions, is that correct?

[3LOK]

@neurosnap yup, exactly.

[yelouafi]

@3LOK How do you handle call effects? are call effects executing the real functions?

[3LOK]

@yelouafi call effects are executed (since our testing infrastructure uses redux-saga to execute the saga), however we never call the 'real functions' in our unit-testing. All our sagas are initialized with the libraries they use, we never directly 'require' (or 'import') the libraries from the saga (as a rule for TDD):

Here's an example of our Feedback saga:

    export default function* feedbackRootSage(libraries) {
        yield [
            fork(waitForFeedback.bind(null, libraries))
        ];
    }

    function* waitForFeedback(libraries) {
        yield* takeEvery(types.FEEDBACK, handleFeedback.bind(null, libraries));
    }

    function* handleFeedback(libraries, action) {
        const {WixRestaurantsApi} = libraries;
        const {payload} = action;

        try {
            yield call(WixRestaurantsApi.sendFeedback, {
                accessToken:payload.accessToken,
                feedback:payload.feedbackRequest
             });
             yield put(actions.feedbackSent());
        } catch (e) {
             yield put(actions.feedbackError(e));
        }
    }

As you can see, during testing we can simply initialize the saga with mock libraries (for example a mock WixRestaurantsApi) and simulate different results.

[jfairbank]

I guess I'll be the voice of dissension :)

While I admit that unit testing sagas necessarily ties the tests to the implementation, I can now make guarantees on the effects my saga deals with. I'm not only interested in the actions it dispatches. What if I care which API calls it makes? What if I care what arguments the call descriptor passes to my API functions?

Of course I love dependency injection (i.e. passing in the libraries that my saga will use), but I want to know that it uses the libraries correctly. Sure, I could create spies/mocks and assert that they're called with certain arguments, but then why bother using sagas? I can use spies with thunks. I want to test with as few spies and mocks as possible.

Reusing this example:

    function* handleFeedback(libraries, action) {
        const {WixRestaurantsApi} = libraries;
        const {payload} = action;

        try {
            yield call(WixRestaurantsApi.sendFeedback, {
                accessToken:payload.accessToken,
                feedback:payload.feedbackRequest
             });
             yield put(actions.feedbackSent());
        } catch (e) {
             yield put(actions.feedbackError(e));
        }
    }

Maybe it's personal preference or being overly anxious, but I would like to know that the sendFeedback function is called with the correct arguments. I also write some sagas intentionally such that ordering IS important. So for me, I want to test that ordering.

(Aside: Even my react tests take a hybrid unit testing approach along with snapshots because I'm not convinced snapshots are enough. If my component uses certain props, then I'd like to see those represented in the rendered output. If I'm writing stateless components with functions, then passing in props should result in a certain type of output based on those props. How is that any different from a normal pure function? Should I snapshot all my normal functions then too and just hope the initial snapshot is producing the correct result?)

I agree that refactoring a saga can be slightly annoying for unit testing, which is why I created redux-saga-test-plan. It has helped tremendously for testing and refactoring sagas, so fixing a test isn't that big of a deal for me personally.

Maybe I'm reading too much into it, but if I think about sagas as something similar to IO monads (#505), then they are similar-ish to compositions of function calls. I'd more than likely test the individual composed functions, so why should a saga be any different for testing?

So far, I haven't been burnt by this testing strategy for the project I've been working on. I have found more value and produced fewer bugs by just unit testing react, redux, and redux saga on this project, so I'm not convinced that I need integration testing.

Realistically, I think testing will be a subjective practice no matter how much we analyze it. Ultimately the testing strategy has to align with the goals of the business, the size of the team, and the pace of development of the product. For some teams, taking a more wholistic, integration testing approach might be better. For us, we've found unit testing more valuable. So maybe both testing approaches need to represented in the documentation along with the pros and cons and what each approach focuses on achieving.