New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Documentation: Testing #1791

Closed
oskarols opened this Issue Jun 28, 2016 · 32 comments

Comments

Projects
None yet
@oskarols
Copy link

oskarols commented Jun 28, 2016

It would be splendid if there was documentation for how one would go about testing complex Observable operator chains.

This is coming from a Angular 2 perspective, where one would probably use dependency injection to inject a different Scheduler inside of the tests: i.e when doing Observable.timer(1000, AsyncScheduler), one would change the AsyncScheduler to a TestScheduler.

I've tried finding documentation (or pretty much any instructions from anyone) on how one would go about actually instrumenting these Schedulers to test Observable sequences, but it's very hard to actually figure out since the APIs have seemingly changed quite a bit going from version 4 to 5 (e.g. scheduleAbsolute which is used in most testing examples for Rxjs 4, is totally absent).

Neither the unit tests nor the Scheduler docs were very helpful either in this regard.

@kwonoj

This comment has been minimized.

Copy link
Member

kwonoj commented Jun 28, 2016

Coincidence, I was came to think of this topic recently too. It'd be nice to have practical examples of testing.

@iensu

This comment has been minimized.

Copy link

iensu commented Jul 6, 2016

Yes, some clear example of you can control the ticking of the TestScheduler would be nice. I've skimmed through the source code and spec-files but couldn't find any clear examples of how to achieve this.

@mjprude

This comment has been minimized.

Copy link

mjprude commented Jul 14, 2016

Agreed. This has been a big blocker for my team in trying to get a production-ready, ground-up 5.0 application off the ground. At least adding something to the migration document, since it seems a lot has changed in the way TestScheduler works, would be very helpful.

@frapontillo

This comment has been minimized.

Copy link

frapontillo commented Aug 15, 2016

I am really having a hard time trying to figure out how to specify a TestScheduler for my Angular 2 app. Some documentation, or even a couple of notes in this issue, may greatly help.

@latobibor

This comment has been minimized.

Copy link

latobibor commented Aug 26, 2016

I'd like to second this!

It would help a lot for beginners if all tutorials/code exampled would be accompanied with their unit tests.

If you can try out and verify your code quickly without firing up a server and clicking on a button would help a lot. People can learn much more quickly if they can play with things.

Also a cheat sheet would be very useful:

  • how to mock angular events (for example the event from $scope.$destroy())
  • helpers for common cases (start an observable do a thing and then unsubscribe), etc.
@marcusnielsen

This comment has been minimized.

Copy link
Contributor

marcusnielsen commented Aug 30, 2016

I made a medium article on this here: https://medium.com/@marcus.nielsen82/simplified-testing-of-user-events-in-rxjs-411efa02a341#.rtq8thwgb

But I'm still a pretty bad writer, so the text is pretty verbose I think. The snippets might be worth something to someone.

Basically I inject Rx.Observable.empty() when not testing the emitted events.
I use Subject to manually mock a stream I want to test.
I use fooStream.take(1).subscribe to end hot observable streams.
I call fooSubject.next({value: 'foo'}) and assert inside subscribe.
If I have a playback of last value (BehaviorSubject() / publishReplay(1)), then I call next before subscribe.
Else, I call next after the subscribe call.

This is entirely without any scheduling simply because we haven't needed any yet. It is just a starting point, but it got us far with very complicated compositions of streams and saved my ass many times.

We also have a set of (user-)triggered actions that we can expose temporarily on the DOM by doing window.myComponent = myComponent and then open the web console and write myComponent.behaviors.triggers.addTodo({title: 'foo', body: 'bar'}) to fake user interactions without any renderable content.

It also enables us to programmatically play user behaviors on different components and check that the app state is correct. And that could ofc be sent to a database log.

@elldawg

This comment has been minimized.

Copy link

elldawg commented Oct 24, 2016

anyone make any progress on this?

@jayphelps

This comment has been minimized.

Copy link
Member

jayphelps commented Oct 24, 2016

@elldawg yes, progress is being made but nothing released yet. It's one of our primary focuses during the RC process.

@elldawg

This comment has been minimized.

Copy link

elldawg commented Oct 24, 2016

I naively believe that a very simple example here would be most helpful.

If I do the following:

        describe("",
            () => {
                let testScheduler: TestScheduler;
                let timerObservable: Observable<number>; 

                beforeEach(()=> {
                    testScheduler = new TestScheduler((a,b)=>{expect(a).toEqual(b)});
                    const originalTimer = Observable.timer;
                    spyOn(Observable, "timer").and.callFake((initialDelay, dueTime) =>{
                        return timerObservable = originalTimer.call(this, initialDelay, testScheduler);
                    });
                });

                it("",
                    () => {

                        Observable.timer(0, 1000).subscribe((num: number)=>{console.log(num)});
                        testScheduler.createTime("-1-2-3-|");
                        testScheduler.flush();
                    });
            });

I expect that 1,2,3 should be emmitted. I am obviously missing something.

@elldawg

This comment has been minimized.

Copy link

elldawg commented Oct 26, 2016

So I did some fiddling and came up with something that allows me to control timing:

function installMockTimer(maxFrames?: number): TestScheduler {

    //Create a test scheduler
    const scheduler = new TestScheduler((a, b) => expect(a).toEqual(b));

    //schedular.maxFrames controls how many frames (i.e. milliseconds) will be simulated
    //once flush is invoked.  It defaults to 750, so if you setup a timer for more than 750
    //frames, then nothing will happen
    scheduler.maxFrames = Number(maxFrames) || scheduler.maxFrames;

    //replace timer with a fake
    const originalTimer = Observable.timer;
    spyOn(Observable, "timer")
        .and.callFake(function(initialDelay, dueTime) {
            return originalTimer.call(this, initialDelay, dueTime, scheduler);
        });

    //return the scheduler.  This is what the client code uses to advance time
    return scheduler;
}

it("", ()=>{
    const scheduler = installMockTimer(5000);
    //we don't need to change max frames here. Just did it to illustrate that it's possible
    scheduler.maxFrames = 2000;
    Observable.timer(0, 1000).subscribe((data)=> console.log(scheduler.frame);
    scheduler.flush();
    //LOG : 0
    //LOG: 1000
    //LOG: 2000
});
@OzzieOrca

This comment has been minimized.

Copy link

OzzieOrca commented Nov 17, 2016

Yes @elldawg's solution above using maxFrames to specify what frame a flush() should execute until seems to work.

Hopefully we get something like scheduleAbsolute back in the future or, if it works to do it all synchronously, maybe just the ability to tell flush how many frames to execute.

@intellix

This comment has been minimized.

Copy link

intellix commented Jan 18, 2017

Did this progress anywhere?
Also stuck with testing and can't find any resource about how to test my current services. Whenever I ask about it people direct me to the RxJS tests, but I don't control the source Observables unless I copy/paste my logic, but that's not actually testing.

Tried stealing these helpers but I have no idea how to control time: https://github.com/ngrx/store/blob/6a526aef35cbb6340dc011201e56c7d790132281/spec/helpers/test-helper.ts

@marcuswhit

This comment has been minimized.

Copy link

marcuswhit commented Feb 9, 2017

+1 on all this. The marble diagrams are nice, but it's entirely unclear how you'd use them to test an observable containing multiple time-based operators (i.e. real world problems). Having previously worked with Rx.net, RxJava, and RxJs4, with easy testability being one of the greatest features of Rx, it's fairly frustrating that all the standard Rx testing conventions have been seemingly abandoned.

I have stumbled across https://github.com/kwonoj/rxjs-testscheduler-compat, which in the absence of any better official guidance, looks to be the best way forward for getting our app tested.

@maxruby

This comment has been minimized.

Copy link

maxruby commented Mar 26, 2017

+1

I am very keen also to find out when we will have good documentation and stable interfaces to set up testing for complex Observable.timer cases. Would be good to know at least whether there are clear plans to have this in place in the near future.

@ofabricio

This comment has been minimized.

Copy link

ofabricio commented Mar 27, 2017

I was struggling with this a few days ago. I'll describe how I managed to test in a way I think it's really nice (I don't know if it's the right way, though).

I had a class like this:

@Injectable()
export class TitleNotificationService {
  constructor(private dataService: DataService) {

    const chatlog$ = this.dataService.chatLogMessages();
    const addedMsg$ = Observable.fromEventPattern(
      handler => chatlog$.$ref.on('child_added', handler),
      handler => chatlog$.$ref.off('child_added', handler),
    );

    const visibility$ = Observable.fromEvent(document, 'visibilitychange')
      .map(e => e.target.hidden);

    addedMsg$
      .skipUntil(start$.take(1))
      .windowToggle(visibility$.startWith(document.hidden).filter(hidden => hidden), () => visibility$)
      .mergeMap(win => win
        .map((v, i) => i + 1)
        .concat(Observable.of(0))
      )
      .withLatestFrom(Observable.of(document.title))
      .subscribe(([count, originalTitle]) => {
        const max = this.dataService.getMaxMessages();
        document.title = count === 0
          ? originalTitle
          : `(${count > max ? max + '+' : count}) ` + originalTitle;
      });
  }
}

Then I wanted to test that observable chain logic. The best way I found was to split it into a new method (I didn't like this because now I have a method exposed (public) only for test purpose, but whatever):

@Injectable()
export class TitleNotificationService {
  constructor(private dataService: DataService) {

    /* same code as before */

    this.unreadMsgsCount(addedMsg$, visibility$, chatlog$, document.hidden)
      .withLatestFrom(Observable.of(document.title))
      .subscribe(([count, originalTitle]) => {
        /* same code as before */
      });
  }

  /* method with the logic I want to test */
  unreadMsgsCount(addedMsg$, visibility$, start$, initialVisibility) {
    return addedMsg$
      .skipUntil(start$.take(1))
      .windowToggle(visibility$.startWith(initialVisibility).filter(hidden => hidden), () => visibility$)
      .mergeMap(win => win
        .map((v, i) => i + 1)
        .concat(Observable.of(0))
      )
  }
}

Then in order to test I did:

describe('TitleNotificationService', () => {

  beforeEach(async(() => {
    TestBed.configureTestingModule({
      providers: [
        TitleNotificationService,
        { provide: DataService, useValue: dataServiceMock },
      ],
    });
  }));

  beforeEach(() => {
    spyOn(Observable, 'fromEventPattern').and.returnValue(Observable.empty());
    spyOn(Observable, 'fromEvent').and.returnValue(Observable.empty());
  });

  it('should work for the default flow', inject([TitleNotificationService], (service: TitleNotificationService) => {
    var scheduler = new TestScheduler(null);
    const m = scheduler.createHotObservable('-m---m-m-m---m-m---m-m---|');
    const v = scheduler.createHotObservable('---h-------v-----h-----v-|', { v: false, h: true });
    const s = scheduler.createHotObservable('---s---------------------|');
    const e = scheduler.createHotObservable('-----1-2-3-0-------1-2-0-|', { 0: 0, 1: 1, 2: 2, 3: 3 }); // the expected output
    const o = service.unreadMsgsCount(m, v, s, false); // the method I'm testing
    Observable.combineLatest(
      o.toArray(),
      e.toArray()
    )
    .subscribe(([result, expected]) => {
      expect(result).toEqual(expected);
    });
    scheduler.flush();
  }));
});

The nice thing about this is that I can draw marbles with the input and output in the format I want to test.
I also compare the results as an array, so I can see where exactly the process is failing:

Expected [ 1, 2, 3, 0, 1, 2, 3 ] to equal [ 1, 2, 3, 0, 1, 2, 0 ].
                             ^ here

Please let me know your impressions about this approach. This basically mimics the way they test the rxjs lib.
Oh, and I don't know if this would work for operations using interval for example (prob not), otherwise it seems to work.

@intellix

This comment has been minimized.

Copy link

intellix commented Mar 27, 2017

TL;DR - Observable.defer is your friend

the above looks great. One of the problems I had with testing was that I have an API class containing lots of Observable properties that all get merged/chained together at some point.

When testing, I found it hard to be able to test them individually because of the eager creation of Observables, example:

class UserService {

  currentUser = this.sessionService.session
    .map(session => session ? session.identity : null)
    .distinctUntilChanged()
    .publishReplay(1)
    .refCount();

  wallets: Observable<IWallet[]> = this.currentUser
    .switchMap(() => {
      return Observable.timer(0, 30000)
        .switchMap(() => this.http.get(`/wallet`))
        .takeUntil(this.currentUser.filter(u => !u));
    })
    .publishReplay(1)
    .refCount();

  mainWallet: Observable<IWallet> = this.balances
    .mergeMap(wallets => wallets.filter(wallet => wallet.name === 'main'))
    .map(wallet => Object.assign({}, wallet, { amount: wallet.amount / 100 }))
    .publishReplay(1)
    .refCount();
}

From that example, I wanted to be able to test that the mainWallet is transforming cents to dollars: 1000 to 10 but upon Instantiation of the class, it's already consumed the previous Observable in the chain.

I found that I can use Observable.defer so that the properties are only consumed when you subscribe to an Observable property, meaning you can test individual pieces by replacing them before they're subscribed to:

class UserService {

  currentUser = this.sessionService.session
    .map(session => session ? session.identity : null)
    .distinctUntilChanged()
    .publishReplay(1)
    .refCount();

  wallets: Observable<IWallet[]> = Observable.defer(() => this.currentUser)
    .switchMap(() => {
      return Observable.timer(0, 30000)
        .switchMap(() => this.http.get(`/wallet`))
        .takeUntil(this.currentUser.filter(u => !u));
    })
    .publishReplay(1)
    .refCount();

  mainWallet: Observable<IWallet> = Observable.defer(() => this.balances)
    .mergeMap(wallets => wallets.filter(wallet => wallet.name === 'main'))
    .map(wallet => Object.assign({}, wallet, { amount: wallet.amount / 100 }))
    .publishReplay(1)
    .refCount();
}

Now during testing if you want to test UserService.mainWallet you can replace this.balances with a Subject and next the values you need in there.

Now I'm testing in Angular using fakeAsync and tick to control time. I'll have to start using what @ofabricio wrote though :)

@naugtur

This comment has been minimized.

Copy link

naugtur commented Apr 7, 2017

Could a step 1 of documenting testing be a doc about where to get the methods for marble tests at least?
I'm not using wallaby and I'm trying to find the reference to expectObservable but I can't find it anywhere in Rx

@kwonoj

This comment has been minimized.

Copy link
Member

kwonoj commented Apr 7, 2017

@naugtur wallaby's just test runner and nothing related to actual test. https://github.com/ReactiveX/rxjs/blob/master/src/testing/TestScheduler.ts#L74

@naugtur

This comment has been minimized.

Copy link

naugtur commented Apr 7, 2017

Is there a way at all to write a test without external dependencies? I'm trying to keep it simple, as in functions with assertions (or tape runner). The only examples of a full working tests I found used either wallaby or karma and while I know these are not related, I can't seem to find out how to run a test without them.

@kwonoj

This comment has been minimized.

Copy link
Member

kwonoj commented Apr 7, 2017

Is there a way at all to write a test without external dependencies?

@naugtur I don't get this part, since our test is only relying on test runner (mocha and some assertions) only. Wallaby.js is just another test runner configuration and we don't have karma even. if you're doing npm test in our repo, it just works with mocha only.

@naugtur

This comment has been minimized.

Copy link

naugtur commented Apr 7, 2017

I'm happy to move elsewhere not to derail this thread.
What I mean is I'm trying to get a TestScheduler, set it up and get my code to run. Maybe assert something. I can't find what I need to import/require to get all the necessary parts. scheduler.createHotObservable was easy to find, but I don't know how to get the scheduler to start and how to assert.

@kwonoj

This comment has been minimized.

Copy link
Member

kwonoj commented Apr 7, 2017

@naugtur I'll prep something simple can be used meanwhile doc / refactoring is ongoing.

@kwonoj

This comment has been minimized.

Copy link
Member

kwonoj commented Apr 7, 2017

@naugtur check https://www.npmjs.com/package/rxjs-testscheduler-bootstrapper out wrapping up bootstrappings. If you're strongly want to avoid any external packages, you can just copy-paste those instead of relying on installing modules.

@naugtur

This comment has been minimized.

Copy link

naugtur commented Apr 7, 2017

Great, thanks. Once I get the idea of how that's supposed to work, I'll be happy contribute some docs if you're looking for volunteers.

@martinsik

This comment has been minimized.

@dfbaskin

This comment has been minimized.

Copy link

dfbaskin commented Apr 11, 2017

When you are trying to test with a VirtualTimeScheduler, it seems that the only way to do this is by passing the scheduler through your entire operator chain.

import {ActionsObservable as Observable} from 'redux-observable'
import {VirtualTimeScheduler} from 'rxjs/scheduler/VirtualTimeScheduler';

import 'rxjs/add/observable/empty';
import 'rxjs/add/observable/interval';
import 'rxjs/add/operator/delay';
import 'rxjs/add/operator/concat';
import 'rxjs/add/operator/reduce';
import 'rxjs/add/operator/takeUntil';
import 'rxjs/add/operator/switchMap';
import 'rxjs/add/operator/toArray';
import 'rxjs/add/operator/do';

const intervalSeconds = 60;
let scheduler;

const intervalEpic = (action$) => {
    return Observable
        .interval(intervalSeconds * 1000, scheduler)
        .takeUntil(action$.reduce((v, action) => 0, 0))
        .switchMap(() => {
            return Observable.of({
                type: "SOME_ACTION"
            })
        });
};

export function runObservableEpicTest(observable, epic, done, verify) {
    let mockStore = {
        getState() {
            return {};
        }
    };
    Observable
        .empty()
        .concat(epic(observable, mockStore))
        .toArray()
        .do((actions) => verify(actions))
        .subscribe({
            error: (err) => done.fail(err),
            complete: () => done()
        });
}

describe('test with interval', () => {

    beforeEach(() => {
        scheduler = new VirtualTimeScheduler();
    });

    it('should allow testing of interval', (done) => {
        let observable = Observable
            .empty()
            .delay((intervalSeconds + 1) * 1000, scheduler);
        runObservableEpicTest(observable, intervalEpic, done, (actions) => {
            expect(actions).toEqual([
                { type: "SOME_ACTION" }
            ]);
        });
        scheduler.flush();
    });
});

Should the documentation reflect this requirement? Should it demonstrate how to do so using shared variables, factories, or some other technique?

Or will the changes to testing in the upcoming version of RxJS provide us an easier way to test complex operator chains?

@naugtur

This comment has been minimized.

Copy link

naugtur commented Apr 21, 2017

@martinsik I saw that stackoverflow thread and it's been a source of great confusion to me, as scheduler doesn't have the methods used in the top answer. (not even .flush)
Is the answer outdated, or should I use the package in a different way? I've built the code with webpack1 and commonjs syntax, so it couldn't have removed the methods.

@martinsik

This comment has been minimized.

Copy link
Contributor

martinsik commented Apr 21, 2017

@naugtur The TestScheduler class has all the methods mentioned in the answer.
https://github.com/ReactiveX/rxjs/blob/master/src/testing/TestScheduler.ts

@jongunter

This comment has been minimized.

Copy link

jongunter commented Jul 27, 2017

Is there any sort of documentation/examples available for TestScheduler? I can't seem to find anything related to RxJs5 on the web.

@gsans

This comment has been minimized.

Copy link
Contributor

gsans commented Nov 30, 2017

I can recommend you to look at this project while there's no official solution.
https://github.com/cartant/rxjs-marbles
Supports: Jasmine, Mocha, Jest, AVA and Tape. Syntax differs slightly but most marble-test features are there, although not the ones from RxJS4.
See Stackblitz for some examples.

@kwonoj

This comment has been minimized.

Copy link
Member

kwonoj commented Nov 30, 2017

it's not official Rx core module though, my own rx-sandbox (https://github.com/kwonoj/rx-sandbox) is framework agnostic, feature parity to TestScheduler class.

@cartant

This comment has been minimized.

Copy link
Collaborator

cartant commented Jan 28, 2019

Closing this as testing documentation as been added.

@cartant cartant closed this Jan 28, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment