Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Browse files

Merge branch 'master' of github.com:creationix/howtonode.org

  • Loading branch information...
commit d83e1eda85f62bd593dd8501eed5d90ec1c56047 2 parents 03c558d + 509eced
Tim Caswell authored
View
57 articles/make-your-tests-deterministic.markdown
@@ -0,0 +1,57 @@
+Title: Make Your Tests Deterministic
+Author: Vojta Jina
+Date: Tue Aug 13 2012 01:12:01 GMT-0800 (PST)
+Node: v0.8.4
+
+
+Non-deterministic issues like [race conditions] or [thread deadlocks] are very difficult to test, because they are by nature hard to reproduce. Fortunately, in the JavaScript world, we only have a single thread, so we are safe, right? Well, not really because...
+
+**Execution order of callbacks can cause the same race condition issues that plague multi-threaded environments.**
+
+
+## Example
+
+Let's say we need a function that will return an array of timestamps representing the last modification time of a set of files.
+
+<make-your-tests-deterministic/get-last-modified.js>
+
+To unit test this code, we need to mock out `fs` module (using real file system would require setting up some initial state, cleaning up afterwards and would make the test slow). That's pretty easy with [node-mocks] - we can use a fake in-memory file system.
+
+<make-your-tests-deterministic/get-last-modified.spec.js>
+
+This unit test passes, but the production code is broken. Even worse, it sometimes works, sometimes it does not... The problem is that this code relies on the order of `fs.stat` callbacks, which is preserved in the mock version of `fs`, but can't be guaranteed on real fs - it's non-deterministic.
+
+
+## Solution
+
+Our solution needs to simulate the problematic behavior during the test.
+
+We could make the fake `fs` to fire callbacks in a random order, which would be very similar to the real `fs` behavior. However, that would result in a flaky tests that sometimes pass, sometimes not - just like our production code. Yep, we could do "stress" testing - execute the test multiple times, to make the probability of failing bigger, but that's still flaky.
+
+What we really want is our tests to be always deterministic - reliable, not flaky. So we need to simulate this behavior in a predictable - controlled way.
+That's why the `fs` mock uses something called `predictableNextTick`, which behaves similar to [process.nextTick], but fires callbacks in a specific order.
+
+Let's add this line into our unit test:
+
+ mocks.predictableNextTick.pattern = [1, 0];
+
+Suddenly, the test is failing, because now the `fs.stat` callbacks are fired in different order than they were registered. They are fired in order specified by the pattern - second callback first, then first, then fourth, then third, etc...
+
+**The important thing is, they are ALWAYS fired in THIS order. It's predictable, deterministic !**
+
+This was very simple example of an issue, that can easily happen when dealing with asynchronous APIs. The same type of issues can happen in web applications as well, for instance by sending multiple xhr requests.
+**In production, this behavior is out of our control. However, in order to guarantee correct behavior of our code, we need to be sure that it handles correctly all these different situations. The best way to do that is by simulating these situations in a fully controlled - a deterministic way.**
+
+
+
+The full source code of `predictableNextTick` implementation can be found on [github].
+
+The full source code of this example is on [gist].
+
+
+[thread deadlocks]: http://en.wikipedia.org/wiki/Deadlock
+[race conditions]: http://en.wikipedia.org/wiki/Race_condition
+[node-mocks]: https://github.com/vojtajina/node-mocks
+[github]: https://github.com/vojtajina/node-mocks/blob/master/lib/util.js
+[gist]: https://gist.github.com/3283670
+[process.nextTick]: http://nodejs.org/api/process.html#process_process_nexttick_callback
View
15 articles/make-your-tests-deterministic/get-last-modified.js
@@ -0,0 +1,15 @@
+var fs = require('fs');
+
+var getLastModified = function(files, done) {
+ var timestamps = [];
+
+ files.forEach(function(file) {
+ fs.stat(file, function(err, stat) {
+ timestamps.push(stat.mtime);
+
+ if (timestamps.length === files.length) {
+ done(null, timestamps);
+ }
+ });
+ });
+};
View
30 articles/make-your-tests-deterministic/get-last-modified.spec.js
@@ -0,0 +1,30 @@
+// Jasmine Syntax
+describe('getLastModified', function() {
+
+ var mocks = require('mocks');
+ var mockery = {
+ // reate a fake in-memory FS
+ fs: mocks.fs.create({
+ 'one.js': mocks.fs.file('2012-01-01'),
+ 'two.js': mocks.fs.file('2012-02-02'),
+ 'three.js': mocks.fs.file('2012-02-02')
+ })
+ };
+
+ // load the module (using fake FS)
+ var getLastModified = mocks.loadFile('get-last-modified.js', mockery).getLastModified;
+
+
+ it('should return last modified timestamps for every file', function() {
+ var spy = jasmine.createSpy('done').andCallFake(function(err, timestamps) {
+ expect(timestamps).toEqual([
+ new Date('2012-01-01'), new Date('2012-02-02'), new Date('2012-02-02')
+ ]);
+ });
+
+ getLastModified(['/one.js', '/two.js', '/three.js'], spy);
+
+ // wait for done callback
+ waitsFor(function() {return spy.callCount;});
+ });
+});
View
4 articles/testing-private-state-and-mocking-deps.markdown
@@ -4,7 +4,7 @@ Date: Sun Jan 08 2012 00:11:06 GMT-0800 (PST)
Node: v0.6.5
-During Christmas I've been working on [SlimJim] and found some tricks how to make my testing life
+During Christmas I've been working on [Testacular] and found some tricks how to make my testing life
easier. It's nothing special at all, just a simple way **how to access private state of a module**
and **how to mock out some dependencies**. I've found these two techniques pretty usefull, so I
believe it might help someone else as well...
@@ -114,7 +114,7 @@ stuff. The important point here is, that **it's fast, because it doesn't touch t
and still does test what needs to be tested - our code**.
-[SlimJim]: http://github.com/vojtajina/slim-jim/
+[Testacular]: http://github.com/vojtajina/testacular/
[G.Meszaros]: http://xunitpatterns.com/Test%20Double.html
[vm.runInNewContext]: http://nodejs.org/docs/latest/api/vm.html#vm.runInNewContext
[Dart]: http://www.dartlang.org/
Please sign in to comment.
Something went wrong with that request. Please try again.