/
atom.xml
250 lines (191 loc) · 11.7 KB
/
atom.xml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
<?xml version="1.0" encoding="utf-8"?>
<feed xmlns="http://www.w3.org/2005/Atom">
<title><![CDATA[Category: Unit Testing | TJ VanToll]]></title>
<link href="http://tjvantoll.com/blog/categories/unit-testing/atom.xml" rel="self"/>
<link href="http://tjvantoll.com/"/>
<updated>2013-01-21T21:41:18-05:00</updated>
<id>http://tjvantoll.com/</id>
<author>
<name><![CDATA[TJ VanToll]]></name>
<email><![CDATA[tj.vantoll@gmail.com]]></email>
</author>
<generator uri="http://octopress.org/">Octopress</generator>
<entry>
<title type="html"><![CDATA[Logging Test Failures in a PhantomJS QUnit Runner]]></title>
<link href="http://tjvantoll.com/2012/08/22/logging-test-failures-in-a-phantomjs-qunit-runner/"/>
<updated>2012-08-22T00:00:00-04:00</updated>
<id>http://tjvantoll.com/2012/08/22/logging-test-failures-in-a-phantomjs-qunit-runner</id>
<content type="html"><![CDATA[<p><a href="http://phantomjs.com">PhantomJS</a> provides an easy means of automating <a href="http://qunitjs.com">QUnit</a> tests; it even provides a <a href="https://github.com/ariya/phantomjs/blob/master/examples/run-qunit.js">test runner</a> that you can simply copy into your project to run them.</p>
<p>The output of said runner displays the number of tests ran and the number that passed. For example, here's an example of the output when I use the default runner on jQuery UI's spinner test suite:</p>
<p><code>text PhantomJS + QUnit Run Output
tj-cpu:spinner tj$ phantomjs run-qunit.js spinner.html
Tests completed in 492 milliseconds
489 tests of 489 passed, 0 failed.
'waitFor()' finished in 587ms.
</code></p>
<p>Which is great, but if something fails you only get the following:</p>
<p><code>text PhantomJS + QUnit Run Output with Failures
tj-cpu:myproject tj$ phantomjs run-qunit.js spinner.html
'waitFor()' finished in 630ms.
Tests completed in 535 milliseconds.
486 tests of 489 passed, 3 failed.
</code></p>
<p>The provided runner doesn't provide any additional information about the tests that failed. Luckily PhantomJS and QUnit make it trivial to customize the output to meet your needs.</p>
<!--more-->
<h3>Logging</h3>
<p>PhantomJS's <a href="http://code.google.com/p/phantomjs/wiki/Interface#onConsoleMessage">page.onConsoleMessage</a> callback can be used to redirect the browser's JavaScript console logging. The provided test runner uses this callback to redirect the output to the command line instead of the headless browser (where you would never see it).</p>
<p>```javascript Snippet from Default run-qunit.js
// Route "console.log()" calls from within the Page context to the main Phantom context (i.e. current "this")
page.onConsoleMessage = function(msg) {</p>
<pre><code>console.log(msg);
</code></pre>
<p>};
```</p>
<p>Meaning, when running QUnit tests via PhantomJS, the output of any <code>console.log</code> statements will appear on the command line.</p>
<h3>QUnit</h3>
<p>QUnit provides a <a href="http://api.qunitjs.com/">comprehensive API</a> including <a href="http://api.qunitjs.com/category/callbacks/">callback hooks</a> for common tasks such as tests starting and finishing. This is perfect for logging information about the tests themselves.</p>
<p>For example you could use the following to log the associated message for every test that fails:</p>
<p>``` javascript Logging Each Test Failure with QUnit
QUnit.log(function(details) {</p>
<pre><code>if (!details.result) {
console.log(details.message);
}
</code></pre>
<p>});
```</p>
<p>If we apply this our failing test suite we now get the following:</p>
<p><code>text
tj-mac:spinner tj3$ phantomjs run-qunit.js spinner.html
min from markup
stop from options
blur after many keys
'waitFor()' finished in 579ms.
Tests completed in 483 milliseconds.
486 tests of 489 passed, 3 failed.
</code></p>
<p>Better, but still not terribly useful. In order to provide a useful report of failed tests you need to combine more of QUnit's API callbacks with some basic text formatting. Here's a more comprehensive example:</p>
<p>``` javascript QUnit Test Results with a Formatted Display of Failed Tests
(function() {</p>
<pre><code>var module = '',
test = '',
lastModuleLogged = '',
lastTestLogged = '',
failuresOnCurrentTest = 0,
failureFound = false;
QUnit.moduleStart(function(details) {
module = details.name;
});
QUnit.testStart(function(details) {
test = details.name;
});
QUnit.log(function(details) {
if (!details.result) {
if (!failureFound) {
failureFound = true;
console.log('');
console.log('/*********************************************************************/');
console.log('/************************** FAILURE SUMMARY **************************/');
console.log('/*********************************************************************/');
}
if (lastModuleLogged != module) {
console.log('');
console.log('-----------------------------------------------------------------------');
console.log('Module: ' + module);
}
if (lastTestLogged != test) {
failuresOnCurrentTest = 1;
console.log('-----------------------------------------------------------------------');
console.log('Test: ' + test);
} else {
failuresOnCurrentTest++;
}
console.log(' ' + failuresOnCurrentTest + ') Message: ' + details.message);
if (typeof details.expected !== 'undefined') {
console.log(' Expected: ' + details.expected);
console.log(' Actual: ' + details.actual);
}
if (typeof details.source !== 'undefined') {
console.log(' Source: ' + details.source);
}
lastModuleLogged = module;
lastTestLogged = test;
}
});
QUnit.done(function(details) {
if (details.failed > 0) {
console.log('-----------------------------------------------------------------------');
console.log('');
}
});
</code></pre>
<p>}());
```</p>
<p>Now running tests with failures will produce something like the following:</p>
<p>``` text PhantomJS + QUnit Output with Failures
tj-cpu:spinner tj$ phantomjs run-qunit.js spinner.html</p>
<p>/<strong><strong><strong><strong><strong><strong><strong><strong><em><strong><strong><strong><strong><strong><strong><strong><strong><strong><strong><strong><strong><strong><strong><strong><strong><strong><strong><strong><strong><strong><strong><strong><strong><strong><strong>/
/</strong></strong></strong></strong></strong></strong></strong></strong></strong></strong></strong></strong></strong> FAILURE SUMMARY </strong></strong></strong></strong></strong></strong></strong></strong></strong></strong></strong></strong></strong>/
/</em></strong></strong></strong></strong></strong></strong></strong></strong>****************************************************/</p>
<hr />
<h2>Module: spinner: core</h2>
<p>Test: reading HTML5 attributes
1) Message: min from markup</p>
<pre><code>Expected: -1000
Actual: -100
Source: at file:///Applications/XAMPP/xamppfiles/htdocs/jquery/jquery-ui/external/qunit.js:447
at file:///Applications/XAMPP/xamppfiles/htdocs/jquery/jquery-ui/tests/unit/spinner/spinner_core.js:137
at file:///Applications/XAMPP/xamppfiles/htdocs/jquery/jquery-ui/external/qunit.js:134
at file:///Applications/XAMPP/xamppfiles/htdocs/jquery/jquery-ui/external/qunit.js:277
at process (file:///Applications/XAMPP/xamppfiles/htdocs/jquery/jquery-ui/external/qunit.js:1233)
at file:///Applications/XAMPP/xamppfiles/htdocs/jquery/jquery-ui/external/qunit.js:376
</code></pre>
<p> 2) Message: stop from options</p>
<pre><code>Expected: 50
Actual: 5
Source: at file:///Applications/XAMPP/xamppfiles/htdocs/jquery/jquery-ui/external/qunit.js:447
at file:///Applications/XAMPP/xamppfiles/htdocs/jquery/jquery-ui/tests/unit/spinner/spinner_core.js:148
at file:///Applications/XAMPP/xamppfiles/htdocs/jquery/jquery-ui/external/qunit.js:134
at file:///Applications/XAMPP/xamppfiles/htdocs/jquery/jquery-ui/external/qunit.js:277
at process (file:///Applications/XAMPP/xamppfiles/htdocs/jquery/jquery-ui/external/qunit.js:1233)
at file:///Applications/XAMPP/xamppfiles/htdocs/jquery/jquery-ui/external/qunit.js:376
</code></pre>
<hr />
<h2>Module: spinner: events</h2>
<p>Test: change
1) Message: blur after many keys</p>
<pre><code>Source: at file:///Applications/XAMPP/xamppfiles/htdocs/jquery/jquery-ui/external/qunit.js:426
at file:///Applications/XAMPP/xamppfiles/htdocs/jquery/jquery-ui/tests/unit/spinner/spinner_events.js:130
at file:///Applications/XAMPP/xamppfiles/htdocs/jquery/jquery-ui/ui/jquery.ui.widget.js:454
at file:///Applications/XAMPP/xamppfiles/htdocs/jquery/jquery-ui/ui/jquery.ui.spinner.js:109
at handlerProxy (file:///Applications/XAMPP/xamppfiles/htdocs/jquery/jquery-ui/ui/jquery.ui.widget.js:371)
at file:///Applications/XAMPP/xamppfiles/htdocs/jquery/jquery-ui/tests/jquery-1.8.0.js:3061
at file:///Applications/XAMPP/xamppfiles/htdocs/jquery/jquery-ui/tests/jquery-1.8.0.js:2677
at file:///Applications/XAMPP/xamppfiles/htdocs/jquery/jquery-ui/tests/jquery-1.8.0.js:2941
at file:///Applications/XAMPP/xamppfiles/htdocs/jquery/jquery-ui/tests/jquery-1.8.0.js:3607
at file:///Applications/XAMPP/xamppfiles/htdocs/jquery/jquery-ui/tests/jquery-1.8.0.js:611
at file:///Applications/XAMPP/xamppfiles/htdocs/jquery/jquery-ui/tests/jquery-1.8.0.js:241
at file:///Applications/XAMPP/xamppfiles/htdocs/jquery/jquery-ui/tests/jquery-1.8.0.js:3608
at file:///Applications/XAMPP/xamppfiles/htdocs/jquery/jquery-ui/tests/jquery-1.8.0.js:3660
at file:///Applications/XAMPP/xamppfiles/htdocs/jquery/jquery-ui/tests/unit/spinner/spinner_events.js:165
at file:///Applications/XAMPP/xamppfiles/htdocs/jquery/jquery-ui/external/qunit.js:134
at file:///Applications/XAMPP/xamppfiles/htdocs/jquery/jquery-ui/external/qunit.js:277
at process (file:///Applications/XAMPP/xamppfiles/htdocs/jquery/jquery-ui/external/qunit.js:1233)
at file:///Applications/XAMPP/xamppfiles/htdocs/jquery/jquery-ui/external/qunit.js:376
</code></pre>
<hr />
<p>'waitFor()' finished in 590ms.
Tests completed in 494 milliseconds.
486 tests of 489 passed, 3 failed.
```</p>
<p>This might be a bit excessive for some but I like being able to quickly see information about what failed from the command line. Feel free to use this and alter it to your liking.</p>
<h3>TAP Format</h3>
<p>If you want to output the test results in <a href="http://en.wikipedia.org/wiki/Test_Anything_Protocol">TAP format</a> the <a href="https://github.com/twada/qunit-tap">QUnit-tap</a> plugin provides an excellent implementation using the same approach described above.</p>
<h3>Update (August 31st, 2012)</h3>
<p><div class='embed tweet'><blockquote class="twitter-tweet" data-in-reply-to="238820410278436864"><p><a href="https://twitter.com/tjvantoll"><s>@</s><b>tjvantoll</b></a> did you try the phantomjs addon? <a href="https://t.co/6Orpiwda" title="https://github.com/jquery/qunit/tree/master/addons/phantomjs">github.com/jquery/qunit/t…</a></p>— QUnit (@qunitjs) <a href="https://twitter.com/qunitjs/status/240049283997503488" data-datetime="2012-08-27T11:33:03+00:00">August 27, 2012</a></blockquote>
<script src="//platform.twitter.com/widgets.js" charset="utf-8"></script></div></p>
<p>QUnit provides its own <a href="https://github.com/jquery/qunit/tree/master/addons/phantomjs">test runner for use with PhantomJS</a> which logs information on the test run using its own APIs.</p>
<p>One difference with doing the logging in the runner rather than in the HTML is that you will not get the console logging when you run the tests in the browser, which is usually what you want.</p>
<p>Either approach can be used and customized to provide the logging you'd like.</p>
]]></content>
</entry>
</feed>