Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
tree: 60b9f37431
Fetching contributors…

Cannot retrieve contributors at this time

368 lines (353 sloc) 21.809 kb
<html>
<head>
<title>JSCheck</title>
<style>
body {background-color: linen}
h1 {font-family: Arial, Helvetica, sans-serif; font-size: 18pt;
font-weight: bolder; font-variant: small-caps; letter-spacing: 4pt;
text-align: left; padding-top: 0pt; margin-top: 0pt;
padding-bottom: 0pt; margin-bottom: 0pt; border-color: black;
border-style: solid; border-bottom-width: 2pt;
border-top-width: 0pt; border-right-width: 0pt; border-left-width: 0pt}
h2 {font-family: Arial, Helvetica, sans-serif; font-size: 18pt;
font-weight: bold; font-variant: normal;
text-align: left; padding-bottom: 0pt; border-color: black;
border-style: solid; border-bottom-width: 1pt;
border-top-width: 0pt; border-right-width: 0pt; border-left-width: 0pt}
h3 {font-family: Arial, Helvetica, sans-serif; font-size: 16pt;
font-weight: bold;
text-align: left; padding-bottom: 0pt; border-color: black;}
h4 {font-family: Monospace; font-size: 14pt;
font-weight: bold;
text-align: left; padding-bottom: 0pt; }
i {
font-family: serif;
}
pre {
font-family: Monospace;
margin-left: 1in;
}
p {margin-left: 10pt}
</style>
</head>
<body>
<h1>JSCheck</h1>
<p>Douglas Crockford<br>
2012-04-24
</p>
<p>JSCheck is a testing tool for JavaScript. It was inspired by <a href="http://en.wikipedia.org/wiki/QuickCheck">QuickCheck</a>, a testing tool for Haskell developed by Koen Claessen and John Hughes of Chalmers University of Technology.</p>
<p>JSCheck is a specification-driven testing tool. From a description of the properties of a system, function, or object, it will generate random test cases attempting to prove those properties, and then report its findings. That can be especially effective in managing the evolution of a program because it can show the conformance of new code to old code. It also provides an interesting level of self-documentation, because the executable specifications it relies on can provide a good view of the workings of a program.</p>
<p>The JSCheck program is loaded from the <code>jscheck.js</code> file. It produces a single global variable, <code>JSC</code>, which contains the JSCheck object.</p>
<p>JSCheck is concerned with the specification and checking of <i>claims</i>. (We use the term <i>claim</i> instead of <i>property</i> to avoid confusion with JavaScript's use of <i>property</i> to mean a member of an object.) To create a claim, call <code>JSC.claim</code>, passing in</p>
<ul>
<li>A name, which is a description of the claim</li>
<li>A predicate, which is a function that returns <code>true</code> when the claim holds</li>
<li>An array of specifiers, which describe the types of the predicate's parameters</li>
<li>Optionally, a classifier function that takes the same arguments as the predicate</li>
</ul>
<p><code>JSC.claim</code> returns a claim function, which may be passed as an argument to the <code>JSC.check</code> function, which will randomly generate the cases that will attempt to reject the claim. You can set the number of cases generated per claim with the <code>JSC.reps</code> function.</p>
<p>The source is available at <a href="https://github.com/douglascrockford/JSCheck">https://github.com/douglascrockford/JSCheck</a>. This page is available at <a href="http://www.JSCheck.org/">http://www.JSCheck.org/</a>.</p>
<h2>Making a Claim</h2>
<p>To make a claim, you pass three or four components to <code>JSC.claim</code>, which will then return a function.</p>
<h3>name</h3>
<p>The name is descriptive text that will be used in making the report.</p>
<h3>predicate</h3>
<p>The predicate is a function that will return true if the claim holds. The predicate will do something with the system in question, perhaps examining its result examining the consistency of its data structures. If you are testing functions that do encoding and decoding, the predicate can assert things like</p>
<pre>return value === decode(encode(value))</pre>
<p>You won't need to select the <code>value</code>. JSCheck will generate random values for you.</p>
<h3>specifiers</h3>
<p>An array of specifiers describe the types of the predicate's arguments. (From a procedural perspective, specifiers are generators, but JavaScript may get a new generator feature which is very different, so avoid confusion, we will take a declarative view.)</p>
<p>JSCheck provides a small library of specifiers which you can use in your claim. For example, <code>JSC.integer(10)</code> declares that a parameter should an integer between 1 and 10. <code>JSC.one_of(['Curly, 'Larry', 'Moe'])</code> declares that a parameter can be one of three strings. Some of the specifiers can be combined, so <code>JSC.array(JSC.integer(10), JSC.character('a', 'z'))</code> declares that a parameter can be an array of 1 to 10 lowercase letters.</p>
<p>An array of specifiers can also contain constants (such as string, numbers, or objects), so you can pass anything you need to into the predicate. If you need to pass in a function, then you must to wrap the function value with the <code>JSC.literal</code> specifier.</p>
<p>You can also create your own specifiers.</p>
<h3>classifier</h3>
<p>You can optionally pass a classifier function as part of the claim. The classifier will receive the same arguments as the predicate. A classifier can do two things:</p>
<ol>
<li>It can examine the arguments, and return a string which classifies the case. The string is descriptive. The report can include how many of the cases belong to each classification. This can be used to identify the classes that are trivial or problematic, or to help analyze the results.</li>
<li>Since the cases are being generated randomly, some cases might not be meaningful or useful. The classifier can have a case rejected by returning <code>false</code>. JSCheck will attempt to generate another case to replace it. It is recommended that the classifier reject fewer than 90% of the cases. If you accepting less than 10% of the potential cases, then you should probably reformulate your claim.</li>
</ol>
<h2>The JSCheck functions</h2>
<p>The JSCheck object contains several functions.</p>
<h3>Configuration:</h3>
<p>The configuration functions set up the JSCheck object.</p>
<h4>JSC.clear()</h4>
<p>The <code>clear</code> function reinitializes the JSCheck object, discarding its collection of claims and groups.</p>
<p>It returns the JSCheck object.</p>
<h4>
JSC.detail(<i>level</i>)</h4>
<p>By setting the <i>level</i> of detail to a particular number, you can determine how much information is included in the report. The report will be delivered to the function designated with <code>JSC.on_report</code>. The default is 3.</p>
<ol start=0>
<li><i>none</i>: There will be no report</li>
<li> <i>terse</i>: There will be a minimal report, showing the pass score of each claim</li>
<li> <i>failures</i>: The cases that fail will be reported</li>
<li> <i>qualification</i>: The qualification summaries will be reported (default)</li>
<li> <i>verbose</i>: All cases will be reported</li>
</ol>
<p>It returns the JSCheck object.</p>
<h4>JSC.on_fail(<i>function(object)</i>)</h4>
<p>The <code>on_fail</code> function allows the registration of a callback <i>function</i> that will be given an object for each failed case. This can be used to trigger a throw to cancel further tests or to begin deeper processing. The callback will be passed an object containing these properties:</p>
<ul>
<li><code>args</code>: The array of arguments of the case.</li>
<li><code>claim</code>: The claim function.</li>
<li><code>classifier</code>: The classifier function.</li>
<li><code>classification</code>: The classification string of the case, if there is one.<br>
</li>
<li><code>exception</code>: The exception object that was thrown by the predicate, if there was one.<br>
</li>
<li><code>group</code>: The claim group name, if there is one.<br>
</li>
<li><code>name</code>: The name of the claim.<br>
</li>
<li><code>pass</code>: The result value of the claim. It would have been true if it had succeeded.</li>
<li><code>predicate</code>: The predicate function.</li>
<li><code>signature</code>: The signature array.</li>
</ul>
<p>It returns the JSCheck object.</p>
<h4>
JSC.on_pass(<i>function(object)</i>)</h4>
<p>The <code>on_fail</code> function allows the registration of a callback <i>function</i> that will be given an object for each passing case. This can be used to trigger further tests or to begin deeper processing or reporting. The callback will be passed an object containing these properties:</p>
<ul>
<li><code>args</code>: The array of arguments of the case.</li>
<li><code>claim</code>: The claim function.</li>
<li><code>classifier</code>: The classifier function.</li>
<li><code>classification</code>: The classification string of the case, if there is one.<br>
</li>
<li><code>exception</code>: <code>undefined</code>.<br>
</li>
<li><code>group</code>: The claim group name, if there is one.<br>
</li>
<li><code>name</code>: The name of the claim.<br>
</li>
<li><code>pass</code>: <code>true</code>.</li>
<li><code>predicate</code>: The predicate function.</li>
<li><code>signature</code>: The signature array.</li>
</ul>
<p>It returns the JSCheck object.</p>
<h4>JSC.on_report(<i>function(string)</i>)</h4>
<p>The <code>on_report</code> function allows the registration of a callback function that will be given a string containing the results for each claim. The callback could route the report to a console or a log or <code>alert</code>. The level of detail is set with <code>JSC.detail</code>.</p>
<p>It returns the JSCheck object.</p>
<h4>
JSC.reps(<i>repetitions</i>)</h4>
<p>The <code>reps</code> function allows setting the number of repetitions per claim. The default is 100. </p>
<p>It returns the JSCheck object.</p>
<h3>Specifiers:</h3>
<p>A specifier is a function that returns a function that can generate values of a particular type. The specifiers are used in building the array of specifiers that is used to construct a claim.</p>
<h4>
JSC.array(<i>array</i>)</h4>
<p>The <code>array</code> specifier takes an <i>array</i> as a template. It will go through the <i>array</i>, expanding the specifiers it contains. So, for example,</p>
<pre>JSC.array([
JSC.integer(),
JSC.number(100),
JSC.string(8, JSC.character('A', 'Z'))
])</pre>
<p>can generate arrays like</p>
<pre>[3,21.228644298389554,"TJFJPLQA"]
[5,57.05485427752137,"CWQDVXWY"]
[7,91.98980208020657,"QVMGNVXK"]
[11,87.07735128700733,"GXBSVLKJ"]
...</pre>
<h4>
JSC.array(<i>dimension</i>, <i>value</i>)</h4>
<p>The <code>array</code> specifier takes a number and a value, and produces an array using the number as the length of the array, populating the array with the value. So, for example,</p>
<pre>JSC.array(3, JSC.integer(640))</pre>
<p>can generate arrays like</p>
<pre>[305,603,371]
[561,623,477]
[263,534,530]
[163,148,17]
...</pre>
<h4>
JSC.boolean()</h4>
<p>The <code>boolean</code> specifier will produce <code>true</code> and <code>false</code> with equal probability.</p>
<h4>
JSC.boolean(<i>bias</i>)</h4>
<p>The <code>boolean</code> specifier will produce <code>true</code> and <code>false</code>. If the <i>bias</i> is 0.50, it will produce them with equal probability. A lower <i>bias</i> will produce more <code>false</code>s, and a higher <i>bias</i> will produce more <code>true</code>s.</p>
<h4>
JSC.character(<i>code</i>)</h4>
<p>The <code>character</code> specifier treats its argument as a char code, generating a character.</p>
<h4>
JSC.character(<i>min_character</i>, <i>max_character</i>)</h4>
<p>The <code>character</code> specifier generates characters within a given range.</p>
<h4>
JSC.integer()</h4>
<p>The <code>integer</code> specifier generates prime numbers<i></i>. Sometimes when testing formulas, it is useful to plug prime numbers into the variables.</p>
<h4>JSC.integer(<i>i</i>)</h4>
<p>The <code>integer</code> specifier generates an integer between 1 and <i>i</i>.</p>
<h4>
JSC.integer(<i>i</i>, <i>j</i>)</h4>
<p>The <code>integer</code> specifier generates an integer between <i>i</i> and <i>j</i>.</p>
<h4>
JSC.literal(<i>value</i>)</h4>
<p>The <code>literal</code> specifier generates the <i>value</i> without interpreting it. For most values (strings, numbers, boolean, objects, arrays), the <code>literal</code> specifier is not needed. It is needed if you want to pass a function <i>value</i> to a predicate, because function values are assumed to be the products of generators.</p>
<h4>
JSC.number(<i>x</i>)</h4>
<p>The <code>number</code> specifier produces random numbers between 0 and <i>x</i>.</p>
<h4>
JSC.number(<i>x</i>, <i>y</i>)</h4>
<p>The number specifier produces random numbers between <i>x</i> and <i>y</i>.</p>
<h4>
JSC.object(<i>object</i>) </h4>
<p>The <code>object</code> specifier takes an <i>object</i> as a template. It will go through the enumerable own properties of <i>object</i>, expanding the specifiers it contains. So, for example,</p>
<pre>JSC.object({
left: JSC.integer(640),
top: JSC.integer(480),
color: one_of(['black', 'white', 'red', 'blue', 'green', 'gray'])
})</pre>
<p>can generate objects like</p>
<pre>{"left":104,"top":139,"color":"gray"}
{"left":62,"top":96,"color":"white"}
{"left":501,"top":164,"color":"white"}
{"left":584,"top":85,"color":"white"}
...</pre>
<h4>JSC.object(keys, values)</h4>
<p>The <code>object</code> specifier takes an array of keys and produces an object using those keys. The values are taken from an array of values or a specifier. So for example,</p>
<pre>JSC.object(
JSC.array(JSC.integer(3, 8), JSC.string(4, JSC.character('a', 'z'))),
JSC.boolean()
)
</pre>
<p>can generate objects like </p>
<pre>{"jodo":true,"zhzm":false,"rcqz":true}
{"odcr":true,"azax":true,"bnfx":true,"hmmc":false}
{"wjew":true,"kgqj":true,"abid":true,"cjva":false,"qsgj":true,"wtsu":true}
{"qtbo":false,"vqzc":false,"zpij":true,"ogss":false,"lxnp":false,"psso":true,"irha":true,"ghnj":true}
...</pre>
<p>and</p>
<pre>JSC.object(
['x', 'y', 'z'],
[JSC.integer(320), JSC.integer(240), JSC.integer(100)]
)</pre>
<p>can generate objects like</p>
<pre>{"x":99,"y":51,"z":51}
{"x":114,"y":166,"z":82}
{"x":35,"y":124,"z":60}
{"x":13,"y":41,"z":63}
...</pre>
<h4>JSC.one_of(<i>array</i>)</h4>
<p>The <code>one_of</code> specifier takes an array of specifiers, and selects values from the <i>array</i> with equal probability. So, for example</p>
<pre>JSC.one_of([
JSC.number(),
JSC.boolean(),
null
])</pre>
<p>produces values like</p>
<pre>0.09817210142500699
0.3351482313591987
null
false
...</pre>
<h4>
JSC.one_of(<i>array</i>, <i>weights</i>)</h4>
<p>The one_of specifier takes an <i>array</i> of specifiers and an array of <i>weights</i>. The <i>weights</i> are used to adjust the probabilities. So for example,</p>
<pre>JSC.one_of(
[1,2,3,4,5,6,7,8,9,10],
[1,2,3,4,5,6,7,8,9,10]
)</pre>
produces values like
<pre>8
10
1
10
...</pre>
<h4>
JSC.sequence(<i>array</i>)</h4>
<p>The <code>sequence</code> specifier takes an <i>array</i> of values, and produces them in sequence, repeating the sequence as needed.</p>
<h4>
JSC.string(<i>value</i>)</h4>
<p>The <code>string</code> specifier generates the stringification of the <i>value</i>. So for example,</p>
<pre>JSC.string(JSC.integer(1000, 9999))</pre>
<p>produces values like</p>
<pre>"4791"
"9523"
"2774"
"4288"
...</pre>
<h4>
JSC.string(<i>number</i>, <i>value</i>)</h4>
<p>The <code>string</code> specifier generates strings by joining some number of values. So for example,</p>
<pre>JSC.string(JSC.integer(1, 8), JSC.one_of("aeiou")))</pre>
<p>produces values like</p>
<pre>"ieauae"
"io"
"iuieio"
"iuu"
...</pre>
<h3>Claim processing:</h3>
<p>The <code>JSC.claim</code> function creates claims, and the <code>JSC.check</code> function generates the cases and produces the reports.</p>
<h4>
JSC.check()</h4>
<p>Process all of the claims that have been construct since the beginning or the most recent call to <code>JSC.clear</code>.</p>
<p>It returns <code>true</code> if every case passes.</p>
<h4>
JSC.check(<i>group</i>)</h4>
<p>Process all of the claims of a specific group.</p>
<p>It returns <code>true</code> if every case passes.</p>
<h4>
JSC.check(<i>claim</i>)</h4>
<p>Process the specific <i>claim</i> function.</p>
<p>It returns <code>true</code> if every case passes.</p>
<h4>
JSC.claim(<i>name</i>, <i>predicate</i>, <i>specifiers</i>)</h4>
<p>The <code>claim</code> function takes a <i>name</i>, a <i>predicate</i> function, and an array of <i>specifiers</i>.</p>
<p>The <i>predicate</i> function should return <code>true</code> if a case passes. It will take a list of arguments that is generated by the array of <i>specifiers</i>. The array of <i>specifiers</i> looks like a type declaration for the <i>predicate</i> function.</p>
<p>It returns a function that can be processed by <code>JSC.check</code>.</p>
<h4>
JSC.claim(<i>name</i>, <i>predicate</i>, <i>specifiers</i>, <i>classifier</i>)</h4>
<p>The <code>claim</code> function takes a <i>name</i>, a <i>predicate</i> function, and an array of <i>specifiers</i>.</p>
<p>The <i>predicate</i> function should return <code>true</code> if a case passes. It will take a list of arguments that is generated by the array of <i>specifiers</i>. The array of <i>specifiers</i> looks like a type declaration for the <i>predicate</i> function.</p>
<p>The <i>classifier</i> function is called before the <i>predicate</i> function. It gets a chance to determine if the random values in its arguments will be a reasonable case. It can return <code>false</code> if the case should be thrown out and a new case generated to replace it. The <i>classifier</i> function can also return a descriptive string that describes the case. These can be counted and displayed in the report.</p>
<p>It returns a function that can be processed by <code>JSC.check</code>.</p>
<h4>
JSC.group(<i>name</i>)</h4>
<p>The
<code>group</code> function is used to specify a group <i>name</i> that will be attached to all new claims. This makes it easy to process a group of claims together.</p>
<h4>JSC.test(<i>name</i>, <i>predicate</i>, <i>specifiers</i>, <i>classifier</i>)</h4>
<p>The <code>test</code> function calls the <code>check</code> function, passing the result of the <code>claim</code> function.</p>
<h2>Using JSCheck</h2>
<p>Since JSCheck performs a useful specification and description function as well as a testing function, it is recommended (but not required) that claims be inserted into the relevant source code, and not in separate source files. <a href="https://github.com/douglascrockford/JSDev">JSDev</a> can make this easy to manage, so that claims can be removed automatically from production code. All of the calls to JSC can be hidden in special comments, which are activated during development, and removed by minification in production.</p>
<h2>Writing specifiers</h2>
<p>JSCheck provides a small set of specifiers that can be combined in many ways. But for some purposes, you may need to create your own specifiers.</p>
<p>It is easy to do. A specifier is a function that returns a function. </p>
<pre>my_specifier = function specifier(param1, param2) {
// per claim processing happens in here
return function generator() {
// per case processing happen in here
return value;
};
}</pre>
<p>The generator function that is returned will be stored in the specifiers array, and will be called for each value that needs to be generated. It will have access to the specifier's parameters. Its arguments might be other specifiers, so if an argument is a function, use the result of calling the function.</p>
<pre>intermediate_value = typeof param1 === 'function'
? param1()
: param1;</pre>
<h2>Demonstration</h2>
<p>One difficulty in demonstrating testing systems is that the exposition of the system to be tested is usually significantly more complex than the testing tool being demonstrated. So in this case, we will be testing a trivial function. We will make an incorrect claim. JSCheck will help us to find the error in the claim. It might seem counter productive to demonstrate bad claim making, but it turns out that it is as important to get the claims right as it is to get the program right.</p>
<p>We are going to test the <code>le</code> function.</p>
<pre>function le(a, b) {
return a <= b;
}</pre>
<p>We will construct a claim. Our predicate simply returns the result of <code>le</code>. It takes two integers, one with a max of 10 and another with a max of 20. We will classify the cases by the relationship between the arguments.</p>
<pre>
/*check
JSC.test("Less than", function (a, b) {
return le(a, b);
}, [
JSC.integer(10),
JSC.integer(20)
], function (a, b) {
if (a < b) {
return 'lt';
} else if (a === b) {
return 'eq';
} else {
return 'gt';
}
});
*/</pre>
<p>But when we check the claim, many cases fail. The summary of the specifiers tells the story: </p>
<pre>eq pass 7
gt pass 0 fail 22
lt pass 71
</pre>
<p>The predicate failed because 22 of the generated cases had an <code>a</code> that was larger than <code>b</code>. This is because <code>JSC.integer(10)</code> produces from the range 1 to 10, and <code>JSC.integer(20)</code> produces from the range 1 to 20. Sometimes the first value will be larger. This tells us that we should make the predicate more sophisticated, or we could have the specifier return <code>false</code> instead of <code>'gt'</code> to reject those cases.</p>
</body>
</html>
Jump to Line
Something went wrong with that request. Please try again.