Skip to content


Subversion checkout URL

You can clone with
Download ZIP
100644 109 lines (82 sloc) 4.7 KB
001aed2 Added t/spec/README
lwall authored
1 The Official Perl 6 Test Suite (someday...)
059465d @moritz update README, now that the testsuite is decoupled from the Pugs repo
moritz authored
3 This is the Official Perl 6 test suite. It evolved out of the pugs
4 test suite, and is now maintained in a separate repository.
001aed2 Added t/spec/README
lwall authored
059465d @moritz update README, now that the testsuite is decoupled from the Pugs repo
moritz authored
6 Its purpose is to validate implementations that wish to be known
7 as a conforming Perl 6 implementation.
9 Please consider this test suite to be the result
001aed2 Added t/spec/README
lwall authored
10 of an ongoing negotiation: since many of these tests are inspired
11 by seeing how the various prototype implementations are screwed up
12 (or even more likely, how the design of Perl 6 is screwed up), this
e2676ba @jamesrom Typo.
jamesrom authored
13 test suite should be considered a work in progress until one or more
001aed2 Added t/spec/README
lwall authored
14 of the implementations gets close to passing the entire test suite,
15 at which point we will freeze version 6.0 of the test suite, and any
16 implementation that passes it can then call itself Perl 6.
18 As they develop, different implementations will certainly be in
19 different states of readiness with respect to the test suite, so
20 in order for the various implementations to track their progress
21 independently, we've established a mechanism for "fudging" the
22 tests in a kind of failsoft fashion. To pass a test officially,
23 an implementation must be able to run a test file unmodified, but an
24 implementation may (temporarily) skip tests or mark them as "todo" via
25 the fudging mechanism, which is implemented via the fudge preprocessor.
26 Individual implementations are not allowed to modify the actual test
27 code, but may insert line comments before each actual test (or block
28 of tests) that changes how those tests are to be treated for this
29 platform. The fudge preprocessor pays attention only to the comments
30 that belong to the current platform and ignores all the rest. If your
31 platform is named "humpty" then your special comment lines look like:
33 #?humpty: [NUM] VERB ARGS
fa83ba8 [fudge] Didn't do #?DOES on subs right
lwall authored
35 (The colon is optional.)
001aed2 Added t/spec/README
lwall authored
37 The optional NUM says how many statements or blocks to apply the
a73aff0 [fudge] Clarify meaning of statement in README
lwall authored
38 verb to. (If not supplied, a value of 1 is assumed). A statement
39 is arbitrarily defined as one or more lines starting with a test call
40 and ending in semicolon (with an optional comment).
42 VERBs include:
001aed2 Added t/spec/README
lwall authored
44 skip "reason" # skip test entirely
45 eval "reason" # eval the test because it doesn't parse yet
46 try "reason" # try the test because it throws exception
47 todo "reason" # mark "todo" because "not ok" is expected
cf81ec2 @coke kebab case skip-rest
coke authored
48 emit code # insert code (such as "skip-rest();") inline
001aed2 Added t/spec/README
lwall authored
50 All fudged tests return an exit code of 1 by default, so the test harness
51 will mark it as "dubious" even if all the tests supposedly pass.
3f4f55d @TimToady document new fudge directives, negated and envvar
TimToady authored
53 You may also negate the test:
55 #!humpty: [NUM] VERB ARGS
57 This will apply the verb on any system that *isn't* humpty.
59 Sometimes environment variables distinguish syntactic or semantic
60 variants, so you may apply a verb depending on the presence or absence
61 of such a setting:
66 The environement variable must be uppercase.
fa83ba8 [fudge] Didn't do #?DOES on subs right
lwall authored
68 There is also the following directive which modifies the test count of
69 the next construct:
71 #?DOES count
4fbed9f [fudge] clarify that #?DOES can take an expression (within limits)
lwall authored
73 The count may be an expression as long as any variables referenced in
74 the expression are in scope at the location fudge eventually inserts a
75 "skip()" call.
fa83ba8 [fudge] Didn't do #?DOES on subs right
lwall authored
77 When applied to a subsequent sub definition, registers the sub name as
4fbed9f [fudge] clarify that #?DOES can take an expression (within limits)
lwall authored
78 doing that many tests when called. Note, however, that any skipping
79 is done at the point of the call, not within the subroutine, so the count
80 may not refer to any parameter of the sub.
fa83ba8 [fudge] Didn't do #?DOES on subs right
lwall authored
001aed2 Added t/spec/README
lwall authored
82 When you run the fudge preprocessor, if it decides the test needs
83 fudging, it returns the new fudged filename; otherwise it returns
84 the original filename. (Generally you don't run "fudge" directly,
85 but your test harness runs the "fudgeall" program for you; see below.)
86 If there is already a fudged program in the directory that is newer
87 than the unfudged version, fudge just returns the fudged version
88 without regenerating it. If the fudged version is older, it removes
89 it and then decides anew whether to regenerate it based on the internal
90 fudge comments.
92 The "fudgeall" program may be called to process all the needed fudging
93 for a particular platform:
95 fudgeall humpty */*.t */*/*.t
97 will use the "fudge" program to translate any fudged files to a new
98 file where the extension is not *.t but instead is *.humpty to indicate
99 the platform dependency. It also returns the fudged list of filenames
100 to run, where unfudged tests are just passed through unchanged as *.t.
101 Each test comes through as either fudged or not, but never both.
102 The test harness then runs the selected test files as it normally
103 would (it shouldn't care whether they are named *.t or *.humpty).
cbe775a [t/spec] README: section about CWD in test files
moritz authored
106 In cases where the current working directory makes a difference, the tests
107 assume that the working directory is the root of the test suite, so that the
108 relative path to itself is t/spec/S\d\d-$section/$filename.
Something went wrong with that request. Please try again.