Perl 6 test suite
Perl6 Other
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Failed to load latest commit information.
S01-perl-5-integration Un-TODO passing test Sep 19, 2016
S02-lexical-conventions [TODO FUDGE] Unspace with sigiless terms Jul 12, 2016
S02-lists [coverage] Cover all branches of .tree Sep 18, 2016
S02-literals Replace deprecated is_approx with is-approx Oct 19, 2016
S02-magicals Unfudge tests for JVM Oct 6, 2016
S02-names-vars Unfudge some tests for rakudo-j Jul 16, 2016
S02-names Add use v6 where appropriate Feb 24, 2016
S02-one-pass-parsing Fix incorrect exit code in test Aug 19, 2016
S02-packages prefer RT #ticket Nov 1, 2015
S02-types Test (-Inf).Int throwage Oct 27, 2016
S03-binding Unfudge now passing test Aug 19, 2016
S03-feeds Remove dubious test Sep 19, 2015
S03-junctions [coverage] Cover Oct 11, 2016
S03-metaops Test artity-0 Z returns a Seq Oct 26, 2016
S03-operators Make chained regex test less specific Oct 27, 2016
S03-sequence Change range indexing semantic tweaks. Dec 1, 2015
S03-smartmatch Unfudge tests for JVM Oct 6, 2016
S04-blocks-and-statements Fudge newly added tests for JVM Sep 27, 2016
S04-declarations Replace deprecated is_approx with is-approx Oct 19, 2016
S04-exception-handlers CATCH in a loop lives Sep 2, 2016
S04-exceptions Add more Exceptions::JSON tests Oct 10, 2016
S04-phasers Unfudge now passing test Aug 3, 2016
S04-statement-modifiers prefer RT #ticket Nov 1, 2015
S04-statement-parsing s/lives_ok/lives-ok/g May 20, 2015
S04-statements Unfudge tests for JVM Oct 6, 2016
S05-capture Test to cover RT #129249. Sep 14, 2016
S05-grammar lookbehind that should not match does not match Sep 10, 2016
S05-interpolation Make sure that we only have one "use lib" Mar 11, 2016
S05-mass Unfudge tests that require Java 8 (unicode 6.1) Oct 8, 2016
S05-match Move test to better place Oct 11, 2016
S05-metachars Test for RT #122891. Nov 10, 2015
S05-metasyntax Skip test for ignoremark on JVM (ordbaseat NYI) Jul 22, 2016
S05-modifier Unfudge a lot of tests for rakudo-j Oct 25, 2016
S05-nonstrings Remove any Pugs fudges Jul 11, 2014
S05-substitution Unfudge a lot of tests for rakudo-j Oct 25, 2016
S05-syntactic-categories Expect Exception, not X::AdHoc. Nov 12, 2015
S05-transliteration Adverbs on Cool.trans work the same as on Str.trans Sep 12, 2016
S06-advanced Also cover use of callframe in a hot loop. Oct 19, 2016
S06-currying Make sure that we only have one "use lib" Mar 11, 2016
S06-macros Fix test where eval-lives-ok didn't get right context Dec 10, 2015
S06-multi Make sure where-blocked multi is prior to non-where-blocked multi (#155) Sep 6, 2016
S06-operator-overloading Replace deprecated is_approx with is-approx Oct 19, 2016
S06-other Test multiple *s/expressions in a double closure Oct 2, 2016
S06-parameters Expect Exception, not X::AdHoc. Nov 12, 2015
S06-routine-modifiers Test for RT #127540. Jul 21, 2016
S06-signature Make test less specific to accommodate more specificity Oct 7, 2016
S06-traits Fix broken plan Jul 6, 2016
S07-hyperrace .hyper preserves order Sep 10, 2016
S07-iterators fudge for moar Sep 12, 2016
S07-slip slip tests Oct 1, 2015
S09-autovivification Fix creation of nested arrays in S09-autovivification/autoincrement.t Aug 28, 2015
S09-hashes Add test for RT #111498 Apr 11, 2016
S09-multidim shaped array declaration without numbers does not infini-loop Sep 2, 2016
S09-subscript Add reference to RT #128341 Jun 7, 2016
S09-typed-arrays Replace deprecated is_approx with is-approx Oct 19, 2016
S10-packages Need a little rest to pass on OSX Sep 17, 2016
S11-compunit Use $*EXECUTABLE instead of 'perl6', fudge for JVM Dec 17, 2015
S11-modules Another test for nested packages Oct 26, 2016
S11-repository Fix "no writable path" found in curli-install.t Dec 28, 2015
S12-attributes Fudge tests for RT #128031 on JVM Apr 30, 2016
S12-class Test augment of a nested class. Oct 26, 2016
S12-coercion Unfudge two passing test on JVM Jul 15, 2016
S12-construction Fudge some tests in preparation to add files to Aug 19, 2016
S12-enums [coverage] Cover all routines in Oct 3, 2016
S12-introspection Revert "fudge RT #128579 for the JVM" Sep 28, 2016
S12-meta Make sure that we only have one "use lib" Mar 11, 2016
S12-methods Replace deprecated is_approx with is-approx Oct 19, 2016
S12-subset Add test description Oct 20, 2016
S12-traits Replace remaining eval-dies-ok with throws-like Oct 1, 2015
S13-overloading Expect Exception, not X::AdHoc. Nov 12, 2015
S13-syntax s/lives_ok/lives-ok/g May 20, 2015
S13-type-casting Use isa-ok instead of isa_ok Apr 26, 2015
S14-roles Unfudge tests on JVM Oct 6, 2016
S14-traits Revert "Fudge RT #129375 for rakudo.jvm" Sep 28, 2016
S15-literals Conversion of 'No' chars to numeric is not supported Jul 7, 2016
S15-nfg update charrange tests for RT #128550 Aug 13, 2016
S15-normalization Add use v6 where appropriate Feb 24, 2016
S15-string-types Add use v6 where appropriate Feb 24, 2016
S15-unicode-information Test for Unicode 9 character support. Sep 28, 2016
S16-filehandles Add test for broken symlink (#160) Sep 26, 2016
S16-io Picking first file from /proc/1 for eof testing Sep 23, 2016
S16-unfiled fixed appended RT number to be within quoted string May 5, 2015
S17-channel [coverage] Cover Channel.elems Sep 26, 2016
S17-lowlevel Correct test that did unprotected array writes. Aug 10, 2016
S17-procasync Also test setting Proc::Async encoding by handle. Oct 12, 2016
S17-promise Test golfed from RT #128833. Oct 19, 2016
S17-scheduler Adapt tests because of 2a635cd96279ace1c5d3d6d687 Dec 6, 2015
S17-supply Skip test for RT #128694 for JVM (hangs) Oct 14, 2016
S19-command-line-options Fudge some tests in preparation to add files to Aug 19, 2016
S19-command-line loop controls do not exit the REPL Sep 11, 2016
S22-package-format S22-package-format/local.t: use $*SPEC.catdir() to account for backsl… Dec 21, 2015
S24-testing Replace deprecated is_approx with is-approx Oct 19, 2016
S26-documentation Add missing plan Oct 14, 2016
S28-named-variables Prepare for removal of @*INC Nov 29, 2015
S29-any [coverage] Add tests for Any.[nl-out|print-nl] Sep 20, 2016
S29-context Remove pointless bitshifting in Test::Util is_run() Jul 20, 2016
S29-conversions Z needs flat now Aug 29, 2015
S29-os Tests for Proc.command and Proc.signal (#169) Oct 15, 2016
S32-array Resolves #164: tests callable args on splice (#176) Oct 20, 2016
S32-basics Add tests for pairup. (#174) Oct 20, 2016
S32-container [coverage] cover all* nudity in Sep 26, 2016
S32-exceptions Test to cover RT #129921. Oct 26, 2016
S32-hash [coverage] Cover Map.clone Oct 13, 2016
S32-io Test :enc argument in IO::Socket::Async. Oct 12, 2016
S32-list Add test for reduce with one item (#178) Oct 28, 2016
S32-num Replace deprecated is_approx with is-approx Oct 19, 2016
S32-scalar Unfudge tests on JVM Oct 6, 2016
S32-str Some more passing tests on JVM Oct 25, 2016
S32-temporal Replace deprecated is_approx with is-approx Oct 19, 2016
S32-trig Replace deprecated is_approx with is-approx Oct 19, 2016
integration Replace deprecated is_approx with is-approx Oct 19, 2016
packages Another test for nested packages Oct 26, 2016
rosettacode Fix Windows line ending workarounds. Nov 3, 2015
t Add more tests for fudging Apr 20, 2015
.gitignore ignore .precomp folders Dec 11, 2015
LICENSE Add Artistic License 2 Nov 5, 2013 correct typo Oct 24, 2016
TODO Add a note about RTs. Jul 20, 2014
fudge Replace deprecated is_approx with is-approx Oct 19, 2016
fudgeall Added quotes in case fudge path has spaces. (Tested on Linux and Win3… Jun 18, 2008
test_summary avoid unnecessary differences in output Mar 4, 2015

The Official Perl 6 Test Suite

The purpose of the test suite is to validate implementations that wish to be known as a conforming Perl 6 implementation.



Please consider this test suite to be the bleeding edge of Perl 6 development. New tests, tests for experimental new features, etc., will live on this 'master' branch. Once a specification is cut, a branch will be created for that version of the spec, e.g., 6.c for Christmas.

As they develop, different implementations will certainly be in different states of readiness with respect to the test suite, so in order for the various implementations to track their progress independently, we've established a mechanism for fudging the tests in a kind of failsoft fashion. To pass a test officially, an implementation must be able to run a test file unmodified, but an implementation may (temporarily) skip tests or mark them as "todo" via the fudging mechanism, which is implemented via the fudge preprocessor. Individual implementations are not allowed to modify the actual test code, but may insert line comments before each actual test (or block of tests) that changes how those tests are to be treated for this platform. The fudge preprocessor pays attention only to the comments that belong to the current implementation and ignores all the rest. If your implementation is named "rakudo" then your special comment lines look like:

#?rakudo: [NUM] VERB ARGS

(The colon is optional.)

The optional NUM says how many statements or blocks to apply the verb to. (If not supplied, a value of 1 is assumed). A statement is arbitrarily defined as one or more lines starting with a test call and ending in a semicolon (with an optional comment).

VERBs include:

skip "reason"   # skip test entirely
eval "reason"   # eval the test because it doesn't parse yet
try "reason"    # try the test because it throws exception
todo "reason"   # mark "todo" because "not ok" is expected
emit code       # insert code (such as "skip-rest();") inline

All fudged tests return an exit code of 1 by default, so the test harness will mark it as "dubious" even if all the tests supposedly pass.

You may also negate the test:

#!rakudo [NUM] VERB ARGS

This will apply the verb on any system that isn't rakudo.

Sometimes environment variables distinguish syntactic or semantic variants, so you may apply a verb depending on the presence or absence of such a setting:


The environment variable must be uppercase.

There is also the following directive which modifies the test count of the next construct:

#?DOES count

The count may be an expression as long as any variables referenced in the expression are in scope at the location fudge eventually inserts a "skip()" call.

When applied to a subsequent sub definition, fudge registers the sub name as doing that many tests when called. Note, however, that any skipping is done at the point of the call, not within the subroutine, so the count may not refer to any parameter of the sub.

When you run the fudge preprocessor, if it decides the test needs fudging, it returns the new fudged filename; otherwise it returns the original filename. (Generally you don't run fudge directly, but your test harness runs the fudgeall program for you; see below.) If there is already a fudged program in the directory that is newer than the unfudged version, fudge just returns the fudged version without regenerating it. If the fudged version is older, it removes it and then decides anew whether to regenerate it based on the internal fudge comments.

The fudgeall program may be called to process all the needed fudging for a particular implementation:

$ fudgeall rakudo */*.t */*/*.t

Program fudgeall will use the fudge program to translate any fudged files to a new file where the extension is not *.t but instead is *.rakudo to indicate the implementation dependency. It also returns the fudged list of filenames to run, where unfudged tests are just passed through unchanged as *.t. Each test comes through as either fudged or not, but never both. The test harness then runs the selected test files as it normally would (it shouldn't care whether they are named *.t or *.rakudo).

In cases where the current working directory makes a difference, the tests assume that the working directory is the root of the test suite, so that the relative path to itself is t/spec/S\d\d-$section/$filename.

Environment Variables


Some tests rely on a process to complete in a certain amount of time. If you're running on a slowish computer, try setting ROAST_TIMING_SCALE to a larger value that will be used as a multiplier for the time to wait. We don't wait for too long a time by default so as to make the roast run faster. Defaults to 1.


A person who wants to contribute a test to the project should read this Github guide to issues and pull requests (PRs) which describes in great detail the work flow for forks and submitting PRs to a project.

Follow the same general steps for project

  • fork project roast
  • clone your fork of roast to a local directory
  • set the origin and upstream remotes
  • checkout a branch to work on your issue or proposal
  • when through, ensure all desired commits are finished
  • push the issue branch to your origin (your fork of roast on github)
  • go to your github account for project roast and submit the PR

Working the issue

Normal tests

New tests for existing features are usually accomplished by adding the test(s) to an existing test file. Then that file's plan count is updated. The new test(s) are tested locally by executing

$ perl6 <test file(s)>

When all is well, the commits are finalized, the branch is pushed to the user's fork on Github, and there the PR is initiated.

If a new test file has been created, one additional step has to be taken: the new test file has to be added to and a PR for project can be submitted for that. However, it is easier just to ask for help adding the new test file on IRC channel #perl6.

Fudged tests

Let's say you want to propose a new feature, method printf for IO::Handle, for rakudo and, being a believer in test-driven development, are submittng some test for something that can't yet be tested. Thus we will need to create the test but we will fudge it so it will be ignored.

We create a new test file named appropriately, say, S16-io/printf.t , the contents of which are:

use v6;
use Test;
plan 1;

my $f = './.printf-tmpfil';
my $fh = open $f, :w;
my $res = slurp $f;
is $res, "Hi\n", "printf() works with zero args";
unlink $f;

We know the test doesn't work yet:

$ perl6 S16-io/printf.t
No such method 'printf' for invocant of type 'IO::Handle'
  in block <unit> at ./S16-io/printf.t line 9

# Looks like you planned 1 test, but ran 0

so we add the fudge to it to get the new contents:

use v6;
use Test;
plan 1;

#?rakudo skip 'RT #999999 not yet implemented'
    my $f = './.printf-tmpfil';
    my $fh = open $f, :w;
    my $res = slurp $f;
    is $res, "Hi\n", "printf() works with zero args";
    unlink $f;

Notice we put the test in a block. All tests in that block are affected by the fudge line preceding it.

We want to test the fudged file before we submit the PR so we have to manually create the fudged test file:

$ fudge rakudo S16-io/printf.t

which, as we see by the return from the command, produces file S16-io/printf.rakudo whose contents are

use v6;
use Test;
plan 1;

#?rakudo skip 'RT #999999 not yet implemented'
skip('RT #999999 not yet implemented', 1);# {
#     my $f = './.printf-tmpfil';
#     my $fh = open $f, :w;
#     $fh.printf("Hi\n");
#     $fh.close;
#     my $res = slurp $f;
#     is $res, "Hi\n", "printf() works with zero args";
#     unlink $f;
# }

say "# FUDGED!";

We can then test it:

$ perl6 S16-io/printf.rakudo
ok 1 - \# SKIP RT \#999999 not yet implemented

Success! Now we can commit the new test file, but NOT the generated fudge test file—that will be generated automatically by the test harness during the regular testing on the servers. As described earlier, the new test file will have to be added to the file, either via a PR or a request to someone on IRC to add it.