Perl 6 test suite
Perl6 Other
Latest commit 8627307 Aug 24, 2016 @zoffixznet zoffixznet Remove bogus tests
The tests assumed defaults on slurpies are a thing, but they were
giving MoarVM errors when used. In
rakudo/rakudo@560170f
those errors were caught earlier and now cause Rakudo Exceptions to be
thrown, which causes these tests to fail.
Failed to load latest commit information.
S01-perl-5-integration Skip Perl 5 integration tests if requirements not met Aug 20, 2016
S02-lexical-conventions [TODO FUDGE] Unspace with sigiless terms Jul 12, 2016
S02-lists Array out-of-range from-end slicing has "Effective Index" in error Jul 4, 2016
S02-literals Fix order of loading modules Jul 25, 2016
S02-magicals Unfudge some tests for rakudo-j Jul 16, 2016
S02-names-vars Unfudge some tests for rakudo-j Jul 16, 2016
S02-names Add use v6 where appropriate Feb 24, 2016
S02-one-pass-parsing Fix incorrect exit code in test Aug 19, 2016
S02-packages prefer RT #ticket Nov 1, 2015
S02-types Add tests for RT#129044. Aug 23, 2016
S03-binding Unfudge now passing test Aug 19, 2016
S03-feeds Remove dubious test Sep 19, 2015
S03-junctions Fudge tests returning Int instead of Bool for JVM Dec 8, 2015
S03-metaops Test for RT #128703. Aug 16, 2016
S03-operators Test RT#128421 (#128) Aug 24, 2016
S03-sequence Change range indexing semantic tweaks. Dec 1, 2015
S03-smartmatch Skip some tests on rakudo-j Jul 11, 2016
S04-blocks-and-statements Remove fudged tests for RT#128544 Aug 9, 2016
S04-declarations Fudge tests for RT #127958 for rakudo-j Jul 27, 2016
S04-exception-handlers Use 'todo' instead of 'skip' if possible Jul 12, 2016
S04-exceptions Unfudge a few skipped tests Jul 16, 2016
S04-phasers Unfudge now passing test Aug 3, 2016
S04-statement-modifiers prefer RT #ticket Nov 1, 2015
S04-statement-parsing s/lives_ok/lives-ok/g May 20, 2015
S04-statements Fudge some test for rakudo-j Aug 15, 2016
S05-capture Tests to cover RT #88340. Nov 12, 2015
S05-grammar [TODO FUDGE] <return> in grammar must refer to token not &return Jul 20, 2016
S05-interpolation Make sure that we only have one "use lib" Mar 11, 2016
S05-mass Add use v6 where appropriate Feb 24, 2016
S05-match [TODO FUDGE] "$/[0]--" interpolates correctly Jul 21, 2016
S05-metachars Test for RT #122891. Nov 10, 2015
S05-metasyntax Skip test for ignoremark on JVM (ordbaseat NYI) Jul 22, 2016
S05-modifier Fudge some new tests for rakudo-j Aug 20, 2016
S05-nonstrings Remove any Pugs fudges Jul 11, 2014
S05-substitution Avoid spurious warnings in output Aug 17, 2016
S05-syntactic-categories Expect Exception, not X::AdHoc. Nov 12, 2015
S05-transliteration Add test for stringification of StrDistance Jun 21, 2016
S06-advanced Fudge failing tests for rakudo-j Aug 5, 2016
S06-currying Make sure that we only have one "use lib" Mar 11, 2016
S06-macros Fix test where eval-lives-ok didn't get right context Dec 10, 2015
S06-multi Unfudge some tests for rakudo-j Jul 16, 2016
S06-operator-overloading Make sure that we only have one "use lib" Mar 11, 2016
S06-other Tests for RT #127785. Jul 21, 2016
S06-parameters Expect Exception, not X::AdHoc. Nov 12, 2015
S06-routine-modifiers Test for RT #127540. Jul 21, 2016
S06-signature Remove bogus tests Aug 25, 2016
S06-traits Fix broken plan Jul 6, 2016
S07-hyperrace Remove timing related stuff from hyper/race tests Apr 6, 2016
S07-iterators Start to rehabilitate Range Iterator test file post-GLR Mar 17, 2016
S07-slip slip tests Oct 1, 2015
S09-autovivification Fix creation of nested arrays in S09-autovivification/autoincrement.t Aug 28, 2015
S09-hashes Add test for RT #111498 Apr 11, 2016
S09-multidim There is no .plan for the foreseeable future Jul 27, 2016
S09-subscript Add reference to RT #128341 Jun 7, 2016
S09-typed-arrays Add some more sanity tests Jul 21, 2016
S10-packages Fudge two precomp tests for rakudo-j Jul 16, 2016
S11-compunit Use $*EXECUTABLE instead of 'perl6', fudge for JVM Dec 17, 2015
S11-modules Modernize and expand $?MODULE tests. Jul 21, 2016
S11-repository Fix "no writable path" found in curli-install.t Dec 28, 2015
S12-attributes Fudge tests for RT #128031 on JVM Apr 30, 2016
S12-class Adjust tests because of fe2be65806d907efbbaa Apr 21, 2016
S12-coercion Unfudge two passing test on JVM Jul 15, 2016
S12-construction Fudge some tests in preparation to add files to spectest.data Aug 19, 2016
S12-enums Check .enums returns a Map (RT#128138) Jul 4, 2016
S12-introspection ID metamethods on a package are absent by design Jul 8, 2016
S12-meta Make sure that we only have one "use lib" Mar 11, 2016
S12-methods Make sure that we only have one "use lib" Mar 11, 2016
S12-subset :D on subset corretly works in multi dispatch Aug 2, 2016
S12-traits Replace remaining eval-dies-ok with throws-like Oct 1, 2015
S13-overloading Expect Exception, not X::AdHoc. Nov 12, 2015
S13-syntax s/lives_ok/lives-ok/g May 20, 2015
S13-type-casting Use isa-ok instead of isa_ok Apr 26, 2015
S14-roles Fudge test for RT #127916 on JVM Apr 30, 2016
S14-traits Test for RT #112666. Dec 15, 2015
S15-literals Conversion of 'No' chars to numeric is not supported Jul 7, 2016
S15-nfg update charrange tests for RT #128550 Aug 13, 2016
S15-normalization Add use v6 where appropriate Feb 24, 2016
S15-string-types Add use v6 where appropriate Feb 24, 2016
S15-unicode-information Use existing RT number Nov 22, 2015
S16-filehandles Use constistent format Jul 6, 2016
S16-io Unfudge passing test for rakudo-j Aug 20, 2016
S16-unfiled fixed appended RT number to be within quoted string May 5, 2015
S17-channel Remove earliest.t; this feature is going away. Dec 5, 2015
S17-lowlevel Correct test that did unprotected array writes. Aug 10, 2016
S17-procasync Harden test against newline translation. Dec 16, 2015
S17-promise Test start blocks get fresh $/ and $!. Aug 3, 2016
S17-scheduler Adapt tests because of 2a635cd96279ace1c5d3d6d687 Dec 6, 2015
S17-supply Fix race in S17-supply/wait.t Aug 18, 2016
S19-command-line-options Fudge some tests in preparation to add files to spectest.data Aug 19, 2016
S19-command-line Update factual information in the comment Aug 18, 2016
S22-package-format S22-package-format/local.t: use $*SPEC.catdir() to account for backsl… Dec 21, 2015
S24-testing Test PERL6_TEST_DIE_ON_FAIL env var Aug 17, 2016
S26-documentation improve comments on last two tests; add row id on the last table; exp… ( Jun 29, 2016
S28-named-variables Prepare for removal of @*INC Nov 29, 2015
S29-any Replace more eval-dies-ok with throws-like (S15/S29) Sep 28, 2015
S29-context Remove pointless bitshifting in Test::Util is_run() Jul 20, 2016
S29-conversions Z needs flat now Aug 29, 2015
S29-os Add fudging for rakudo-j for new tests Jul 15, 2016
S32-array Remove SINK tests, they're an implementation detail Aug 8, 2016
S32-basics Make sure that we only have one "use lib" Mar 11, 2016
S32-container Tests for RT #126522. Dec 15, 2015
S32-exceptions Fix test now that we also have an Exceptions class Aug 23, 2016
S32-hash add test for :exists on multi-dim assoc subscript literals Aug 13, 2016
S32-io Fudge some new tests for rakudo-j Aug 20, 2016
S32-list Unfudge two tests for RT #128720 for rakudo-j Aug 10, 2016
S32-num .polymod with zero divisor does not reference guts in error Jul 17, 2016
S32-scalar Make sure that we only have one "use lib" Mar 11, 2016
S32-str Unfudge passing test for rakudo-j Aug 20, 2016
S32-temporal Add some DateTime subclassing tests Feb 20, 2016
S32-trig Add use v6 where appropriate Feb 24, 2016
integration Remove obsolete test for RT#127425 Aug 19, 2016
packages Teach is_run_repl to take a Callable to test output streams Aug 18, 2016
rosettacode Fix Windows line ending workarounds. Nov 3, 2015
t Add more tests for fudging Apr 20, 2015
.gitignore ignore .precomp folders Dec 11, 2015
LICENSE Add Artistic License 2 Nov 5, 2013
README Someday is here Dec 24, 2015
TODO Add a note about RTs. Jul 20, 2014
fudge Use kebob case for doesn't-hang() in fudgeables Jul 14, 2016
fudgeall Added quotes in case fudge path has spaces. (Tested on Linux and Win3… Jun 18, 2008
test_summary avoid unnecessary differences in output Mar 4, 2015

README

	The Official Perl 6 Test Suite

Its purpose is to validate implementations that wish to be known
as a conforming Perl 6 implementation. 

Please consider this test suite to be the bleeding edge of Perl 6
development. New tests, tests for experimental new features, etc.
will live on this branch. Once a specification is cut, a branch
will be created for that version of the spec, e.g. 6.c for Christmas.

As they develop, different implementations will certainly be in
different states of readiness with respect to the test suite, so
in order for the various implementations to track their progress
independently, we've established a mechanism for "fudging" the
tests in a kind of failsoft fashion.  To pass a test officially,
an implementation must be able to run a test file unmodified, but an
implementation may (temporarily) skip tests or mark them as "todo" via
the fudging mechanism, which is implemented via the fudge preprocessor.
Individual implementations are not allowed to modify the actual test
code, but may insert line comments before each actual test (or block
of tests) that changes how those tests are to be treated for this
platform.  The fudge preprocessor pays attention only to the comments
that belong to the current platform and ignores all the rest.  If your
platform is named "humpty" then your special comment lines look like:

    #?humpty: [NUM] VERB ARGS

(The colon is optional.)

The optional NUM says how many statements or blocks to apply the
verb to.  (If not supplied, a value of 1 is assumed).  A statement
is arbitrarily defined as one or more lines starting with a test call
and ending in semicolon (with an optional comment).

VERBs include:

    skip "reason"	# skip test entirely
    eval "reason"	# eval the test because it doesn't parse yet
    try "reason"	# try the test because it throws exception
    todo "reason"	# mark "todo" because "not ok" is expected
    emit code		# insert code (such as "skip-rest();") inline

All fudged tests return an exit code of 1 by default, so the test harness
will mark it as "dubious" even if all the tests supposedly pass.

You may also negate the test:

    #!humpty: [NUM] VERB ARGS

This will apply the verb on any system that *isn't* humpty.

Sometimes environment variables distinguish syntactic or semantic
variants, so you may apply a verb depending on the presence or absence
of such a setting:

    #?MYSPECIALVAR: [NUM] VERB ARGS
    #!MYSPECIALVAR: [NUM] VERB ARGS

The environement variable must be uppercase.

There is also the following directive which modifies the test count of
the next construct:

    #?DOES count

The count may be an expression as long as any variables referenced in
the expression are in scope at the location fudge eventually inserts a
"skip()" call.

When applied to a subsequent sub definition, registers the sub name as
doing that many tests when called.  Note, however, that any skipping
is done at the point of the call, not within the subroutine, so the count
may not refer to any parameter of the sub.

When you run the fudge preprocessor, if it decides the test needs
fudging, it returns the new fudged filename; otherwise it returns
the original filename.  (Generally you don't run "fudge" directly,
but your test harness runs the "fudgeall" program for you; see below.)
If there is already a fudged program in the directory that is newer
than the unfudged version, fudge just returns the fudged version
without regenerating it.  If the fudged version is older, it removes
it and then decides anew whether to regenerate it based on the internal
fudge comments.

The "fudgeall" program may be called to process all the needed fudging
for a particular platform:

    fudgeall humpty */*.t */*/*.t

will use the "fudge" program to translate any fudged files to a new
file where the extension is not *.t but instead is *.humpty to indicate
the platform dependency.  It also returns the fudged list of filenames
to run, where unfudged tests are just passed through unchanged as *.t.
Each test comes through as either fudged or not, but never both.
The test harness then runs the selected test files as it normally
would (it shouldn't care whether they are named *.t or *.humpty).


In cases where the current working directory makes a difference, the tests
assume that the working directory is the root of the test suite, so that the
relative path to itself is t/spec/S\d\d-$section/$filename.