Perl 6 test suite
Perl 6 Other
Pull request Compare This branch is 1216 commits behind perl6:master.
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
S01-perl-5-integration
S02-lexical-conventions
S02-lists
S02-literals
S02-magicals
S02-names-vars
S02-names
S02-one-pass-parsing
S02-packages
S02-types
S03-binding
S03-feeds
S03-junctions
S03-metaops
S03-operators
S03-sequence
S03-smartmatch
S04-blocks-and-statements
S04-declarations
S04-exception-handlers
S04-exceptions
S04-phasers
S04-statement-modifiers
S04-statement-parsing
S04-statements
S05-capture
S05-grammar
S05-interpolation
S05-mass
S05-match
S05-metachars
S05-metasyntax
S05-modifier
S05-nonstrings
S05-substitution
S05-syntactic-categories
S05-transliteration
S06-advanced
S06-currying
S06-macros
S06-multi
S06-operator-overloading
S06-other
S06-parameters
S06-routine-modifiers
S06-signature
S06-traits
S07-hyperrace
S07-iterators
S07-slip
S09-autovivification
S09-hashes
S09-multidim
S09-subscript
S09-typed-arrays
S10-packages
S11-compunit
S11-modules
S11-repository
S12-attributes
S12-class
S12-coercion
S12-construction
S12-enums
S12-introspection
S12-meta
S12-methods
S12-subset
S12-traits
S13-overloading
S13-syntax
S13-type-casting
S14-roles
S14-traits
S15-literals
S15-nfg
S15-normalization
S15-string-types
S15-unicode-information
S16-filehandles
S16-io
S16-unfiled
S17-channel
S17-lowlevel
S17-procasync
S17-promise
S17-scheduler
S17-supply
S19-command-line-options
S19-command-line
S22-package-format
S24-testing
S26-documentation
S28-named-variables
S29-any
S29-context
S29-conversions
S29-os
S32-array
S32-basics
S32-container
S32-exceptions
S32-hash
S32-io
S32-list
S32-num
S32-scalar
S32-str
S32-temporal
S32-trig
integration
packages
rosettacode
t
.gitignore
CONTRIBUTING.md
LICENSE
README.md
TODO
fudge
fudgeall
fudgeandrun
super-fudger.p6
test_summary

README.md

The Official Perl 6 Test Suite

The purpose of the test suite is to validate implementations that wish to be known as a conforming Perl 6 implementation.

Contents

Introduction

Please consider this test suite to be the bleeding edge of Perl 6 development. New tests, tests for experimental new features, etc., will live on this 'master' branch. Once a specification is cut, a branch will be created for that version of the spec, e.g., 6.c for Christmas.

As they develop, different implementations will certainly be in different states of readiness with respect to the test suite, so in order for the various implementations to track their progress independently, we've established a mechanism for fudging the tests in a kind of failsoft fashion. To pass a test officially, an implementation must be able to run a test file unmodified, but an implementation may (temporarily) skip tests or mark them as "todo" via the fudging mechanism, which is implemented via the fudge preprocessor. Individual implementations are not allowed to modify the actual test code, but may insert line comments before each actual test (or block of tests) that changes how those tests are to be treated for this platform. The fudge preprocessor pays attention only to the comments that belong to the current implementation and ignores all the rest. If your implementation is named "rakudo" then your special comment lines look like:

#?rakudo: [NUM] VERB ARGS

(The colon is optional.)

The optional NUM says how many statements or blocks to apply the verb to. (If not supplied, a value of 1 is assumed). A statement is arbitrarily defined as one or more lines starting with a test call and ending in a semicolon (with an optional comment).

VERBs include:

skip "reason"	# skip test entirely
eval "reason"	# eval the test because it doesn't parse yet
try "reason"	# try the test because it throws exception
todo "reason"	# mark "todo" because "not ok" is expected
emit code		# insert code (such as "skip-rest();") inline

All fudged tests return an exit code of 1 by default, so the test harness will mark it as "dubious" even if all the tests supposedly pass.

You may also negate the test:

#!rakudo [NUM] VERB ARGS

This will apply the verb on any system that isn't rakudo.

Sometimes environment variables distinguish syntactic or semantic variants, so you may apply a verb depending on the presence or absence of such a setting:

#?MYSPECIALVAR [NUM] VERB ARGS
#!MYSPECIALVAR [NUM] VERB ARGS

The environment variable must be uppercase.

There is also the following directive which modifies the test count of the next construct:

#?DOES count

The count may be an expression as long as any variables referenced in the expression are in scope at the location fudge eventually inserts a "skip()" call.

When applied to a subsequent sub definition, fudge registers the sub name as doing that many tests when called. Note, however, that any skipping is done at the point of the call, not within the subroutine, so the count may not refer to any parameter of the sub.

When you run the fudge preprocessor, if it decides the test needs fudging, it returns the new fudged filename; otherwise it returns the original filename. (Generally you don't run fudge directly, but your test harness runs the fudgeall program for you; see below.) If there is already a fudged program in the directory that is newer than the unfudged version, fudge just returns the fudged version without regenerating it. If the fudged version is older, it removes it and then decides anew whether to regenerate it based on the internal fudge comments.

The fudgeall program may be called to process all the needed fudging for a particular implementation:

$ fudgeall rakudo */*.t */*/*.t

Program fudgeall will use the fudge program to translate any fudged files to a new file where the extension is not *.t but instead is *.rakudo to indicate the implementation dependency. It also returns the fudged list of filenames to run, where unfudged tests are just passed through unchanged as *.t. Each test comes through as either fudged or not, but never both. The test harness then runs the selected test files as it normally would (it shouldn't care whether they are named *.t or *.rakudo).

In cases where the current working directory makes a difference, the tests assume that the working directory is the root of the test suite, so that the relative path to itself is t/spec/S\d\d-$section/$filename.

To fudge and run prove on a specific file:

./fudgeandrun S05-metasyntax/angle-brackets.t

fudgeandrun does not assume any particular implementation but guesses by running perl6 to look at special variables like $*PERL. See fudgeandrun usage to specify a different implementation and other options.

Environmental Variables

  • ROAST_TIMING_SCALE

Some tests rely on a process to complete in a certain amount of time. If you're running on a slowish computer, try setting ROAST_TIMING_SCALE to a larger value that will be used as a multiplier for the time to wait. We don't wait for too long a time by default so as to make the roast run faster. Defaults to 1.

Contributing

See CONTRIBUTING.md