Skip to content
πŸ¦‹ Raku test suite
Raku Other
  1. Raku 99.6%
  2. Other 0.4%
Branch: master
Clone or download

Latest commit

Files

Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
3rdparty Update Emoji and Grapheme Break Tests to version Unicode version 11.0 Oct 1, 2018
APPENDICES Typecheck on default value for attribute changed Jan 28, 2020
MISC [JVM] Fudge some recently added/changed tests May 21, 2020
S01-perl-5-integration Some Perl 6 -> Raku renaming May 7, 2020
S02-lexical-conventions Batch #13 of roast RT -> GH ticket migration Jan 12, 2020
S02-lists Batch #13 of roast RT -> GH ticket migration Jan 12, 2020
S02-literals
S02-magicals Call method .raku instead of .perl in tests Jan 3, 2020
S02-names-vars Call method .raku instead of .perl in tests Jan 3, 2020
S02-names [JVM] Tweak fudging May 24, 2020
S02-one-pass-parsing Batch #16 of roast RT -> GH ticket migration Jan 15, 2020
S02-packages prefer RT #ticket Nov 1, 2015
S02-types
S03-binding Correct Perl name where missed in the previsous commits Oct 16, 2019
S03-buf Add tests for using Buf type objects and write-num* Oct 4, 2019
S03-feeds Remove reference to placeholder tickets from RT Jul 9, 2019
S03-junctions Test to cover value in single junction iteration Feb 5, 2020
S03-metaops
S03-operators Demonstrate rakudo/issues/3346 with HyperSeq Feb 1, 2020
S03-sequence Adaptations for change behaviour of .rotate May 7, 2020
S03-smartmatch Call method .raku instead of .perl in tests Jan 3, 2020
S04-blocks-and-statements Call method .raku instead of .perl in tests Jan 3, 2020
S04-declarations Cover redeclaration of a parameter Feb 12, 2020
S04-exception-handlers Batch #3 of roast RT -> GH ticket migration Jan 11, 2020
S04-exceptions [JVM] Fudge some recently added/changed tests May 21, 2020
S04-phasers Add test for GH #3411 Jan 11, 2020
S04-statement-modifiers Test `M::foo if True` calls M::foo Feb 5, 2020
S04-statement-parsing s/lives_ok/lives-ok/g May 20, 2015
S04-statements [JVM] Update reference for skipped test Apr 18, 2020
S05-capture Change references to Perl 5 to just Perl Nov 26, 2019
S05-grammar Call method .raku instead of .perl in tests Jan 3, 2020
S05-interpolation Change references to Perl 5 to just Perl Nov 26, 2019
S05-mass Add test for GH#2059 Jan 29, 2020
S05-match Use MONKEY-TYPING to correct scope Mar 18, 2020
S05-metachars Batch #21 of roast RT -> GH ticket migration Jan 18, 2020
S05-metasyntax Make correct Bool matchers Feb 11, 2020
S05-modifier Test m:g/... { code block }/ gets the $/ correct Mar 25, 2020
S05-nonstrings Remove any Pugs fudges Jul 11, 2014
S05-substitution Test placeholder params used in S/// construct Feb 11, 2020
S05-syntactic-categories Expect Exception, not X::AdHoc. Nov 12, 2015
S05-transliteration Load "Test" before doing a "use lib" Mar 18, 2020
S06-advanced Mark dispatcher tests as todo May 2, 2020
S06-currying Call method .raku instead of .perl in tests Jan 3, 2020
S06-macros Remove reference to placeholder tickets from RT Jul 9, 2019
S06-multi Load "Test" before doing a "use lib" Mar 18, 2020
S06-operator-overloading Remove reference to placeholder tickets from RT Jul 9, 2019
S06-other Batch #6 of roast RT -> GH ticket migration Jan 12, 2020
S06-parameters [v6.d REVIEW] Compactify test code Dec 25, 2017
S06-routine-modifiers Second batch of roast RT -> GH ticket migration Jan 11, 2020
S06-signature Add test for Parameter.raku with OUTER defaults Mar 27, 2020
S06-traits Test user-defined chain assoc ops Feb 12, 2020
S07-hyperrace Remove accidentally committed debug code Oct 24, 2019
S07-iterators [v6.d REVIEW] Remove pull test in skip-at-least Jul 28, 2018
S07-slip Call method .raku instead of .perl in tests Jan 3, 2020
S09-autovivification Remove old comment Sep 11, 2017
S09-hashes Adding test to dynamic shaped array and dynamic object hash Nov 27, 2018
S09-multidim Update .gist tests for multidim arrays May 12, 2020
S09-subscript Batch #16 of roast RT -> GH ticket migration Jan 15, 2020
S09-typed-arrays
S10-packages Load "Test" before doing a "use lib" Mar 18, 2020
S11-compunit Batch #10 of roast RT -> GH ticket migration Jan 12, 2020
S11-modules Load "Test" before doing a "use lib" Mar 18, 2020
S11-repository Use cmd.exe compatible hash lookup Jan 21, 2019
S12-attributes Add test for GH#3450 Jan 30, 2020
S12-class Added tests for late applied `also is rw` Feb 19, 2020
S12-coercion Remove reference to placeholder tickets from RT Jul 9, 2019
S12-construction Oops, accidentally lost closing quotes in skip Jan 12, 2020
S12-enums Call method .raku instead of .perl in tests Jan 3, 2020
S12-introspection Batch #11 of roast RT -> GH ticket migration Jan 12, 2020
S12-meta Test runtime-created classes being attributes Oct 18, 2019
S12-methods Fix test messages May 6, 2020
S12-subset Second batch of roast RT -> GH ticket migration Jan 11, 2020
S12-traits Replace remaining eval-dies-ok with throws-like Oct 1, 2015
S13-overloading First batch of roast RT -> GH ticket migration Jan 11, 2020
S13-syntax Change Perl 6 with Raku where appropriate Oct 16, 2019
S13-type-casting
S14-roles Add test for mixins of roles that use "is built" at runtime Mar 21, 2020
S14-traits Revert "Fudge RT #129375 for rakudo.jvm" Sep 28, 2016
S15-literals [JVM] Unfudge some passing tests Apr 5, 2019
S15-nfg Call method .raku instead of .perl in tests Jan 3, 2020
S15-normalization Update Unicode tests to Unicode v12.1 Jul 28, 2019
S15-string-types Call method .raku instead of .perl in tests Jan 3, 2020
S15-unicode-information Batch #7 of roast RT -> GH ticket migration Jan 12, 2020
S16-filehandles Batch #11 of roast RT -> GH ticket migration Jan 12, 2020
S16-io [JVM] Unfudge now passing tests May 24, 2020
S16-unfiled Remove reference to placeholder tickets from RT Jul 9, 2019
S17-channel [JVM] Unfudge some more passing tests Mar 18, 2020
S17-lowlevel Change Perl 6 with Raku where appropriate Oct 16, 2019
S17-procasync Fix flappy timing based Proc::Async::kill tests Mar 28, 2020
S17-promise [JVM] Fudge some currently failing tests May 21, 2020
S17-scheduler Remove reference to placeholder tickets from RT Jul 9, 2019
S17-supply [JVM] Tweak fudging May 24, 2020
S19-command-line-options
S19-command-line
S22-package-format Revert "Fix checking of "from", because default is now "Raku"" Feb 22, 2020
S24-testing Adapt Seq.new-consumed tests Mar 25, 2020
S26-documentation
S28-named-variables Change references to Perl 5 to just Perl Nov 26, 2019
S29-any Change Perl 6 with Raku where appropriate Oct 16, 2019
S29-context Move "use nqp" to correct scope Mar 18, 2020
S29-conversions Untodo codepoint range check test Mar 26, 2020
S29-os
S32-array Adaptations for change behaviour of .rotate May 7, 2020
S32-basics Batch #24 of roast RT -> GH ticket migration Jan 25, 2020
S32-container Batch #10 of roast RT -> GH ticket migration Jan 12, 2020
S32-encoding Change Perl 6 with Raku where appropriate Oct 16, 2019
S32-exceptions [JVM] Tweak fudging May 24, 2020
S32-hash Add tests for R#3617 Apr 12, 2020
S32-io [JVM] Unfudge now passing tests May 24, 2020
S32-list Add tests for R#3654 May 2, 2020
S32-num Add test for negative numbers not being prime Feb 8, 2020
S32-scalar Call method .raku instead of .perl in tests Jan 3, 2020
S32-str Add tests for Str.lines(:chomp) May 24, 2020
S32-temporal Add tests for R#3683 May 12, 2020
S32-trig Call method .raku instead of .perl in tests Jan 3, 2020
docs Change Perl 6 with Raku where appropriate Oct 16, 2019
integration [JVM] Fudge some recently added/changed tests May 21, 2020
packages Revert "Test for export of long names" Jan 7, 2020
rosettacode Fix Windows line ending workarounds. Nov 3, 2015
t Load "Test" before doing a "use lib" Mar 18, 2020
tools Print URLs to commits we're launching in browser Oct 14, 2018
.gitignore ignore .precomp folders Dec 11, 2015
CONTRIBUTING.md Update location of spectest.data Jun 17, 2019
LICENSE Add Artistic License 2 Nov 5, 2013
README.md Change Perl 6 with Raku where appropriate Oct 16, 2019
TODO Change Perl 6 with Raku where appropriate Oct 16, 2019
VERSION Changed release guide to reflect the latest state of things. May 31, 2019
fudge Change Perl 6 with Raku where appropriate Oct 16, 2019
fudgeall Fix globbing on windows Nov 18, 2017
fudgeandrun [v6.d REVIEW] Warn if we don't find local binary Sep 16, 2018
spectest.data [JVM] Tweak fudging May 24, 2020
super-fudger.raku Update she-bang Feb 13, 2020
test_summary

README.md

The Official Raku Test Suite

The purpose of the test suite is to validate implementations that wish to be known as a conforming Raku.

Contents

Introduction

Please consider this test suite to be the bleeding edge of Raku development. New tests, tests for proposed new features, etc., will live on this 'master' branch. Once a specification is cut, a branch will be created for that version of the spec, e.g., 6.c for Christmas language release. If errata becomes available, it will be kept in -errata branch for that specific language version, e.g. 6.c-errata for 6.c.

As they develop, different implementations will certainly be in different states of readiness with respect to the test suite, so in order for the various implementations to track their progress independently, we've established a mechanism for fudging the tests in a kind of failsoft fashion. To pass a test officially, an implementation must be able to run a test file unmodified, but an implementation may (temporarily) skip tests or mark them as "todo" via the fudging mechanism, which is implemented via the fudge preprocessor. Individual implementations are not allowed to modify the actual test code, but may insert line comments before each actual test (or block of tests) that changes how those tests are to be treated for this platform. The fudge preprocessor pays attention only to the comments that belong to the current implementation and ignores all the rest. If your implementation is named "rakudo" then your special comment lines look like:

#?rakudo: [NUM] VERB ARGS

(The colon is optional.)

The optional NUM says how many statements or blocks to apply the verb to. (If not supplied, a value of 1 is assumed). A statement is arbitrarily defined as one or more lines starting with a test call and ending in a semicolon (with an optional comment).

VERBs include:

skip "reason"	# skip test entirely
eval "reason"	# eval the test because it doesn't parse yet
try "reason"	# try the test because it throws exception
todo "reason"	# mark "todo" because "not ok" is expected
emit code		# insert code (such as "skip-rest();") inline

All fudged tests return an exit code of 1 by default, so the test harness will mark it as "dubious" even if all the tests supposedly pass.

You may also negate the test:

#!rakudo [NUM] VERB ARGS

This will apply the verb on any system that isn't rakudo.

Sometimes environment variables distinguish syntactic or semantic variants, so you may apply a verb depending on the presence or absence of such a setting:

#?MYSPECIALVAR [NUM] VERB ARGS
#!MYSPECIALVAR [NUM] VERB ARGS

The environment variable must be uppercase.

There is also the following directive which modifies the test count of the next construct:

#?DOES count

The count may be an expression as long as any variables referenced in the expression are in scope at the location fudge eventually inserts a "skip()" call.

When applied to a subsequent sub definition, fudge registers the sub name as doing that many tests when called. Note, however, that any skipping is done at the point of the call, not within the subroutine, so the count may not refer to any parameter of the sub.

When you run the fudge preprocessor, if it decides the test needs fudging, it returns the new fudged filename; otherwise it returns the original filename. (Generally you don't run fudge directly, but your test harness runs the fudgeall program for you; see below.) If there is already a fudged program in the directory that is newer than the unfudged version, fudge just returns the fudged version without regenerating it. If the fudged version is older, it removes it and then decides anew whether to regenerate it based on the internal fudge comments.

The fudgeall program may be called to process all the needed fudging for a particular implementation:

$ fudgeall rakudo */*.t */*/*.t

Program fudgeall will use the fudge program to translate any fudged files to a new file where the extension is not *.t but instead is *.rakudo to indicate the implementation dependency. It also returns the fudged list of filenames to run, where unfudged tests are just passed through unchanged as *.t. Each test comes through as either fudged or not, but never both. The test harness then runs the selected test files as it normally would (it shouldn't care whether they are named *.t or *.rakudo).

In cases where the current working directory makes a difference, the tests assume that the working directory is the root of the test suite, so that the relative path to itself is t/spec/S\d\d-$section/$filename.

To fudge and run prove on a specific file:

./fudgeandrun S05-metasyntax/angle-brackets.t

fudgeandrun does not assume any particular implementation but guesses by running perl6 to look at special variables like $*PERL. See fudgeandrun usage to specify a different implementation and other options.

Test Utilities

This repository contains Test::Util module with helper routines you can use when writing tests. See POD documentation included at the end of the module's source code. To include the module, in your test file, you need to add use lib line to your test file.

use lib $?FILE.IO.parent(2).add: 'packages/Test-Helpers';
use Test::Util;
use Test;

Depending on the location of your test file, the number inside .parent(2) may need to be adjusted to go up the correct number of times from the test files's location to the root of the repository.

Appendices

The APPENDICES directory contains advisory tests implentations may optionally choose to follow. See the README included in that directory for more information.

APPENDICES are not included in released versions of the specification.

Environmental Variables

  • ROAST_TIMING_SCALE

Some tests rely on a process to complete in a certain amount of time. If you're running on a slowish computer, try setting ROAST_TIMING_SCALE to a larger value that will be used as a multiplier for the time to wait. We don't wait for too long a time by default so as to make the roast run faster. Defaults to 1.

Contributing

See CONTRIBUTING.md

You can’t perform that action at this time.