Skip to content

Make use of practice exercise scenarios #703

Open
@mk-mxp

Description

@mk-mxp

Problem

Some practice exercises have scenarios "to allow for selectively including/excluding test cases based on a property of the test case". We currently have not defined how these are treated within the PHP track.

At the time of writing, these are the scenarios in use within exercises:

~/.cache/exercism/configlet/problem-specifications$ fgrep -H -e 'scenario' -r --include='*.json' exercises/
exercises/bank-account/canonical-data.json:      "scenarios": ["concurrent"],
exercises/dnd-character/canonical-data.json:      "scenarios": ["random"],
exercises/high-scores/canonical-data.json:          "scenarios": ["immutable"],
exercises/protein-translation/canonical-data.json:      "scenarios": ["input-validation", "runtime-validation"],
exercises/parallel-letter-frequency/canonical-data.json:      "scenarios": ["unicode"],
exercises/parallel-letter-frequency/canonical-data.json:      "scenarios": ["concurrent"],
exercises/reverse-string/canonical-data.json:      "scenarios": ["unicode"],
exercises/gigasecond/canonical-data.json:      "scenarios": ["date"],
exercises/gigasecond/canonical-data.json:      "scenarios": ["datetime"],
exercises/gigasecond/canonical-data.json:      "scenarios": ["immutable"],
exercises/diffie-hellman/canonical-data.json:      "scenarios": ["random"],
exercises/diffie-hellman/canonical-data.json:      "scenarios": ["immutable"],
exercises/forth/canonical-data.json:          "scenarios": ["local-scope"],
exercises/armstrong-numbers/canonical-data.json:      "scenarios": ["big-integer"],
exercises/simple-cipher/canonical-data.json:          "scenarios": ["random"],
exercises/triangle/canonical-data.json:          "scenarios": ["floating-point"],
exercises/anagram/canonical-data.json:      "scenarios": ["unicode"],

Suggestion

Scenarios can be grouped into categories based on how we want to treat them:

  • Never implemented: We do not implement those test cases at all.
  • Always implemented: We always include those test cases as required tests.
  • Optionally implemented: Students should be able to opt-in to implement their solution so they pass those test cases in addition to the required tests.

We decide, which scenarios we want to treat as "optional", "always" or "never". I think, this a way to use the categories:

  • Never implemented: concurrent, library-test, any unknown scenario
  • Always implemented: date, datetime, floating-point, local-scope
  • Optionally implemented: big-integer, immutable, input-validation, random, runtime-validation, unicode

Based on these categories, test generators can simply skip over "never" scenarios and treat "always" scenarios like "no scenario".

For the "optional" ones I suggest using PHP constants in students files (define('BIG_INTEGER', 1);) and inside the test methods if (!defined('BIG_INTEGER') || constant('BIG_INTEGER' !== 1)) { $this->markTestSkipped(); }. Using the functions to create / read the constants avoids the problems that arise, when including the students file into a class method (as we do in the test class).

The constants provide a means to easily pre-define them in the students stub file:

[...]

// Change this constant value to 1 if you want to test your solution using Unicode strings:
define('UNICODE', 0);

class ExerciseName
{
    [...]
}

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions