From 98b380cad35ff81f548d1c3f069535ce51ec77ec Mon Sep 17 00:00:00 2001 From: Andrew Nicols Date: Fri, 8 Aug 2025 12:01:45 +0800 Subject: [PATCH 1/3] [docs] Add information for behat::execute() context definition --- general/development/tools/behat/writing.md | 64 ++++++++++++++++++++++ 1 file changed, 64 insertions(+) diff --git a/general/development/tools/behat/writing.md b/general/development/tools/behat/writing.md index 0689e04bf6..3959feb07a 100644 --- a/general/development/tools/behat/writing.md +++ b/general/development/tools/behat/writing.md @@ -264,6 +264,70 @@ Sometimes, you will need to set up data that is specific to your plugin, or perf As well as creating completely new steps, you can also extend some of the standard steps: +#### Calling other steps + +When writing custom steps you will often want to perform actions, such as: + +- clicking a link +- pressing a button +- typing into a field +- calling another existing step + +When doing this you **should** use the `\behat_session_trait::execute()` method to call the existing step to perform the action. +You **should not** call the `->click()` method on a `NodeElement` manually as this will bypass some of the error detection states, and pausing to wait for JavaScript actions to take place. + +The `\behat_session_trait::execute()` method accepts: + +- the name of the method to call on a behat context class; and +- any arguments. + +For example: + +```php title="behat_general.php" + /** + * Toggles the specified admin switch. + * + * @When /^I toggle the "(?P(?:[^"]|\\")*)" admin switch "(?Pon|off)"$/ + * @param string $element Element we look for + * @param string $state The state of the switch + * @throws ElementNotFoundException Thrown by behat_base::find + */ + public function i_toggle_admin_switch($element, $state) { + // First check we are running Javascript, otherwise explode. + if (!$this->running_javascript()) { + throw new \Behat\Mink\Exception\DriverException('Switches are only available with JavaScript enabled'); + } + + // Next check that the node is available. + $node = $this->get_selected_node('checkbox', $element); + $this->ensure_node_is_visible($node); + + // Update the state of the switch. + $field = $node->getAttribute('id'); + if ($state == "on") { + $this->execute('behat_forms::i_set_the_field_to', [$field, 1]); + } else if ($state == "off") { + $this->execute('behat_forms::i_set_the_field_to', [$field, 0]); + } else { + throw new \Behat\Mink\Exception\ExpectationException('Invalid state for switch: ' . $state, $this->getSession()); + } + } +``` + + + +The `\behat_session_trait::execute()` method accepts the method on the behat context class in the callable array format: + +```php +$this->execute([\behat_forms::class, 'i_set_the_field_to'], [$field, 0]); +``` + +:::note + +Only the string format, adn the array callable are supported. You cannot pass a callback method. + +::: + #### Custom selectors (... in the "..." "...") There are a load of different steps which can refer to specific items on-screen, for example From 9786336726f71787ff8c99e1ce312c8ac1441b25 Mon Sep 17 00:00:00 2001 From: Andrew Nicols Date: Wed, 13 Aug 2025 23:02:58 +0800 Subject: [PATCH 2/3] [docs] Migrate unit test docs --- data/migratedPages.yml | 3 + docs/guides/testing/index.md | 608 ++++++++++++++++++ .../development/policies/codingstyle/index.md | 4 +- general/development/tools/behat/writing.md | 2 +- general/development/tools/phpunit/index.md | 2 +- general/releases/3.11.md | 2 +- general/releases/3.4.md | 2 +- 7 files changed, 617 insertions(+), 6 deletions(-) create mode 100644 docs/guides/testing/index.md diff --git a/data/migratedPages.yml b/data/migratedPages.yml index 590584f69b..61a2be9bc0 100644 --- a/data/migratedPages.yml +++ b/data/migratedPages.yml @@ -1806,6 +1806,9 @@ Web_services_API: Web_services_files_handling: - filePath: "/docs/apis/subsystems/external/files.md" slug: "/docs/apis/subsystems/external/files" +Writing_PHPUnit_tests: +- filePath: "/docs/guides/testing/index.md" + slug: "/docs/guides/testing/" Writing_acceptance_tests: - filePath: "/general/development/tools/behat/writing.md" slug: "/general/development/tools/behat/writing" diff --git a/docs/guides/testing/index.md b/docs/guides/testing/index.md new file mode 100644 index 0000000000..2fff25b612 --- /dev/null +++ b/docs/guides/testing/index.md @@ -0,0 +1,608 @@ +--- +title: Writing PHPUnit tests +tags: + - Unit testing + - phpunit +--- + +Moodle PHPUnit integration is designed to allow easy adding of new tests. At the start of each test the state is automatically reset to fresh new installation (unless explicitly told not to reset). + +## Namespaces + +All the stuff under **/tests directories is [subject to some simple rules](/general/development/policies/codingstyle#namespaces-within-.2a.2a.2ftests-directories) when using namespaces. They apply to test cases, fixtures, generators and, in general, any class within those directories. Take a look to them! (grand summary = 100% the same rules that are applied to **/classes directories). + +## Testcase classes + +There are three basic test class that are supposed to used in all Moodle unit tests - basic_testcase, advanced_testcase and provider_testcase. **Please note it is strongly recommended to put only one testcase into each class file.** +;basic_testcase : Very simple tests that do not modify database, dataroot or any PHP globals. It can be used for example when trying examples from the official PHPUnit tutorial. +;advanced_testcase : Enhanced testcase class enhanced for easy testing of Moodle code. +;provider_testcase: Enhanced testcase class, enhanced for easy testing of [Privacy Providers](../../apis/subsystems/privacy/index.md). +There is a fourth testcase class that is specially designed for testing of our Moodle database layer, it should not be used for other purposes. + +### Assertions + +The complete list of assertions can be found in the links below. + +| Moodle version | PHPUnit version | Links | +| --- | --- | --- | +| Moodle 3.11 | PHPUnit 9.5 | [Documentation](https://phpunit.readthedocs.io/en/9.5/assertions.html) | +| Moodle 3.10 | PHPUnit 8.5 | [Documentation](https://phpunit.readthedocs.io/en/8.5/assertions.html) | +| Moodle 3.7 - 3.9 | PHPUnit 7.5 | [Documentation](https://phpunit.readthedocs.io/en/7.5/assertions.html) | +| Moodle 3.4 - 3.6 | PHPUnit 6.5 | [Documentation](https://phpunit.de/manual/6.5/en/assertions.html) | + +### Sample plugin testcase + +PHPUnit tests are located in tests/*_test.php files in your plugin, for example mod/myplugin/tests/sample_test.php, the file should contain only one class that extends advanced_testcase: + +```php + namespace mod_myplugin; + + class sample_test extends \advanced_testcase { + public function test_adding() { + $this->assertEquals(2, 1+2); + } + } +``` + +See [PHPUnit integration#Class and file naming rules](https://docs.moodle.org/dev/PHPUnit_integration#Class_and_file_naming_rules) for more information. + +### Inclusion of Moodle library files + +If you want to include some Moodle library files you should always declare **global $CFG**. The reason is that testcase files may be included from non-moodle code which does not make the global $CFG available automatically. + +### Automatic state reset + +By default after each test Moodle database and dataroot is automatically reset to the original state which was present right after installation. make sure to use $this->resetAfterTest() to indicate that the database or changes of standard global variables are expected. + +If you received the error "Warning: unexpected database modification, resetting DB state" it is because the test is not using $this->resetAfterTest(). + +```php + namespace mod_myplugin; + + class test_something extends \advanced_testcase { + public function test_deleting() { + global $DB; + $this->resetAfterTest(true); + $DB->delete_records('user'); + $this->assertEmpty($DB->get_records('user')); + } + public function test_user_table_was_reset() { + global $DB; + $this->assertEquals(2, $DB->count_records('user', array())); + } + } +``` + +## Generators + +Tests that need to modify default installation may use generators to create new courses, users, etc. All examples on this page should be used from test methods of a test class derived from advanced_testcase. + +Note if you are using PHPUnit [@dataProvider](https://phpunit.de/manual/current/en/writing-tests-for-phpunit.html#writing-tests-for-phpunit.data-providers) functions to provide parameters to unit tests, you can not use the data generator or change the user etc in the data provider function. Data providers **must not instantiate/create data**. Just define it. And then, the test body can proceed with the instantiation/creation. + +### Creating users + +At the start of each test there are only two users present - guest and administrator. If you need to add more test accounts use: + +```php + $user = $this->getDataGenerator()->create_user(); +``` + +You may also specify properties of the user account, for example: + +```php + $user1 = $this->getDataGenerator()->create_user(array('email'=>'user1@example.com', 'username'=>'user1')); +``` + +By default no user is logged-in, use setUser() method to change current $USER value: + +```php + $this->setUser($user1); +``` + +Guest and admin accounts have a shortcut methods: + +```php + $this->setGuestUser(); + $this->setAdminUser(); +``` + +Null can be used to set current user back to not-logged-in: + +```php + $this->setUser(null); +``` + +### Creating course categories + +```php + $category1 = $this->getDataGenerator()->create_category(); + $category2 = $this->getDataGenerator()->create_category(array('name'=>'Some subcategory', 'parent'=>$category1->id)); +``` + +### Creating courses + +```php + $course1 = $this->getDataGenerator()->create_course(); + + $category = $this->getDataGenerator()->create_category(); + $course2 = $this->getDataGenerator()->create_course(array('name'=>'Some course', 'category'=>$category->id)); +``` + +### Creating activities + +Some activity plugins include instance generators. The generator class are defined in plugindirectory/tests/generator/lib.php. + +Example of creation of new course with one page resource: + +```php + $course = $this->getDataGenerator()->create_course(); + $generator = $this->getDataGenerator()->get_plugin_generator('mod_page'); + $generator->create_instance(array('course'=>$course->id)); +``` + +The following is functionally the same, but a bit shorter: + +```php + $course = $this->getDataGenerator()->create_course(); + $page = $this->getDataGenerator()->create_module('page', array('course' => $course->id)); +``` + +### Creating cohorts + +{{Moodle 2.4}} +Since 2.4 there the data generator supports creation of new cohorts. + +```php + $cohort = $this->getDataGenerator()->create_cohort(); +``` + +### Simplified user enrolments + +{{Moodle 2.4}} +Instead of standard enrolment API it is possible to use simplified method in data generator. It is intended to be used with self and manual enrolment plugins. + +```php +$this->getDataGenerator()->enrol_user($userid, $courseid); +$this->getDataGenerator()->enrol_user($userid, $courseid, $teacherroleid); +$this->getDataGenerator()->enrol_user($userid, $courseid, $teacherroleid, 'manual'); +``` + +### Creating scales + +```php +$this->getDataGenerator()->create_scale(); +$this->getDataGenerator()->create_scale(array('name' => $name, 'scale' => $scale, 'courseid' => $courseid, 'userid' => $userid, 'description' => description, 'descriptionformat' => $descriptionformat)); +``` + +### Creating roles + +```php +$this->getDataGenerator()->create_role(); +$this->getDataGenerator()->create_role(array('shortname' => $shortname, 'name' => $name, 'description' => description, 'archetype' => $archetype)); +``` + +### Creating tags + +```php +$this->getDataGenerator()->create_tag(); +$this->getDataGenerator()->create_tag(array( + 'userid' => $userid, + 'rawname' => $rawname, + 'name' => $name, + 'description' => $description, + 'descriptionformat' => $descriptionformat, + 'flag' => $flag +)); +``` + +### Groups + +#### Creating groups + +```php +$this->getDataGenerator()->create_group(array('courseid' => $courseid)); +$this->getDataGenerator()->create_group(array('courseid' => $courseid, 'name' => $name, 'description' => $description, 'descriptionformat' => $descriptionformat)); +``` + +#### Adding users to groups + +```php +$this->getDataGenerator()->create_group_member(array('userid' => $userid, 'groupid' => $groupid)); +$this->getDataGenerator()->create_group_member(array('userid' => $userid, 'groupid' => $groupid, 'component' => $component, 'itemid' => $itemid)); +``` + +#### Creating groupings + +```php +$this->getDataGenerator()->create_grouping(array('courseid' => $courseid)); +$this->getDataGenerator()->create_grouping(array('courseid' => $courseid, 'name' => $name, 'description' => $description, 'descriptionformat' => $descriptionformat)); +``` + +#### Adding groups to groupings + +```php +$this->getDataGenerator()->create_grouping_group(array('groupingid' => $groupingid, 'groupid' => $groupid)); +``` + +### Repositories + +#### Creating repository instances + +{{Moodle 2.5}} +Some respository plugins include instance generators. The generator class are defined in plugindirectory/tests/generator/lib.php.. + +```php +$this->getDataGenerator()->create_repository($type, $record, $options); +``` + +#### Creating repository types + +{{Moodle 2.5}} +Some respository plugins include type generators. The generator class are defined in plugindirectory/tests/generator/lib.php.. + +```php +$this->getDataGenerator()->create_repository_type($type, $record, $options); +``` + +### Creating grades + +#### Grade categories + +```php +$this->getDataGenerator()->create_grade_category(array('courseid' => $courseid)); +$this->getDataGenerator()->create_grade_category(array('courseid' => $courseid, 'fullname' => $fullname)); +``` + +#### Grade items + +```php +$this->getDataGenerator()->create_grade_item(); +$this->getDataGenerator()->create_grade_item(array('itemtype' => $itemtype, 'itemname' => $itemname, 'outcomeid' => $outcomeid, 'scaleid' => $scaleid, 'gradetype' => $gradetype)); +``` + +#### Outcomes + +```php +$this->getDataGenerator()->create_grade_outcome(); +$this->getDataGenerator()->create_grade_item(array('fullname' => $fullname)); +``` + +### Other types of plugin + +{{Moodle 2.5}} +Any other type of plugin can have a generator. The generator class should extend component_generator_base, and then you can get an instance using $mygenerator = $this->getDataGenerator()->get_plugin_generator($frankenstylecomponentname); + +For some types of plugin, like mod documented above, there may be a more specific class than component_generator_base to extend, like testing_module_generator. That will give a consistent set of method names to use. Otherwise, you can create whatever methods you like on your generator, to create the different things you need to work whith. + +## Long tests + +All standard test should execute as fast as possible. Tests that take a longer time to execute (>10s) or are otherwise expensive (such as querying external servers that might be flooded by all dev machines) should be execute only when PHPUNIT_LONGTEST is true. This constant can be set in phpunit.xml or directly in config.php. + +## Large test data + +See advanced_testcase::createXMLDataSet() and advanced_testcase::createCsvDataSet() and related functions there for easier ways to manage large test data sets within files rather than arrays in code. See [PHPUnit integration#Extra methods](https://docs.moodle.org/dev/PHPUnit_integration#Extra_methods) + +## Testing sending of messages + +{{Moodle 2.4}} +You can temporarily redirect all messages sent via message_send() to a message sink object. This allows developers to verify that the tested code is sending expected messages. + +To test code using messaging first disable the use of transactions and then redirect the messaging into a new message sink, you can inspect the results later. + +```php +$this->preventResetByRollback(); +$sink = $this->redirectMessages(); +//... code that is sending messages +$messages = $sink->get_messages(); +$this->assertEquals(3, count($messages)); +//.. test messages were generated in correct order with appropriate content +``` + +{{Moodle 4.4}} +Since **4.4** there are two new methods that support getting the messages for specific **components** and **message types**. + +```php +$sink = $this->redirectMessages(); +//... code that is sending messages +$messages = $sink->get_messages_by_component('mod_forum'); +$this->assertEquals(3, count($messages)); +//.. test messages were generated in correct order with appropriate content +``` + +```php +$sink = $this->redirectMessages(); +//... code that is sending messages +$messages = $sink->get_messages_by_component_and_type('core', 'messagecontactrequests'); +$this->assertEquals(3, count($messages)); +//.. test messages were generated in correct order with appropriate content +``` + +## Testing sending of emails + +{{Moodle 2.6}} +You can temporarily redirect emails sent via email_to_user() to a email message sink object. This allows developers to verify that the tested code is sending expected emails. + +To test code using messaging first unset 'noemailever' setting and then redirect the emails into a new message sink where you can inspect the results later. + +```php +unset_config('noemailever'); +$sink = $this->redirectEmails(); +//... code that is sending email +$messages = $sink->get_messages(); +$this->assertEquals(1, count($messages)); +``` + +## Logstores + +You can test events which were written to a logstore, but you must disable transactions, enable at least one valid logstore, and disable logstore buffering to ensure that the events are written to the database before the tests execute. + +```php +$this->preventResetByRollback(); +set_config('enabled_stores', 'logstore_standard', 'tool_log'); +set_config('buffersize', 0, 'logstore_standard'); +get_log_manager(true); +``` + +## Check your coverage + +{{Moodle 3.7}} +PHPUnit has the ability to generate code coverage information for your unit tests. + +Prior to Moodle 3.7, this coverage would load all files and generate coverage for everything regardless of whether that file could be covered at all, or whether it was intentionally covered. + +Since Moodle 3.7 the **phpunit.xml** configuration contains generated coverage include and exclude information for each component. + +### Generating include and exclude configuration + +{{Moodle 3.11}} +You can programatically describe which files will be checked for coverage by creating a coverage.php file alongside the tests that you are writing. + +Since Moodle 4.0, a default configuration is applied for all plugins and it is not necessary to supply a coverage.php unless you wish to cover additional files. + +The coverage.php file allows you to list include and exclude files and folders within the component being +tested. All paths specified are relative to the component being tested. For example, when working with **mod_forum** your +code will be in **mod/forum**, and its unit tests will be in **mod/forum/tests/example_test.php**. The coverage file +for this would be in **mod/forum/tests/coverage.php** and all paths specified would be relative to **mod/forum**. + +It is possible to specify a combination of included files, included folders, excluded files, and excluded folders. This would allow you, for example, to include the entire **classes** directory, but exclude.a specific file or folder within it. + +The following is an example coverage.php file from **mod_forum**: + +Note: For Moodle versions 3.7 to 3.10, the [syntax used](https://docs.moodle.org/dev/index.php?title=Writing_PHPUnit_tests&oldid=58177#Check_your_coverage) was slightly different. + +```php +return new class extends phpunit_coverage_info { + /** @var array The list of folders relative to the plugin root to include in coverage generation. */ + protected $includelistfolders = [ + 'classes', + 'externallib.php', + ]; + + /** @var array The list of files relative to the plugin root to include in coverage generation. */ + protected $includelistfiles = []; + + /** @var array The list of folders relative to the plugin root to exclude from coverage generation. */ + protected $excludelistfolders = []; + + /** @var array The list of files relative to the plugin root to exclude from coverage generation. */ + protected $excludelistfiles = []; +}; +``` + +Also, note that you can better define which class or function each test is effectively covering by using the @covers annotation as [described in the documention](https://phpunit.readthedocs.io/en/9.5/code-coverage-analysis.html#specifying-covered-code-parts). + +Since Moodle 4.0, the following default configuration is applied:{{Moodle 4.0}} + +```php +return new class extends phpunit_coverage_info { + /** @var array The list of folders relative to the plugin root to include in coverage generation. */ + protected $includelistfolders = [ + 'classes', + 'tests/generator', + ]; + + /** @var array The list of files relative to the plugin root to include in coverage generation. */ + protected $includelistfiles = [ + 'externallib.php', + 'lib.php', + 'locallib.php', + 'renderer.php', + 'rsslib.php', + ]; + + /** @var array The list of folders relative to the plugin root to exclude from coverage generation. */ + protected $excludelistfolders = []; + + /** @var array The list of files relative to the plugin root to exclude from coverage generation. */ + protected $excludelistfiles = []; +}; +``` + +If a coverage.php file already exists, then the defaults will be added to the values already defined. + +## Best practice + +There are several best practices, suggestions, and things to avoid which you should consider when writing unit tests. Some of these are described below. + +### Code coverage + +PHPUnit has the ability to generate code coverage information for your unit tests and this is well supported since.Moodle 3.7. We recommend that you consider checking the coverage of your plugins when you write your code. + +We recommend that you explicitly set the @covers annotation as described in the [PHPUnit documentation](https://phpunit.readthedocs.io/en/9.5/annotations.html#appendixes-annotations-covers-tables-annotations). + +### Keep use of resetAfterTest to a minimum + +Although many of the examples described above use the `resetAfterTest` nomenclature to reset the database and filesystem after your test completes, you should ideally not use this unless you have to. +Generally speaking you should aim to write code which is mockable, and does not require real fixtures. +Use of resetAfterTest will also slow your tests down. + +### Be careful with shared setUp and instance variables + +You should be careful of how you create and use instance variables in PHPUnit tests for two main reasons: + +1. If you create any fixtures in the setUp, or call the resetAfterTest function, these fixtures and conditions will apply for _all_ tests in the testsuite. You will not be able to add another test to the suite which does not require these conditions without those conditions being fulfilled anyway. This can lead to slow tests. +1. PHPUnit creates an instance of each testcase during its bootstrap phase, and does not dispose of it for the lifetime of the test run. Anything which causes data to be stored as instance data within the testcase will be stored in memory until the _entire suite_ completes. This means that any fixture which is setup and not actively discarded will not be garbage collected and lead to memory bloat. In severe cases this can lead to memory exhaustion. + +Existing testcases which contain setUp which either generate data, or set resetAfterTest should be phased out, and no new cases should be introduced. + +### Make use of the dataProvider functionality + +The dataProvider functionality of PHPUnit is an extremely powerful and useful feature which allows you to verify a function quickly and easily with a range of different conditions. +However, the following rules should be followed when using dataProviders: + +- Keep addition of resettable data requiring resetAfterTest to a minimum - this will lead to many slow tests +- Data providers **must not instantiate/create data**. Just define it. And then, the test body can proceed with the instantiation/creation. The dataProvider is called after the testSuite is instantiated, but before any tests are run. Each test will run a full setUp and tearDown, which will destroy any data which was created. + +```php +/** + * Test function accepts parameters passed from the specified data provider. + * + * @dataProvider foobar_provider + * @param int $foor + * @param int $bar + */ +public function test_foobar(int $foo, int $bar) { + // Perform the tests here. +} + +/** + * Data provider for {@see self::test_foobar()}. + * + * @return array List of data sets - (string) data set name => (array) data + */ +public function foobar_provider(): array { + return [ + 'Same numbers' => [ + 'foo' => 42, + 'bar' => 42, + ], + 'Different numbers' => [ + 'foo' => 21, + 'bar' => 84, + ], + ]; +} +``` + +## Extra test settings + +Usually the test should not interact with any external systems and it should work the same on all systems. But sometimes you need to specify some option for connection to external systems or system configuration. It is intentionally not possible to use $CFG settings from config.php. + +There are several ways how to inject your custom settings: + +- define test setting constants in your phpunit.xml file +- define test setting constants in your config.php +These constants may be then used in your test or plugin code. + +## Upgrading unit tests to work with Moodle 4.4 and up (PHPUnit 9.6) + +{{Moodle 4.4}} + +With Moodle 4.4, **PHPUnit was upgraded to 9.6** (from 9.5 being used in previous versions). This was done to **warn to developers in advance** about functionality that has been deprecated in the 9.5 series and will be removed so will stop working with the next major update to PHPUnit 10.x (see [MDL-81266](https://moodle.atlassian.net/browse/MDL-81266) and linked issues for more details). + +While everything should continue working without modification with PHPUnit 9.6, you will may get a **good number of deprecation warnings** ("W" in the tests output) that **should be analysed**, replacing or removing them as soon as possible, because all those warnings will become errors with next PHPUnit upgrade. + +A good summary of all the **changes and replacements to perform** is available in the [lib/upgrade.txt](https://github.com/stronk7/moodle/commit/b2131ceff74da4c23928936f238d676a08e07d7f) file. With main points being: + +- [MDL-81281](https://moodle.atlassian.net/browse/MDL-81281). A number of attribute-related assertions have been deprecated, will be removed with PHPUnit 10. Alternatives for **some** of them are available: + - assertClassHasAttribute() + - assertClassNotHasAttribute() + - assertClassHasStaticAttribute() + - assertClassNotHasStaticAttribute() + - assertObjectHasAttribute() => assertObjectHasProperty() + - assertObjectNotHasAttribute() => assertObjectNotHasProperty() +- [MDL-81266](https://moodle.atlassian.net/browse/MDL-81266). A number of deprecation, notice, warning and error expectations have been deprecated, will be removed with PHPUnit 10. No alternative exists. A working replacement is available in the linked issue, hopefully there aren't many cases. + - expectDeprecation() + - expectDeprecationMessage() + - expectDeprecationMessageMatches() + - expectError() + - expectErrorMessage() + - expectErrorMessageMatches() + - expectNotice() + - expectNoticeMessage() + - expectNoticeMessageMatches() + - expectWarning() +- expectWarningMessage() +- expectWarningMessageMatches() +- [MDL-81308](https://moodle.atlassian.net/browse/MDL-81308). The ->withConsecutive() functionality on PHPUnit mocks has been **silently** deprecated, will be removed with PHPUnit 10. Note that this won't affect PHPUnit 9.6 runs and an alternative path will be proposed in the linked issue, part of the PHPUnit 10 epic. +- PHPUnit\Framework\TestCase::getMockClass() has been deprecated, will be removed with PHPUnit 10. No clear alternative exists and won't be investigated, because there aren't cases in core. +- Cannot use the Test suffix on abstract test case classes. Proceed to rename them to end with TestCase instead. + +## Upgrading unit tests to work with Moodle 3.11 and up (PHPUnit 9.5) + +{{Moodle 3.11}} + +With Moodle 3.11, **PHPUnit was upgraded to 9.5** (from 8.5 being used in previous versions). This was done to **better align** the testing environment with PHP versions supported by Moodle 3.11 (7.3, 7.4 and [8.0](/general/development/policies/php)) (see [MDL-71036](https://moodle.atlassian.net/browse/MDL-71036) and linked issues for more details). + +While a lot of existing tests will work without modification with PHPUnit 9.5, you will get a **good number of deprecation warnings** ("W" in the tests output) that **should be replaced by their new counterparts as soon as possible**, because all those warnings will become errors with next PHPUnit upgrade. + +To find more information about the changes coming with PHPUnit 9.5, it's recommended to read the following resources: + +- [A good article](https://thephp.cc/news/2020/02/migrating-to-phpunit-9) explaining all the main changes in the release. +- [PHPUnit 9 release Announcement](https://phpunit.de/announcements/phpunit-9.html). +- These multiple detailed changelogs (because all them have included modifications requiring changes): [9.0](https://github.com/sebastianbergmann/phpunit/blob/9.0.0/ChangeLog-9.0.md), [9.1](https://github.com/sebastianbergmann/phpunit/blob/9.1.0/ChangeLog-9.1.md), [9.3](https://github.com/sebastianbergmann/phpunit/blob/9.3.0/ChangeLog-9.3.md), [9.5](https://github.com/sebastianbergmann/phpunit/blob/9.5.0/ChangeLog-9.5.md) +- [Official PHPUnit manual](https://phpunit.readthedocs.io/en/9.5/). +A good summary of all the **changes and replacements to perform** is available in the [lib/upgrade.txt](https://github.com/moodle/moodle/blob/e3a46964dc6d8ca1558c6e1e8dfdf3c1745eeaed/lib/upgrade.txt#L5-L65) file. With main points being: +- All the changes that were deprecated with PHPUnit 8.5 (see the section below) are now removed and will lead to errors. +- assertContains() now performs stricter comparison (like assertSame() does). New assertContainsEquals() has been created to provide the old behavior. +- Changes to the phpunit.xml schema, mostly internal. These only will impact if you are using custom phpunit.xml files: + - The previous section is now (better) called . And, within it: + - has been replaced by . + - is not a child of anymore, but of . + - But with implications when defining the [coverage information](https://docs.moodle.org/dev/#Check_your_coverage) because $whitelistxxx properties used by the coverage.php files have been deprecated, instead use includelistfolders and includelistfiles (to better map the elements in the xml). +- Warning: It's not possible to run individual test files any more. Use any of the alternative execution methods (filter, suite, config) to specify which tests you want to run. This will be hopefully fixed in [MDL-71049](https://moodle.atlassian.net/browse/MDL-71049) once it has been agreed which the best way to proceed is. +- Deprecations, deprecations, deprecations. Lots of them in often used assertions: file assertions, regexp assertions, exception expectations... again, note that all them will become errors with the next PHPUnit update, so **the recommendation is to update them ASAP**. +Finally, it's also a good idea to [browse the changes associated with the issue](https://github.com/moodle/moodle/compare/fc335f5...713722c), hopefully with useful explanations in the commit messages. + +## Upgrading unit tests to work with Moodle 3.10 and up (PHPUnit 8.5) + +{{Moodle 3.10}} + +With Moodle 3.10, **PHPUnit was upgraded to 8.5** (from 7.5 being used in older version). This was done to **better align** the testing environment with PHP versions supported by Moodle 3.10 (7.2, 7.3 and 7.4) and also to provide an **easier jump to PHPUnit 9.x** that will be needed for Moodle 3.11 (php versions 7.3, 7.4 and 8.0), removing a lot of deprecated stuff in advance. (see [MDL-67673](https://moodle.atlassian.net/browse/MDL-67673) and [MDL-64600](https://moodle.atlassian.net/browse/MDL-64600) for more details). + +While 99% of existing tests will work without modification with PHPUnit 8.5, you will get a **good number of deprecation warnings** ("W" in the tests output) that **should be replaced by their new counterparts as soon as possible**, because all those warnings will become errors with next PHPUnit upgrade. + +To find more information about the changes coming with PHPUnit 8.5, it's recommended to read the following resources: + +- [A good article](https://thephp.cc/news/2019/02/help-my-tests-stopped-working) explaining all the main changes in the release. +- [PHPUnit 8 release Announcement](https://phpunit.de/announcements/phpunit-8.html). +- [Detailed changelog of the release](https://github.com/sebastianbergmann/phpunit/blob/130104cf796a88dd1547dc5beb8bd555c2deb55e/ChangeLog-8.0.md). +- [Official PHPUnit manual](https://phpunit.readthedocs.io/en/8.5/). +- A good summary of all the **changes and replacements to perform** is available in the [lib/upgrade.txt](https://github.com/moodle/moodle/blob/6594c54b2eef62499d304bfa0939999e3a14246e/lib/upgrade.txt#L5-L37) file. With main points being: + - Support for PHP 7.0 dropped (because all template methods (setUp(), tearDown()..) now require to return void. This will mostly impact 3rd-part plugins that were still running the same tests against old branches of Moodle with PHP 7.0 support. + - PHPUnit/DBUnit has been removed are replaced by a lightweight alternative. + - Deprecations, deprecations, deprecations. Lots of them in often used assertions (assertContains(), assertEquals()...) and also @expectedExceptionXXX annotations. Again, note that all them will become errors with PHPUnit 9. +- Finally, it's also a good idea to [browse the changes associated with the issue](https://github.com/moodle/moodle/compare/5903054...b13ec3c), hopefully with useful enough explanations in the commit messages. + +## Upgrading unit tests to work with Moodle 3.7 and up (PHPUnit 7.5) + +{{Moodle 3.7}} + +With Moodle 3.7, **PHPUnit was upgraded to 7.5** (from 6.x being used in older version). This was done to better align the testing environment with PHP versions supported by Moodle 3.7 (7.1, 7.2 and 7.3). (see [MDL-65204](https://moodle.atlassian.net/browse/MDL-65204) and linked issues for more details). While internally [a lot of things changed with PHPUnit 7](https://phpunit.de/announcements/phpunit-7.html) (PHP 7.1-isms, typed signatures, void returns, assertEquals() changes), thanks to our **wrapping layer** (basic and advanced testcases...) impact expected into old existing unit tests is expected to be reduced and upgrades, easy to achieve. + +To find more information about the changes coming with PHPUnit 7, it's recommended to read the following resources: + +- [PHPUnit 7 release Announcement](https://phpunit.de/announcements/phpunit-7.html). +- [Detailed changelog of the release](https://github.com/sebastianbergmann/phpunit/blob/520723129e2b3fc1dc4c0953e43c9d40e1ecb352/ChangeLog-7.5.md), paying special attention to the added, deprecated, changed and deleted sections. +- [Official PHPUnit manual](https://phpunit.readthedocs.io/en/7.5/). +- Changes performed into core, mainly: new, stricter, signatures ([26218b7](https://github.com/moodle/moodle/commit/26218b7)) and assertEquals() changes, specially important when comparing strings, now performed using strict (===) equals ([85f47ba](https://github.com/moodle/moodle/commit/85f47ba)). + +## Upgrading unit tests to work with Moodle 3.4 and up (PHPUnit 6) + +{{Moodle 3.4}} + +With Moodle 3.4, **PHPUnit was upgraded to 6.4** (from 5.5 being used in older version). This was done to better align the testing environment with PHP versions supported by Moodle 3.4 (7.0, 7.1 and 7.2). (see [MDL-60611](https://moodle.atlassian.net/browse/MDL-60611) and linked issues for more details). While internally [a lot of things changed with PHPUnit 6](https://github.com/sebastianbergmann/phpunit/wiki/Release-Announcement-for-PHPUnit-6.0.0#backwards-compatibility-issues) (namespaced classes being the more noticeable), thanks to our **wrapping layer** (basic and advanced testcases...) impact expected into old existing unit tests is expected to be reduced and upgrades, easy to achieve. + +Still, in **some cases, it will impossible to maintain compatibility** of tests between old (pre 3.4) tests and new ones, especially **when direct use of any phpunit class is performed**. Luckily, both GitHub Actions and CI tests will detect this situation and it shouldn't be hard to keep all supported branches in core passing ok. Plugins may be trickier, if the same branch is attempting to work against multiple core branches and they are using some phpunit class directly. + +To find more information about the changes coming with PHPUnit 6, it's recommended to read the following resources: + +- [A very good mini-guide](https://thephp.cc/news/2017/02/migrating-to-phpunit-6) showing all the important changes. +- [PHPUnit 6 release Announcement](https://github.com/sebastianbergmann/phpunit/wiki/Release-Announcement-for-PHPUnit-6.0.0#backwards-compatibility-issues). +- [Detailed changelog of the release](https://github.com/sebastianbergmann/phpunit/blob/9d0c024d2099531442d862b66b0ad7cf35ed8e78/ChangeLog-6.0.md), paying special attention to the changed and deleted sections. +- Changes performed into core, mainly: namespace class renaming ([801a372](https://github.com/moodle/moodle/commit/801a372dadb6e11c8781547603e3f0a59ce5638f)) and deprecated stuff ([796e48a](https://github.com/moodle/moodle/commit/796e48a58bf18533bdca423fff7949ab119101c4)) + +## See also + +- [PHPUnit integration](https://docs.moodle.org/dev/PHPUnit_integration) +- [PHPUnit](/general/development/tools/phpunit) diff --git a/general/development/policies/codingstyle/index.md b/general/development/policies/codingstyle/index.md index 36bfd4b29e..b341068c67 100644 --- a/general/development/policies/codingstyle/index.md +++ b/general/development/policies/codingstyle/index.md @@ -1582,8 +1582,8 @@ This tag is valid and can be used optionally to indicate the method or function There are some tags that are only allowed within some contexts and not globally. More precisely: - `@Given`, `@When`, `@Then`, within the [behat steps definitions](../../tools/behat/writing.md#writing-new-acceptance-test-step-definitions). -- `@covers`, `@coversDefaultClass`, `@coversNothing`, `@uses` to better control coverage within [unit tests](https://docs.moodle.org/dev/Writing_PHPUnit_tests#Generators). -- `@dataProvider` and `@testWith`, to provide example data and expectations, within [unit tests](https://docs.moodle.org/dev/Writing_PHPUnit_tests#Generators). +- `@covers`, `@coversDefaultClass`, `@coversNothing`, `@uses` to better control coverage within [unit tests](/docs/guides/testing/#generators). +- `@dataProvider` and `@testWith`, to provide example data and expectations, within [unit tests](/docs/guides/testing/#generators). - `@depends`, to express dependencies between tests, where each producer returned data in passed to consumers. See [`@depends` examples](https://docs.phpunit.de/en/9.6/writing-tests-for-phpunit.html#writing-tests-for-phpunit-examples-stacktest2-php) for more information. - `@group`, for easier collecting unit tests together, following the guidelines in the [PHPUnit MoodleDocs](../../tools/phpunit/index.md#using-the-group-annotation). - `@requires`, to specify unit test requirements and skip if not fulfilled. See [`@requires` usages](https://docs.phpunit.de/en/9.6/incomplete-and-skipped-tests.html#incomplete-and-skipped-tests-requires-tables-api) for more information. diff --git a/general/development/tools/behat/writing.md b/general/development/tools/behat/writing.md index 3959feb07a..48c97f0d0b 100644 --- a/general/development/tools/behat/writing.md +++ b/general/development/tools/behat/writing.md @@ -368,7 +368,7 @@ new behat step definitions for your plugin, and allows you to re-use data genera Full documentation of this process and all available options can be found in the [PHPDoc for behat_generator_base](https://github.com/moodle/moodle/blob/1d4fdb0d1c60448104bc9eac79b5123863c67cbd/lib/behat/classes/behat_generator_base.php#L33). A core example of this can be found in [/mod/quiz/tests/generator](https://github.com/moodle/moodle/tree/main/mod/quiz/tests/generator) and [quiz_reset.feature](https://github.com/moodle/moodle/blob/1d4fdb0d1c60448104bc9eac79b5123863c67cbd/mod/quiz/tests/behat/quiz_reset.feature#L51). What follows is a simple example. -To begin, you need a [generator](https://docs.moodle.org/dev/Writing_PHPUnit_tests#Generators) in `/*your*/*plugin*/tests/generator/lib.php`. If you are generating a type of entity called "thing", your generator will need a method called create_thing, which accepts an object: +To begin, you need a [generator](/docs/guides/testing/#generators) in `/*your*/*plugin*/tests/generator/lib.php`. If you are generating a type of entity called "thing", your generator will need a method called create_thing, which accepts an object: ```php class local_myplugin_generator extends component_generator_base { diff --git a/general/development/tools/phpunit/index.md b/general/development/tools/phpunit/index.md index 63abcb7342..0f2440898a 100644 --- a/general/development/tools/phpunit/index.md +++ b/general/development/tools/phpunit/index.md @@ -283,7 +283,7 @@ define('TEST_EXTERNAL_FILES_HTTP_URL', 'http://localhost/moodle-exttests'); ## Writing new tests - read [official PHPUnit online documentation](https://docs.phpunit.de/en/9.6/) -- see [Writing PHPUnit tests](https://docs.moodle.org/dev/Writing_PHPUnit_tests) +- see [Writing PHPUnit tests](/docs/guides/testing/) ## PHPUnit support in IDEs diff --git a/general/releases/3.11.md b/general/releases/3.11.md index 8bdf8a60cc..07ae3b8f80 100644 --- a/general/releases/3.11.md +++ b/general/releases/3.11.md @@ -215,7 +215,7 @@ For the best experience and optimum security, we recommend that you keep your br ## For developers -The PHPUnit upgrade will almost certainly break your tests. See [Writing PHPUnit tests#Upgrading unit tests to work with Moodle 3.11 and up .28PHPUnit 9.5.29](https://docs.moodle.org/dev/Writing_PHPUnit_tests#Upgrading_unit_tests_to_work_with_Moodle_3.11_and_up_.28PHPUnit_9.5.29) +The PHPUnit upgrade will almost certainly break your tests. See [Writing PHPUnit tests#Upgrading unit tests to work with Moodle 3.11 and up .28PHPUnit 9.5.29](/docs/guides/testing/#upgrading-unit-tests-to-work-with-moodle-3.11-and-up-.28phpunit-9.5.29) - [MDL-52817](https://moodle.atlassian.net/browse/MDL-52817) - New sql_group_concat db method - [MDL-64554](https://moodle.atlassian.net/browse/MDL-64554) - Add module for displaying moodleform in a modal window diff --git a/general/releases/3.4.md b/general/releases/3.4.md index 1e6b86540c..bd43fc6331 100644 --- a/general/releases/3.4.md +++ b/general/releases/3.4.md @@ -154,7 +154,7 @@ This list only includes security issues fixed after 3.3.2 release. Refer to othe ### For developers -- [MDL-60611](https://moodle.atlassian.net/browse/MDL-60611) - Upgrade PHPUnit to 6.4 to ensure compatibility with PHP 7.2 - [may require changes in unittests](https://docs.moodle.org/dev/Writing_PHPUnit_tests#Upgrading_unit_tests_to_work_with_Moodle_3.4_and_up_.28PHPUnit_6.29). +- [MDL-60611](https://moodle.atlassian.net/browse/MDL-60611) - Upgrade PHPUnit to 6.4 to ensure compatibility with PHP 7.2 - [may require changes in unittests](/docs/guides/testing/#upgrading-unit-tests-to-work-with-moodle-3.4-and-up-.28phpunit-6.29). - [MDL-58948](https://moodle.atlassian.net/browse/MDL-58948) - Compatibility with chrome mink driver - [MDL-53169](https://moodle.atlassian.net/browse/MDL-53169) - Provide a way to retrieve all courses a user can potentially access. - [MDL-59459](https://moodle.atlassian.net/browse/MDL-59459) - Global Search: Increase file indexing coverage From 0bfb61abc626e3bf3a5238cadd5285f44a11824e Mon Sep 17 00:00:00 2001 From: Andrew Nicols Date: Wed, 13 Aug 2025 23:54:37 +0800 Subject: [PATCH 3/3] [docs] Tidy up PHPUnit docs --- docs/guides/testing/index.md | 579 +++++++++++------- .../development/policies/codingstyle/index.md | 2 +- 2 files changed, 363 insertions(+), 218 deletions(-) diff --git a/docs/guides/testing/index.md b/docs/guides/testing/index.md index 2fff25b612..21f59d9d64 100644 --- a/docs/guides/testing/index.md +++ b/docs/guides/testing/index.md @@ -5,36 +5,73 @@ tags: - phpunit --- -Moodle PHPUnit integration is designed to allow easy adding of new tests. At the start of each test the state is automatically reset to fresh new installation (unless explicitly told not to reset). +Moodle PHPUnit integration is designed to allow easy creation of new tests. +At the start of each test the state is automatically reset to fresh new installation (unless explicitly told not to reset). ## Namespaces -All the stuff under **/tests directories is [subject to some simple rules](/general/development/policies/codingstyle#namespaces-within-.2a.2a.2ftests-directories) when using namespaces. They apply to test cases, fixtures, generators and, in general, any class within those directories. Take a look to them! (grand summary = 100% the same rules that are applied to **/classes directories). +All the files under `**/tests` directories are [subject to some simple rules](/general/development/policies/codingstyle#namespaces-within-tests) when using namespaces. +They apply to test cases, fixtures, generators and, in general, any class within those directories. +These are the same rules that are applied to the standard `**/classes` directories. ## Testcase classes -There are three basic test class that are supposed to used in all Moodle unit tests - basic_testcase, advanced_testcase and provider_testcase. **Please note it is strongly recommended to put only one testcase into each class file.** -;basic_testcase : Very simple tests that do not modify database, dataroot or any PHP globals. It can be used for example when trying examples from the official PHPUnit tutorial. -;advanced_testcase : Enhanced testcase class enhanced for easy testing of Moodle code. -;provider_testcase: Enhanced testcase class, enhanced for easy testing of [Privacy Providers](../../apis/subsystems/privacy/index.md). -There is a fourth testcase class that is specially designed for testing of our Moodle database layer, it should not be used for other purposes. +There are two basic test case classes that should be used in all other Moodle unit tests: -### Assertions +- basic_testcase; and +- advanced_testcase. + +:::note + +Moodle's coding style mandates that only one testcase is placed into each file, and that the file name **must** match the class name. + +::: + +The `basic_testcase` is intended for very simple tests that do not modify the database, file system, or any PHP globals. It can be used for pure unit tests which do not modify data in any way. + +The `advanced_testcase` is an enhanced testcase class which includes a number of test helpers for easier testing of Moodle code. + +:::note Creating additional test case classes + +Moodle supports the creation of custom testcase classes for more specific purposes. + +These **must** be defined as abstract and **must not** contain any tests. + +::: + +## Assertions + +Where possible the standard PHPUnit assertions should be used. The complete list of assertions can be found in the links below. | Moodle version | PHPUnit version | Links | | --- | --- | --- | -| Moodle 3.11 | PHPUnit 9.5 | [Documentation](https://phpunit.readthedocs.io/en/9.5/assertions.html) | +| Moodle 5.0+ | PHPUnit 11 | [Documentation](https://phpunit.readthedocs.io/en/11.4/assertions.html) | +| Moodle 4.4 - 4.5 | PHPUnit 9.6 | [Documentation](https://phpunit.readthedocs.io/en/9.6/assertions.html) | +| Moodle 3.11 - 4.3 | PHPUnit 9.5 | [Documentation](https://phpunit.readthedocs.io/en/9.5/assertions.html) | | Moodle 3.10 | PHPUnit 8.5 | [Documentation](https://phpunit.readthedocs.io/en/8.5/assertions.html) | | Moodle 3.7 - 3.9 | PHPUnit 7.5 | [Documentation](https://phpunit.readthedocs.io/en/7.5/assertions.html) | | Moodle 3.4 - 3.6 | PHPUnit 6.5 | [Documentation](https://phpunit.de/manual/6.5/en/assertions.html) | -### Sample plugin testcase +## Writing new test files -PHPUnit tests are located in tests/*_test.php files in your plugin, for example mod/myplugin/tests/sample_test.php, the file should contain only one class that extends advanced_testcase: +PHPUnit tests are located in the `tests` directory of each component. -```php +Test files: + +- **MUST** be located in the `tests` directory of your component; +- **MUST** be named in lowercase characters; +- **MUST** have a filename ending in `_test.php`; +- **MUST** have a class: + - whose name exactly matches the filename; and + - which extends a `_testcase` class (for example `\advanced_testcase`); +- **MUST** only have one test class in each file; and +- _SHOULD_ use the same namespace as the unit under test. + +For example: + +```php title="mod/myplugin/tests/sample_test.php" namespace mod_myplugin; class sample_test extends \advanced_testcase { @@ -44,30 +81,38 @@ PHPUnit tests are located in tests/*_test.php files in your plugin, for } ``` -See [PHPUnit integration#Class and file naming rules](https://docs.moodle.org/dev/PHPUnit_integration#Class_and_file_naming_rules) for more information. +## Inclusion of Moodle library files + +Wherever possible, class autoloading _should_ be used. -### Inclusion of Moodle library files +Where autoloading of classes is not possible: -If you want to include some Moodle library files you should always declare **global $CFG**. The reason is that testcase files may be included from non-moodle code which does not make the global $CFG available automatically. +- classes which are _required_ to define the test class _may_ be included in the file; but +- fixtures and code under test _should_ be loaded in the `setUpBeforeClass()` method. ### Automatic state reset -By default after each test Moodle database and dataroot is automatically reset to the original state which was present right after installation. make sure to use $this->resetAfterTest() to indicate that the database or changes of standard global variables are expected. +Any test making use of the standard Moodle test cases (`advanced_testcase` and `basic_testcase`) will automatically reset the Moodle database and data root to their original state between tests, however you should make sure to call `$this->resetAfterTest()` to indicate that the database or changes of standard global variables are expected. -If you received the error "Warning: unexpected database modification, resetting DB state" it is because the test is not using $this->resetAfterTest(). +If you modify data and _do not_ call `$this->resetAfterTest()` then you will receive a warning such as: -```php +> Warning: unexpected database modification, resetting DB state + +```php title="Demonstration of database reset" namespace mod_myplugin; class test_something extends \advanced_testcase { public function test_deleting() { global $DB; + $this->resetAfterTest(true); $DB->delete_records('user'); $this->assertEmpty($DB->get_records('user')); } + public function test_user_table_was_reset() { global $DB; + $this->assertEquals(2, $DB->count_records('user', array())); } } @@ -75,13 +120,21 @@ If you received the error "Warning: unexpected database modification, resetting ## Generators -Tests that need to modify default installation may use generators to create new courses, users, etc. All examples on this page should be used from test methods of a test class derived from advanced_testcase. +Tests that need to modify default installation data may make use of data generators to create objects including courses, users, enrolments, and so on. All examples on this page should be used from test methods of a test class derived from `advanced_testcase`. + +:::note Use of generators in data providers + +If you are using PHPUnit [@dataProvider](https://phpunit.de/manual/current/en/writing-tests-for-phpunit.html#writing-tests-for-phpunit.data-providers) functions to provide parameters to unit tests, you **can not** use the data generator or change the user etc in the data provider function. -Note if you are using PHPUnit [@dataProvider](https://phpunit.de/manual/current/en/writing-tests-for-phpunit.html#writing-tests-for-phpunit.data-providers) functions to provide parameters to unit tests, you can not use the data generator or change the user etc in the data provider function. Data providers **must not instantiate/create data**. Just define it. And then, the test body can proceed with the instantiation/creation. +Data providers **must not instantiate or create data**. They may only _define_ it. + +The data providers are called _before_ any test has been run and the database will be reset before the test is actually run. + +::: ### Creating users -At the start of each test there are only two users present - guest and administrator. If you need to add more test accounts use: +At the start of each test there are only two users present - `guest` and `admin. If you need to add more test accounts use: ```php $user = $this->getDataGenerator()->create_user(); @@ -90,23 +143,26 @@ At the start of each test there are only two users present - guest and administr You may also specify properties of the user account, for example: ```php - $user1 = $this->getDataGenerator()->create_user(array('email'=>'user1@example.com', 'username'=>'user1')); +$user1 = $this->getDataGenerator()->create_user([ + 'email' => 'user1@example.com', + 'username' => 'user1', +]); ``` -By default no user is logged-in, use setUser() method to change current $USER value: +By default no user is logged-in. You can use the `setUser()` method to simulate a user login: ```php - $this->setUser($user1); +$this->setUser($user1); ``` Guest and admin accounts have a shortcut methods: ```php - $this->setGuestUser(); - $this->setAdminUser(); +$this->setGuestUser(); +$this->setAdminUser(); ``` -Null can be used to set current user back to not-logged-in: +To reset back to the original state where no user is logged in you can pass a `null` value to `setUser()`, for example: ```php $this->setUser(null); @@ -115,17 +171,23 @@ Null can be used to set current user back to not-logged-in: ### Creating course categories ```php - $category1 = $this->getDataGenerator()->create_category(); - $category2 = $this->getDataGenerator()->create_category(array('name'=>'Some subcategory', 'parent'=>$category1->id)); +$category1 = $this->getDataGenerator()->create_category(); +$category2 = $this->getDataGenerator()->create_category([ + 'name' => 'Some subcategory', + 'parent' => $category1->id, +]); ``` ### Creating courses ```php - $course1 = $this->getDataGenerator()->create_course(); +$course1 = $this->getDataGenerator()->create_course(); - $category = $this->getDataGenerator()->create_category(); - $course2 = $this->getDataGenerator()->create_course(array('name'=>'Some course', 'category'=>$category->id)); +$category = $this->getDataGenerator()->create_category(); +$course2 = $this->getDataGenerator()->create_course([ + 'name' => 'Some course', + 'category' => $category->id, +]); ``` ### Creating activities @@ -135,64 +197,79 @@ Some activity plugins include instance generators. The generator class are defin Example of creation of new course with one page resource: ```php - $course = $this->getDataGenerator()->create_course(); - $generator = $this->getDataGenerator()->get_plugin_generator('mod_page'); - $generator->create_instance(array('course'=>$course->id)); +$course = $this->getDataGenerator()->create_course(); +$generator = $this->getDataGenerator()->get_plugin_generator('mod_page'); +$generator->create_instance([ + 'course' => $course->id, +]); ``` The following is functionally the same, but a bit shorter: ```php - $course = $this->getDataGenerator()->create_course(); - $page = $this->getDataGenerator()->create_module('page', array('course' => $course->id)); +$course = $this->getDataGenerator()->create_course(); +$page = $this->getDataGenerator()->create_module('page', ['course' => $course->id]); ``` ### Creating cohorts -{{Moodle 2.4}} -Since 2.4 there the data generator supports creation of new cohorts. - ```php - $cohort = $this->getDataGenerator()->create_cohort(); +$cohort = $this->getDataGenerator()->create_cohort(); ``` ### Simplified user enrolments -{{Moodle 2.4}} Instead of standard enrolment API it is possible to use simplified method in data generator. It is intended to be used with self and manual enrolment plugins. ```php $this->getDataGenerator()->enrol_user($userid, $courseid); $this->getDataGenerator()->enrol_user($userid, $courseid, $teacherroleid); -$this->getDataGenerator()->enrol_user($userid, $courseid, $teacherroleid, 'manual'); +$this->getDataGenerator()->enrol_user( + $userid, + $courseid, + $teacherroleid, + 'manual', +); ``` ### Creating scales ```php $this->getDataGenerator()->create_scale(); -$this->getDataGenerator()->create_scale(array('name' => $name, 'scale' => $scale, 'courseid' => $courseid, 'userid' => $userid, 'description' => description, 'descriptionformat' => $descriptionformat)); +$this->getDataGenerator()->create_scale([ + 'name' => $name, + 'scale' => $scale, + 'courseid' => $courseid, + 'userid' => $userid, + 'description' => description, + 'descriptionformat' => $descriptionformat, +]); ``` ### Creating roles ```php $this->getDataGenerator()->create_role(); -$this->getDataGenerator()->create_role(array('shortname' => $shortname, 'name' => $name, 'description' => description, 'archetype' => $archetype)); +$this->getDataGenerator()->create_role([ + 'shortname' => $shortname, + 'name' => $name, + 'description' => description, + 'archetype' => $archetype, +]); ``` ### Creating tags ```php $this->getDataGenerator()->create_tag(); -$this->getDataGenerator()->create_tag(array( +$this->getDataGenerator()->create_tag([ 'userid' => $userid, 'rawname' => $rawname, 'name' => $name, 'description' => $description, 'descriptionformat' => $descriptionformat, 'flag' => $flag -)); +]); ``` ### Groups @@ -200,36 +277,56 @@ $this->getDataGenerator()->create_tag(array( #### Creating groups ```php -$this->getDataGenerator()->create_group(array('courseid' => $courseid)); -$this->getDataGenerator()->create_group(array('courseid' => $courseid, 'name' => $name, 'description' => $description, 'descriptionformat' => $descriptionformat)); +$this->getDataGenerator()->create_group(['courseid' => $courseid]); +$this->getDataGenerator()->create_group([ + 'courseid' => $courseid, + 'name' => $name, + 'description' => $description, + 'descriptionformat' => $descriptionformat, +]); ``` #### Adding users to groups ```php -$this->getDataGenerator()->create_group_member(array('userid' => $userid, 'groupid' => $groupid)); -$this->getDataGenerator()->create_group_member(array('userid' => $userid, 'groupid' => $groupid, 'component' => $component, 'itemid' => $itemid)); +$this->getDataGenerator()->create_group_member([ + 'userid' => $userid, + 'groupid' => $groupid, +]); +$this->getDataGenerator()->create_group_member([ + 'userid' => $userid, + 'groupid' => $groupid, + 'component' => $component, + 'itemid' => $itemid, +]); ``` #### Creating groupings ```php -$this->getDataGenerator()->create_grouping(array('courseid' => $courseid)); -$this->getDataGenerator()->create_grouping(array('courseid' => $courseid, 'name' => $name, 'description' => $description, 'descriptionformat' => $descriptionformat)); +$this->getDataGenerator()->create_grouping(['courseid' => $courseid]); +$this->getDataGenerator()->create_grouping([ + 'courseid' => $courseid, + 'name' => $name, + 'description' => $description, + 'descriptionformat' => $descriptionformat, +]); ``` #### Adding groups to groupings ```php -$this->getDataGenerator()->create_grouping_group(array('groupingid' => $groupingid, 'groupid' => $groupid)); +$this->getDataGenerator()->create_grouping_group([ + 'groupingid' => $groupingid, + 'groupid' => $groupid, +]); ``` ### Repositories -#### Creating repository instances +Some repository plugins include instance generators. Repositories which define a generator will have a generator defined in `path/to/repository/tests/generator/lib.php`. -{{Moodle 2.5}} -Some respository plugins include instance generators. The generator class are defined in plugindirectory/tests/generator/lib.php.. +#### Creating repository instances ```php $this->getDataGenerator()->create_repository($type, $record, $options); @@ -237,9 +334,6 @@ $this->getDataGenerator()->create_repository($type, $record, $options); #### Creating repository types -{{Moodle 2.5}} -Some respository plugins include type generators. The generator class are defined in plugindirectory/tests/generator/lib.php.. - ```php $this->getDataGenerator()->create_repository_type($type, $record, $options); ``` @@ -249,85 +343,125 @@ $this->getDataGenerator()->create_repository_type($type, $record, $options); #### Grade categories ```php -$this->getDataGenerator()->create_grade_category(array('courseid' => $courseid)); -$this->getDataGenerator()->create_grade_category(array('courseid' => $courseid, 'fullname' => $fullname)); +$this->getDataGenerator()->create_grade_category(['courseid' => $courseid]); +$this->getDataGenerator()->create_grade_category([ + 'courseid' => $courseid, + 'fullname' => $fullname, +]); ``` #### Grade items ```php $this->getDataGenerator()->create_grade_item(); -$this->getDataGenerator()->create_grade_item(array('itemtype' => $itemtype, 'itemname' => $itemname, 'outcomeid' => $outcomeid, 'scaleid' => $scaleid, 'gradetype' => $gradetype)); +$this->getDataGenerator()->create_grade_item([ + 'itemtype' => $itemtype, + 'itemname' => $itemname, + 'outcomeid' => $outcomeid, + 'scaleid' => $scaleid, + 'gradetype' => $gradetype, +]); ``` #### Outcomes ```php $this->getDataGenerator()->create_grade_outcome(); -$this->getDataGenerator()->create_grade_item(array('fullname' => $fullname)); +$this->getDataGenerator()->create_grade_item(['fullname' => $fullname]); ``` ### Other types of plugin -{{Moodle 2.5}} -Any other type of plugin can have a generator. The generator class should extend component_generator_base, and then you can get an instance using $mygenerator = $this->getDataGenerator()->get_plugin_generator($frankenstylecomponentname); +Any other type of plugin may define their own generator. + +The generator class **must** extend the `\component_generator_base` class, after which it can be fetched using: -For some types of plugin, like mod documented above, there may be a more specific class than component_generator_base to extend, like testing_module_generator. That will give a consistent set of method names to use. Otherwise, you can create whatever methods you like on your generator, to create the different things you need to work whith. +```php +$mygenerator = $this->getDataGenerator()->get_plugin_generator("my_componentname"); +``` + +:::note Alternatives to `\component_generator_base` + +Some plugin types, such as Activity modules (`mod_` prefix) may be a more specific class to extend, for example `testing_module_generator`. + +This should be used instead to give a consistent set of method names to use. + +Otherwise, you can create whatever methods you like on your generator, to create the different things you need to work whith. + +::: ## Long tests -All standard test should execute as fast as possible. Tests that take a longer time to execute (>10s) or are otherwise expensive (such as querying external servers that might be flooded by all dev machines) should be execute only when PHPUNIT_LONGTEST is true. This constant can be set in phpunit.xml or directly in config.php. +All standard test should execute as fast as possible. Tests that take a longer time to execute (>10s) or are otherwise expensive (such as querying external servers that might be flooded by all dev machines) should be execute only when running long tests is enabled. ## Large test data -See advanced_testcase::createXMLDataSet() and advanced_testcase::createCsvDataSet() and related functions there for easier ways to manage large test data sets within files rather than arrays in code. See [PHPUnit integration#Extra methods](https://docs.moodle.org/dev/PHPUnit_integration#Extra_methods) +See `\advanced_testcase::createXMLDataSet()` and `\advanced_testcase::createCsvDataSet()` and related functions there for easier ways to manage large test data sets within files rather than arrays in code. See [PHPUnit integration#Extra methods](https://docs.moodle.org/dev/PHPUnit_integration#Extra_methods) ## Testing sending of messages -{{Moodle 2.4}} -You can temporarily redirect all messages sent via message_send() to a message sink object. This allows developers to verify that the tested code is sending expected messages. +You can temporarily redirect all messages sent via `message_send()` to a message sink object. This allows developers to verify that the tested code is sending expected messages. + +:::note + +In several database implementations Moodle makes use of Database transactions during unit tests. + +Because message sending is deferred until after the transaction has closed, you must disable the test-in-transaction for messaging to work. -To test code using messaging first disable the use of transactions and then redirect the messaging into a new message sink, you can inspect the results later. +::: + +To test code using messaging first disable the use of transactions, and then redirect the messaging into a new message sink. You can inspect the results using the message sink: ```php $this->preventResetByRollback(); $sink = $this->redirectMessages(); + //... code that is sending messages + $messages = $sink->get_messages(); $this->assertEquals(3, count($messages)); //.. test messages were generated in correct order with appropriate content ``` -{{Moodle 4.4}} -Since **4.4** there are two new methods that support getting the messages for specific **components** and **message types**. + -```php +Since Moodle 4.4 there are two new methods that support getting the messages for specific **components** and **message types**. + +```php title="Fetching all messages sent for a component" $sink = $this->redirectMessages(); + //... code that is sending messages + $messages = $sink->get_messages_by_component('mod_forum'); $this->assertEquals(3, count($messages)); //.. test messages were generated in correct order with appropriate content ``` -```php +```php title="Fetching all messages sent for a specific message type" $sink = $this->redirectMessages(); + //... code that is sending messages -$messages = $sink->get_messages_by_component_and_type('core', 'messagecontactrequests'); + +$messages = $sink->get_messages_by_component_and_type( + 'core', + 'messagecontactrequests', +); $this->assertEquals(3, count($messages)); //.. test messages were generated in correct order with appropriate content ``` ## Testing sending of emails -{{Moodle 2.6}} -You can temporarily redirect emails sent via email_to_user() to a email message sink object. This allows developers to verify that the tested code is sending expected emails. +You can temporarily redirect emails sent via `email_to_user()` to a email message sink object. This allows developers to verify that the tested code is sending expected emails. To test code using messaging first unset 'noemailever' setting and then redirect the emails into a new message sink where you can inspect the results later. ```php unset_config('noemailever'); $sink = $this->redirectEmails(); + //... code that is sending email + $messages = $sink->get_messages(); $this->assertEquals(1, count($messages)); ``` @@ -345,63 +479,65 @@ get_log_manager(true); ## Check your coverage -{{Moodle 3.7}} PHPUnit has the ability to generate code coverage information for your unit tests. Prior to Moodle 3.7, this coverage would load all files and generate coverage for everything regardless of whether that file could be covered at all, or whether it was intentionally covered. -Since Moodle 3.7 the **phpunit.xml** configuration contains generated coverage include and exclude information for each component. +Since Moodle 3.7 the `phpunit.xml` configuration contains generated coverage include and exclude information for each component. ### Generating include and exclude configuration -{{Moodle 3.11}} -You can programatically describe which files will be checked for coverage by creating a coverage.php file alongside the tests that you are writing. + + +You can programatically describe which files will be checked for coverage by creating a `coverage.php` file alongside the tests that you are writing. -Since Moodle 4.0, a default configuration is applied for all plugins and it is not necessary to supply a coverage.php unless you wish to cover additional files. + -The coverage.php file allows you to list include and exclude files and folders within the component being -tested. All paths specified are relative to the component being tested. For example, when working with **mod_forum** your -code will be in **mod/forum**, and its unit tests will be in **mod/forum/tests/example_test.php**. The coverage file -for this would be in **mod/forum/tests/coverage.php** and all paths specified would be relative to **mod/forum**. +From Moodle 4.0, a default configuration is applied for all plugins and it is not necessary to supply a `coverage.php` unless you wish to cover additional files. -It is possible to specify a combination of included files, included folders, excluded files, and excluded folders. This would allow you, for example, to include the entire **classes** directory, but exclude.a specific file or folder within it. +The `coverage.php` file allows you to list include and exclude files and folders within the component being tested. -The following is an example coverage.php file from **mod_forum**: +All paths specified are relative to the component being tested. +For example, when working with `mod_forum` your code will be in `mod/forum`, and its unit tests will be in `mod/forum/tests/`. +The coverage file for this would be in `mod/forum/tests/coverage.php` and all paths specified would be relative to `mod/forum` -Note: For Moodle versions 3.7 to 3.10, the [syntax used](https://docs.moodle.org/dev/index.php?title=Writing_PHPUnit_tests&oldid=58177#Check_your_coverage) was slightly different. +It is possible to specify a combination of included files, included folders, excluded files, and excluded folders. +This would allow you, for example, to include the entire `classes` directory, but exclude a specific file or folder within it. + +The following is an example `coverage.php` file from `mod_forum`: ```php return new class extends phpunit_coverage_info { - /** @var array The list of folders relative to the plugin root to include in coverage generation. */ + /** @var array The list of folders relative to the plugin root to include */ protected $includelistfolders = [ 'classes', 'externallib.php', ]; - /** @var array The list of files relative to the plugin root to include in coverage generation. */ + /** @var array The list of files relative to the plugin root to include */ protected $includelistfiles = []; - /** @var array The list of folders relative to the plugin root to exclude from coverage generation. */ + /** @var array The list of folders relative to the plugin root to exclude */ protected $excludelistfolders = []; - /** @var array The list of files relative to the plugin root to exclude from coverage generation. */ + /** @var array The list of files relative to the plugin root to exclude */ protected $excludelistfiles = []; }; ``` -Also, note that you can better define which class or function each test is effectively covering by using the @covers annotation as [described in the documention](https://phpunit.readthedocs.io/en/9.5/code-coverage-analysis.html#specifying-covered-code-parts). + -Since Moodle 4.0, the following default configuration is applied:{{Moodle 4.0}} +Since Moodle 4.0, the following default configuration is applied: ```php return new class extends phpunit_coverage_info { - /** @var array The list of folders relative to the plugin root to include in coverage generation. */ + /** @var array The list of folders relative to the plugin root to include */ protected $includelistfolders = [ 'classes', 'tests/generator', ]; - /** @var array The list of files relative to the plugin root to include in coverage generation. */ + /** @var array The list of files relative to the plugin root to include */ protected $includelistfiles = [ 'externallib.php', 'lib.php', @@ -410,31 +546,117 @@ return new class extends phpunit_coverage_info { 'rsslib.php', ]; - /** @var array The list of folders relative to the plugin root to exclude from coverage generation. */ + /** @var array The list of folders relative to the plugin root to exclude */ protected $excludelistfolders = []; - /** @var array The list of files relative to the plugin root to exclude from coverage generation. */ + /** @var array The list of files relative to the plugin root to exclude */ protected $excludelistfiles = []; }; ``` -If a coverage.php file already exists, then the defaults will be added to the values already defined. +If a `coverage.php` file already exists, then the defaults will be added to the values already defined. + +### Defining covered code + +In addition to defining the files which should be considered in generating coverage reports, each test class **must** define what functions or classes it is _intended_ to cover. + +For PHPUnit 9.6 and earlier this can be done using the [`@covers` annotation and related annotations](https://phpunit.readthedocs.io/en/9.6/code-coverage-analysis.html#specifying-covered-code-parts). + +From Moodle 5.0 and PHPUnit 10 onwards, this can alternatively be done using PHP _Attributes_. + +### Guidance on defining Coverage + +The following guidance is based on recommendation from PHPUnit documentation. + +- Coverage should be defined for an entire class rather than individual methods wherever possible +- Coverage should be defined at the class level, not for individual test methods + +:::note Attributes + +From PHPUnit 10, with the introduction of Attributes for coverage, the defined attributes are much more robust. + +However, if you are writing a plugin and wish to test it on Moodle 4.5 or earlier _and_ Moodle 5.0 or later then you cannot use Attributes for testing. + +::: + +#### Common issues + +#### Using the `@covers ::methodName` syntax + +When defining coverage for a specific method under test, it must be either: + +- defined with the class it relates to; or +- the `@coversDefaultClass` annotation must be used on the test class. + +```php +/** + * @coversDefaultClass \mod_myplugin\example + */ +class example_test extends \advanced_testcase { + /** + * This test will cover \mod_myplugin\example::some_method + * @covers ::some_method + */ + public function test_some_method(): void {} +} +``` + +#### Qualification of classes + +When using the `@covers` annotations, all class names **must** be fully-qualified, for example: + +```php +namespace mod_myplugin; + +/** + * Correct coverage definition: + * @covers \mod_myplugin\example + * + * Incorrect coverage definition: + * @covers example + */ +``` + +When using Attributes the standard rules of PHP namespace resolution apply: + +```php + +namespace mod_myplugin; + +#[\PHPUnit\Framework\Attributes\CoversClass(example::class)] +``` ## Best practice There are several best practices, suggestions, and things to avoid which you should consider when writing unit tests. Some of these are described below. -### Code coverage +### Using the magic `::class` constant + +PHP supports the use of a magic `::class` constant to correctly and consistently define class names. This can be used in a range of situations, including: -PHPUnit has the ability to generate code coverage information for your unit tests and this is well supported since.Moodle 3.7. We recommend that you consider checking the coverage of your plugins when you write your code. +- `$this->assertInstanceOf(\some\example::class, new \some\example());` +- `#[\PHPUnit\Framework\Attributes\CoversClass(\some\example::class)]` -We recommend that you explicitly set the @covers annotation as described in the [PHPUnit documentation](https://phpunit.readthedocs.io/en/9.5/annotations.html#appendixes-annotations-covers-tables-annotations). +One of the benefits of using the constant is that it supports class aliasing at runtime, meaning that a plugin's codebase can support multiple versions of Moodle where class aliasing has been used to move a class to a new location. -### Keep use of resetAfterTest to a minimum +:::note Using on classes which do not exist + +The class **does not** need to exist. You can use this constant on any arbitrary namespace. + +::: + +### Code coverage + +PHPUnit has the ability to generate code coverage information for your unit tests and this is well supported since Moodle 3.7. +We _strongly_ recommend that you consider checking the coverage of your plugins when you write your code. + +### Keep use of `resetAfterTest` to a minimum Although many of the examples described above use the `resetAfterTest` nomenclature to reset the database and filesystem after your test completes, you should ideally not use this unless you have to. + Generally speaking you should aim to write code which is mockable, and does not require real fixtures. -Use of resetAfterTest will also slow your tests down. + +Use of `resetAfterTest` will also slow your tests down. ### Be careful with shared setUp and instance variables @@ -470,34 +692,31 @@ public function test_foobar(int $foo, int $bar) { * * @return array List of data sets - (string) data set name => (array) data */ -public function foobar_provider(): array { - return [ - 'Same numbers' => [ - 'foo' => 42, - 'bar' => 42, - ], - 'Different numbers' => [ - 'foo' => 21, - 'bar' => 84, - ], +public function foobar_provider(): \Generator { + yield 'Same numbers' => [ + 'foo' => 42, + 'bar' => 42, + ]; + yield 'Different numbers' => [ + 'foo' => 21, + 'bar' => 84, ]; } ``` ## Extra test settings -Usually the test should not interact with any external systems and it should work the same on all systems. But sometimes you need to specify some option for connection to external systems or system configuration. It is intentionally not possible to use $CFG settings from config.php. +Usually the test should not interact with any external systems and it should work the same on all systems. But sometimes you need to specify some option for connection to external systems or system configuration. It is intentionally not possible to use `$CFG` settings from `config.php`. There are several ways how to inject your custom settings: -- define test setting constants in your phpunit.xml file -- define test setting constants in your config.php +- define test setting constants in your `phpunit.xml` file +- define test setting constants in your `config.php` + These constants may be then used in your test or plugin code. ## Upgrading unit tests to work with Moodle 4.4 and up (PHPUnit 9.6) -{{Moodle 4.4}} - With Moodle 4.4, **PHPUnit was upgraded to 9.6** (from 9.5 being used in previous versions). This was done to **warn to developers in advance** about functionality that has been deprecated in the 9.5 series and will be removed so will stop working with the next major update to PHPUnit 10.x (see [MDL-81266](https://moodle.atlassian.net/browse/MDL-81266) and linked issues for more details). While everything should continue working without modification with PHPUnit 9.6, you will may get a **good number of deprecation warnings** ("W" in the tests output) that **should be analysed**, replacing or removing them as soon as possible, because all those warnings will become errors with next PHPUnit upgrade. @@ -505,102 +724,28 @@ While everything should continue working without modification with PHPUnit 9.6, A good summary of all the **changes and replacements to perform** is available in the [lib/upgrade.txt](https://github.com/stronk7/moodle/commit/b2131ceff74da4c23928936f238d676a08e07d7f) file. With main points being: - [MDL-81281](https://moodle.atlassian.net/browse/MDL-81281). A number of attribute-related assertions have been deprecated, will be removed with PHPUnit 10. Alternatives for **some** of them are available: - - assertClassHasAttribute() - - assertClassNotHasAttribute() - - assertClassHasStaticAttribute() - - assertClassNotHasStaticAttribute() - - assertObjectHasAttribute() => assertObjectHasProperty() - - assertObjectNotHasAttribute() => assertObjectNotHasProperty() + - `assertClassHasAttribute()` + - `assertClassNotHasAttribute()` + - `assertClassHasStaticAttribute()` + - `assertClassNotHasStaticAttribute()` + - `assertObjectHasAttribute()` => `assertObjectHasProperty()` + - `assertObjectNotHasAttribute()` => `assertObjectNotHasProperty()` - [MDL-81266](https://moodle.atlassian.net/browse/MDL-81266). A number of deprecation, notice, warning and error expectations have been deprecated, will be removed with PHPUnit 10. No alternative exists. A working replacement is available in the linked issue, hopefully there aren't many cases. - - expectDeprecation() - - expectDeprecationMessage() - - expectDeprecationMessageMatches() - - expectError() - - expectErrorMessage() - - expectErrorMessageMatches() - - expectNotice() - - expectNoticeMessage() - - expectNoticeMessageMatches() - - expectWarning() -- expectWarningMessage() -- expectWarningMessageMatches() -- [MDL-81308](https://moodle.atlassian.net/browse/MDL-81308). The ->withConsecutive() functionality on PHPUnit mocks has been **silently** deprecated, will be removed with PHPUnit 10. Note that this won't affect PHPUnit 9.6 runs and an alternative path will be proposed in the linked issue, part of the PHPUnit 10 epic. -- PHPUnit\Framework\TestCase::getMockClass() has been deprecated, will be removed with PHPUnit 10. No clear alternative exists and won't be investigated, because there aren't cases in core. -- Cannot use the Test suffix on abstract test case classes. Proceed to rename them to end with TestCase instead. - -## Upgrading unit tests to work with Moodle 3.11 and up (PHPUnit 9.5) - -{{Moodle 3.11}} - -With Moodle 3.11, **PHPUnit was upgraded to 9.5** (from 8.5 being used in previous versions). This was done to **better align** the testing environment with PHP versions supported by Moodle 3.11 (7.3, 7.4 and [8.0](/general/development/policies/php)) (see [MDL-71036](https://moodle.atlassian.net/browse/MDL-71036) and linked issues for more details). - -While a lot of existing tests will work without modification with PHPUnit 9.5, you will get a **good number of deprecation warnings** ("W" in the tests output) that **should be replaced by their new counterparts as soon as possible**, because all those warnings will become errors with next PHPUnit upgrade. - -To find more information about the changes coming with PHPUnit 9.5, it's recommended to read the following resources: - -- [A good article](https://thephp.cc/news/2020/02/migrating-to-phpunit-9) explaining all the main changes in the release. -- [PHPUnit 9 release Announcement](https://phpunit.de/announcements/phpunit-9.html). -- These multiple detailed changelogs (because all them have included modifications requiring changes): [9.0](https://github.com/sebastianbergmann/phpunit/blob/9.0.0/ChangeLog-9.0.md), [9.1](https://github.com/sebastianbergmann/phpunit/blob/9.1.0/ChangeLog-9.1.md), [9.3](https://github.com/sebastianbergmann/phpunit/blob/9.3.0/ChangeLog-9.3.md), [9.5](https://github.com/sebastianbergmann/phpunit/blob/9.5.0/ChangeLog-9.5.md) -- [Official PHPUnit manual](https://phpunit.readthedocs.io/en/9.5/). -A good summary of all the **changes and replacements to perform** is available in the [lib/upgrade.txt](https://github.com/moodle/moodle/blob/e3a46964dc6d8ca1558c6e1e8dfdf3c1745eeaed/lib/upgrade.txt#L5-L65) file. With main points being: -- All the changes that were deprecated with PHPUnit 8.5 (see the section below) are now removed and will lead to errors. -- assertContains() now performs stricter comparison (like assertSame() does). New assertContainsEquals() has been created to provide the old behavior. -- Changes to the phpunit.xml schema, mostly internal. These only will impact if you are using custom phpunit.xml files: - - The previous section is now (better) called . And, within it: - - has been replaced by . - - is not a child of anymore, but of . - - But with implications when defining the [coverage information](https://docs.moodle.org/dev/#Check_your_coverage) because $whitelistxxx properties used by the coverage.php files have been deprecated, instead use includelistfolders and includelistfiles (to better map the elements in the xml). -- Warning: It's not possible to run individual test files any more. Use any of the alternative execution methods (filter, suite, config) to specify which tests you want to run. This will be hopefully fixed in [MDL-71049](https://moodle.atlassian.net/browse/MDL-71049) once it has been agreed which the best way to proceed is. -- Deprecations, deprecations, deprecations. Lots of them in often used assertions: file assertions, regexp assertions, exception expectations... again, note that all them will become errors with the next PHPUnit update, so **the recommendation is to update them ASAP**. -Finally, it's also a good idea to [browse the changes associated with the issue](https://github.com/moodle/moodle/compare/fc335f5...713722c), hopefully with useful explanations in the commit messages. - -## Upgrading unit tests to work with Moodle 3.10 and up (PHPUnit 8.5) - -{{Moodle 3.10}} - -With Moodle 3.10, **PHPUnit was upgraded to 8.5** (from 7.5 being used in older version). This was done to **better align** the testing environment with PHP versions supported by Moodle 3.10 (7.2, 7.3 and 7.4) and also to provide an **easier jump to PHPUnit 9.x** that will be needed for Moodle 3.11 (php versions 7.3, 7.4 and 8.0), removing a lot of deprecated stuff in advance. (see [MDL-67673](https://moodle.atlassian.net/browse/MDL-67673) and [MDL-64600](https://moodle.atlassian.net/browse/MDL-64600) for more details). - -While 99% of existing tests will work without modification with PHPUnit 8.5, you will get a **good number of deprecation warnings** ("W" in the tests output) that **should be replaced by their new counterparts as soon as possible**, because all those warnings will become errors with next PHPUnit upgrade. - -To find more information about the changes coming with PHPUnit 8.5, it's recommended to read the following resources: - -- [A good article](https://thephp.cc/news/2019/02/help-my-tests-stopped-working) explaining all the main changes in the release. -- [PHPUnit 8 release Announcement](https://phpunit.de/announcements/phpunit-8.html). -- [Detailed changelog of the release](https://github.com/sebastianbergmann/phpunit/blob/130104cf796a88dd1547dc5beb8bd555c2deb55e/ChangeLog-8.0.md). -- [Official PHPUnit manual](https://phpunit.readthedocs.io/en/8.5/). -- A good summary of all the **changes and replacements to perform** is available in the [lib/upgrade.txt](https://github.com/moodle/moodle/blob/6594c54b2eef62499d304bfa0939999e3a14246e/lib/upgrade.txt#L5-L37) file. With main points being: - - Support for PHP 7.0 dropped (because all template methods (setUp(), tearDown()..) now require to return void. This will mostly impact 3rd-part plugins that were still running the same tests against old branches of Moodle with PHP 7.0 support. - - PHPUnit/DBUnit has been removed are replaced by a lightweight alternative. - - Deprecations, deprecations, deprecations. Lots of them in often used assertions (assertContains(), assertEquals()...) and also @expectedExceptionXXX annotations. Again, note that all them will become errors with PHPUnit 9. -- Finally, it's also a good idea to [browse the changes associated with the issue](https://github.com/moodle/moodle/compare/5903054...b13ec3c), hopefully with useful enough explanations in the commit messages. - -## Upgrading unit tests to work with Moodle 3.7 and up (PHPUnit 7.5) - -{{Moodle 3.7}} - -With Moodle 3.7, **PHPUnit was upgraded to 7.5** (from 6.x being used in older version). This was done to better align the testing environment with PHP versions supported by Moodle 3.7 (7.1, 7.2 and 7.3). (see [MDL-65204](https://moodle.atlassian.net/browse/MDL-65204) and linked issues for more details). While internally [a lot of things changed with PHPUnit 7](https://phpunit.de/announcements/phpunit-7.html) (PHP 7.1-isms, typed signatures, void returns, assertEquals() changes), thanks to our **wrapping layer** (basic and advanced testcases...) impact expected into old existing unit tests is expected to be reduced and upgrades, easy to achieve. - -To find more information about the changes coming with PHPUnit 7, it's recommended to read the following resources: - -- [PHPUnit 7 release Announcement](https://phpunit.de/announcements/phpunit-7.html). -- [Detailed changelog of the release](https://github.com/sebastianbergmann/phpunit/blob/520723129e2b3fc1dc4c0953e43c9d40e1ecb352/ChangeLog-7.5.md), paying special attention to the added, deprecated, changed and deleted sections. -- [Official PHPUnit manual](https://phpunit.readthedocs.io/en/7.5/). -- Changes performed into core, mainly: new, stricter, signatures ([26218b7](https://github.com/moodle/moodle/commit/26218b7)) and assertEquals() changes, specially important when comparing strings, now performed using strict (===) equals ([85f47ba](https://github.com/moodle/moodle/commit/85f47ba)). - -## Upgrading unit tests to work with Moodle 3.4 and up (PHPUnit 6) - -{{Moodle 3.4}} - -With Moodle 3.4, **PHPUnit was upgraded to 6.4** (from 5.5 being used in older version). This was done to better align the testing environment with PHP versions supported by Moodle 3.4 (7.0, 7.1 and 7.2). (see [MDL-60611](https://moodle.atlassian.net/browse/MDL-60611) and linked issues for more details). While internally [a lot of things changed with PHPUnit 6](https://github.com/sebastianbergmann/phpunit/wiki/Release-Announcement-for-PHPUnit-6.0.0#backwards-compatibility-issues) (namespaced classes being the more noticeable), thanks to our **wrapping layer** (basic and advanced testcases...) impact expected into old existing unit tests is expected to be reduced and upgrades, easy to achieve. - -Still, in **some cases, it will impossible to maintain compatibility** of tests between old (pre 3.4) tests and new ones, especially **when direct use of any phpunit class is performed**. Luckily, both GitHub Actions and CI tests will detect this situation and it shouldn't be hard to keep all supported branches in core passing ok. Plugins may be trickier, if the same branch is attempting to work against multiple core branches and they are using some phpunit class directly. - -To find more information about the changes coming with PHPUnit 6, it's recommended to read the following resources: - -- [A very good mini-guide](https://thephp.cc/news/2017/02/migrating-to-phpunit-6) showing all the important changes. -- [PHPUnit 6 release Announcement](https://github.com/sebastianbergmann/phpunit/wiki/Release-Announcement-for-PHPUnit-6.0.0#backwards-compatibility-issues). -- [Detailed changelog of the release](https://github.com/sebastianbergmann/phpunit/blob/9d0c024d2099531442d862b66b0ad7cf35ed8e78/ChangeLog-6.0.md), paying special attention to the changed and deleted sections. -- Changes performed into core, mainly: namespace class renaming ([801a372](https://github.com/moodle/moodle/commit/801a372dadb6e11c8781547603e3f0a59ce5638f)) and deprecated stuff ([796e48a](https://github.com/moodle/moodle/commit/796e48a58bf18533bdca423fff7949ab119101c4)) + - `expectDeprecation()` + - `expectDeprecationMessage()` + - `expectDeprecationMessageMatches()` + - `expectError()` + - `expectErrorMessage()` + - `expectErrorMessageMatches()` + - `expectNotice()` + - `expectNoticeMessage()` + - `expectNoticeMessageMatches()` + - `expectWarning()` + - `expectWarningMessage()` + - `expectWarningMessageMatches()` + - [MDL-81308](https://moodle.atlassian.net/browse/MDL-81308). The `->withConsecutive()` functionality on PHPUnit mocks has been deprecated, and will be removed with PHPUnit 10. + - `\PHPUnit\Framework\TestCase::getMockClass()` has been deprecated, will be removed with PHPUnit 10. No clear alternative exists and won't be investigated, because there aren't cases in core. + - You **MUST NOT** use the `_test` suffix on abstract test case classes. Test case classes **MUST** be named with the `__testcase` suffix. ## See also diff --git a/general/development/policies/codingstyle/index.md b/general/development/policies/codingstyle/index.md index b341068c67..76c206ff89 100644 --- a/general/development/policies/codingstyle/index.md +++ b/general/development/policies/codingstyle/index.md @@ -637,7 +637,7 @@ There are no rules limiting what can be used as a level 3 namespace. This is where a plugin or addon can make extensive use of namespaces with no chance of conflict with any other plugin or api, now and forever onwards. -#### Namespaces within `**/tests` directories +#### Namespaces within `**/tests` directories(#namespaces-within-tests) :::info