diff --git a/docs/testing/automated/concepts.md b/docs/testing/automated/concepts.md new file mode 100644 index 00000000..2bebb213 --- /dev/null +++ b/docs/testing/automated/concepts.md @@ -0,0 +1,16 @@ +--- +sidebar_position: 1 +--- + +Concepts +============= + +For Joomla! we are using different strategies to test the application. + +# System Testing versus Unit Testing + +System Testing and Unit Testing are complementary test strategies. In Joomla!, Unit Testing is mainly used to test the framework classes (for example, the classes in libraries/joomla). Unit tests test that an individual method (also known as function) in a class does what it is supposed to do. For example, a unit test is used to test that the methods in the JString class work as expected. Unit testing provides confidence that the framework classes work as expected and allows you to refactor these classes (improve the code without changing the functionality) and still have confidence that they still work correctly. + +System tests test that the application works correctly from the user point of view. For example, a system test can test something simple, for example, that you can create a new menu item for a single article and show the menu item on the site. Or a system test can test something more detailed, for example that the parameters for a module all work as expected. + +Designing and creating system tests requires that you know how to use the application. It does not require that you understand how the program is written. Application knowledge is more important than programming knowledge, so system tests can be designed and written by people with less technical knowledge of PHP or the Joomla! framework. diff --git a/docs/testing/automated/index.md b/docs/testing/automated/index.md new file mode 100644 index 00000000..b03defa0 --- /dev/null +++ b/docs/testing/automated/index.md @@ -0,0 +1,8 @@ +--- +sidebar_position: 1 +--- + +Automated Testing +============= + +Here you will find information about our automated testing, how to write tests and how to set it up. \ No newline at end of file diff --git a/docs/testing/automated/system/assets/cypress-window1.jpg b/docs/testing/automated/system/assets/cypress-window1.jpg new file mode 100644 index 00000000..9f6d1a57 Binary files /dev/null and b/docs/testing/automated/system/assets/cypress-window1.jpg differ diff --git a/docs/testing/automated/system/assets/cypress-window2.jpg b/docs/testing/automated/system/assets/cypress-window2.jpg new file mode 100644 index 00000000..dc7406f4 Binary files /dev/null and b/docs/testing/automated/system/assets/cypress-window2.jpg differ diff --git a/docs/testing/automated/system/index.md b/docs/testing/automated/system/index.md new file mode 100644 index 00000000..7dda1533 --- /dev/null +++ b/docs/testing/automated/system/index.md @@ -0,0 +1,9 @@ +--- +sidebar_position: 2 +--- + + +System Testing +=============== + +We are using [Cypress](https://docs.cypress.io/guides/overview/why-cypress) for our system (end2end) testing. \ No newline at end of file diff --git a/docs/testing/automated/system/setup.md b/docs/testing/automated/system/setup.md new file mode 100644 index 00000000..421c1fcf --- /dev/null +++ b/docs/testing/automated/system/setup.md @@ -0,0 +1,82 @@ +--- +sidebar_position: 1 +--- + + +Setup your testing environment +=============== + +## Prepare your Workstation + +You need a set of tools to have a good testing setup. Tools you should have: + +* git +* node (16.16.0 is the current LTS) +* database (mysql 5.6+, mariaDB 10.1+, postgres 11.0+) +* PHP (good to have different versions and the ability to switch) +* Composer, [Installation instructions here](https://getcomposer.org/doc/00-intro.md#installation-linux-unix-macos) +* Webserver (apache 2.4+, nginx 1.18+) +* Editor (PhpStorm, Visual Studio Code) + +:::note + +This is the same toolset you need for unit testing + +::: + +## MAC OS + +As always there are different ways of installing the listed software. One way is the use of [Valet](https://laravel.com/docs/9.x/valet) in combination with [Homebrew](https://brew.sh/) + +Now as you have all tools installed you can clone the [joomla-cms repository](https://github.com/joomla/joomla-cms). + +1. Open a terminal +2. Go into a directory on you workstation. +3. clone the joomla-cms repository: ```git clone https://github.com/joomla/joomla-cms.git``` another option here is to fork the joomla-cms repo and then clone your fork. We recommend the 2nd way because then you can make changes and Pull Request directly. +4. Go into the joomla-cms directory +5. If you have installed valet, run ```valet link``` +5. Run ```composer install``` +6. Run ```npm ci``` +7. Create a ```cypress.env.json``` file. This file allows to overwrite config setting from ```cypress.config.js``` + + Here is a example ```cypress.env.json``` + + ```json + { + "sitename": "Joomla CMS Test Local", + "name": "jane doe", + "email": "admin@example.com", + "username": "local-admin", + "password": "joomla-17082005", + "db_type": "MySQLi", + "db_host": "localhost", + "db_name": "test_joomla", + "db_user": "root", + "db_password": "password", + "db_prefix": "jos_" + } + ``` + You don't need all settings, just look what you have to change for your local environment compared to ```cypress.config.js``` + +8. Run ```cypress open --e2e --browser=chrome --config baseUrl=http://joomla-cms.test``` + + This will open two windows, one you can ignore and one to run the tests + + + ![Cypress Window 1](./assets/cypress-window1.jpg) + + In the following window you can select test and let them run. You need to install first. + + ![Cypress Window 2](./assets/cypress-window2.jpg) + + +## Windows + + + + +:::caution TODO + +This page is unfinished, please use the **Edit this Page** link at the bottom of this page to help make it more useful. + +::: diff --git a/docs/testing/automated/system/writing-test.md b/docs/testing/automated/system/writing-test.md new file mode 100644 index 00000000..56ffb96e --- /dev/null +++ b/docs/testing/automated/system/writing-test.md @@ -0,0 +1,10 @@ +--- +sidebar_position: 2 +--- + + +Writing Tests +=============== + + + diff --git a/docs/testing/automated/unit/index.md b/docs/testing/automated/unit/index.md new file mode 100644 index 00000000..26471fff --- /dev/null +++ b/docs/testing/automated/unit/index.md @@ -0,0 +1,13 @@ +--- +sidebar_position: 3 +--- + + +Unit Testing +=============== + +:::caution TODO + +This page is unfinished, please use the **Edit this Page** link at the bottom of this page to help make it more useful. + +::: \ No newline at end of file diff --git a/docs/testing/automated/unit/setup.md b/docs/testing/automated/unit/setup.md new file mode 100644 index 00000000..d9b2a1fd --- /dev/null +++ b/docs/testing/automated/unit/setup.md @@ -0,0 +1,78 @@ +--- +sidebar_position: 1 +--- + + +Setup your testing environment +=============== + +## Prepare your Workstation + +You need a set of tools to have a good testing setup. Tools you should have: + +* git +* node (16.16.0 is the current LTS) +* database (mysql 5.6+, mariaDB 10.1+, postgres 11.0+) +* PHP (good to have different versions and the ability to switch) +* Composer, [Installation instructions here](https://getcomposer.org/doc/00-intro.md#installation-linux-unix-macos) +* Webserver (apache 2.4+, nginx 1.18+) - **optional** +* Editor (PhpStorm, Visual Studio Code) + +:::note + +This is pretty much the same toolset you need for system testing + +::: + +## MAC OS + +As always there are different ways of installing the listed software. One way is the use of [Valet](https://laravel.com/docs/9.x/valet) in combination with [Homebrew](https://brew.sh/) + +Now as you have all tools installed you can clone the [joomla-cms repository](https://github.com/joomla/joomla-cms). + +1. Open a terminal +2. Go into a directory on you workstation. +3. clone the joomla-cms repository: ```git clone https://github.com/joomla/joomla-cms.git``` another option here is to fork the joomla-cms repo and then clone your fork. We recommend the 2nd way because then you can make changes and Pull Request directly. +4. Go into the joomla-cms directory +5. **Optional** - If you have installed valet, run ```valet link``` +5. Run ```composer install``` +6. Run ```npm ci``` +7. Copy ```phpunit.xml.dist``` file to ```phpunit.xml```. This file allows config setting for phpunit. + + Here is a example ```phpunit.xml``` + + ```xml + + + + + ./tests/Unit/Libraries + + + ./tests/Integration/Libraries + + + + + + + + + + + ``` + +8. Run ```phpunit --testdox``` + + + +## Windows + + + + +:::caution TODO + +This page is unfinished, please use the **Edit this Page** link at the bottom of this page to help make it more useful. + +::: diff --git a/docs/testing/index.md b/docs/testing/index.md new file mode 100644 index 00000000..cd08f550 --- /dev/null +++ b/docs/testing/index.md @@ -0,0 +1,17 @@ +--- +sidebar_position: 10 +--- + +Testing +============= + +# Overview + +Testing Software is an important part of software development. For Joomla! we have different levels of testing: + +## Automated Testing + +For the automated testing we are using a continus integration (CI) server drone. Any change that is made runs a series of tests on the CI system. We test if the code style for PHP, CSS and javascript is correct, run unit test for the supported PHP versions and run end to end tests. All this not only runs on different PHP version we are also testing different database version. At the end of the test we create an installable package with the changes included to support our manually testing. If something fails we save information about the reason. + +## Manually Testing +While automated testing is more focused on making sure that a change doesn't break existing functionality, is manually testing focused on the change itself. Always tow people have to confirm that a change does for what is made. This can be a bugfix or new functionality. diff --git a/docs/testing/manually/index.md b/docs/testing/manually/index.md new file mode 100644 index 00000000..77bbe5c9 --- /dev/null +++ b/docs/testing/manually/index.md @@ -0,0 +1,12 @@ +--- +sidebar_position: 2 +--- + +Manually Testing +============= + +:::caution TODO + +This page is unfinished, please use the **Edit this Page** link at the bottom of this page to help make it more useful. + +:::