Testing Plan and Continuous Integration

System Testing Method

Our system is a web-based app implemented with Typescript for the frontend and python for the backend and simulation. Thus, two different tools will need to be used for each testing phase according to the new code's language.

Unit Tests and Coverage

All pull requests will be required to have unit tests covering at least 70% of the new code. Hence, the team member that created the pull request will need to create unit tests for their code before it can be approved. Frontend will be able to use npm coverage to test and get the coverage report for their code, while backend/sim will use pytest for their unit tests. Codecov will be used to provide coverage reports for all tests within each branch.

Other Tests

To guarantee the quality of the system's code, various other tests will be run against it. The following extra checks will be done for both the front and back/sim.

1. Linting : Checks for programming and stylistic errors
   • Frontend: npm run lint
   • Backend/Sim: flake8 back or flake8 sim
2. Type Check : Checks if correct type of data is entered
   • Frontend: npm run typecheck
   • Backend/Sim: mypy back or mypy sim
3. Check Formatting : Checks the code's formatting
   • Frontend: npx prettier --check app/**/*.{js,jsx,ts,tsx,json,css,md} --no-error-on-unmatched-pattern
   • Backend/Sim: black --check back or black --check sim

These commands will be run automatically through the CI after each commit to a pull request. Further explanations in the next section.

Continuous Integration Strategy

Discuss continuous integration strategy.

Code Review Process

Each pull request will require at least two code reviews and approvals to be considered completed before merging. Thanks to the codeowners file, as explained previously, at least one of the final approvals must be a member of the same sub-team as the branch subsystem (e.g. a PR under a branch from the sim subsystem would require at least one review/approval from a member in the sim team).

Pull requests have been separated into two categories: bug fix and regular PR. The code review process both types must be followed by the team before the pull request can be approved.

Bug Fix PRs

1. Access the bug issue referenced on the PR. Each bug fix PR should have a reference to a bug report issue in the Related Issues section. It is important to understand what problem is being fixed.
2. Follow the steps to reproduce the bug. These steps are provided in the bug report issue.
3. Ensure the steps result to the bug issue's expected behavior. The fix should have resolved the bug and resulted in the expected behavior as described.
   1. If not the case, add a comment to the PR and discuss the differences.
   2. If changes were made, repeat steps 1 to 3 to ensure the code follows the expected behavior.
4. Approve the pull request by leaving a comment. The comment should include the match in behavior between the tested one and the expected one.

General PRs

1. Access the referenced task issue and its user story issue on the PR. Each PR should have a reference to a task issue in the Related Issues section.
2. Cross-check the behavior/UI of the code against its AC/mockups. Ensure the pushed feature looks and behaves as intended through the task issue.
   • If not the case, add a comment to the PR and discuss the differences.
3. Verify the unit tests written. The written test cases should be effective in testing the new feature and cover it properly to ensure the quality of the code.
   1. Tests should cover 70% of the new code.
   2. Verify that they cover the main edge cases.
4. Approve the pull request by leaving a comment. The comment should include the validation of the code's behavior and UI, as well as an approval of the written tests.