Testing
Static Testing is a testing technique in which the software is tested without executing the code. In this project, we do both, execute Static Code Analysis and conduct reviews. Both of these static testing approaches are considered mandatory.
Static code analysis is executed automatically on every commit according to Continous Integration workflow configuration. We use prospector as test runner and manager for all static testing tools that are used:
Prospector run is configured in prospector policy file.
Additionally, security scans are configured in codeql analysis workflow file to execute daily. It is performed to confirm that the code is safe to use. Results of security scans are available for all the users.
Review is started automatically when pull request is created.
Each review shall have at least 2 reviewers. Number of reviewers shall not exceed 4 (extreme case) as we value our time. Major studies in field of computer science proves that there is very little value from having 5th and each additional reviewer. At least one of the reviewers shall be experienced project member (desirably @mdabrowski1990) and have domain knowledge.
There are no strick rules how to perform the review (you can always do it ad-hoc or use any reading technique you want). Consider this checklist as a guideline and recommendation what to focus on:
- Are all features (that are described in linked stories) implemented?
- Are all new functionalities accessable and explained? Please answer this question from user perspective.
- Is implementation compatible with related designs?
- Are unit tests included?
- Does unit tests provide 100% decisions coverage?
- Are all provided features tested as described in linked stories?
- Is documentation updated accordingly or is there link for proper documentation task?
- Are all Acceptance Criteria met?
Security tests are executed daily according to the schedule defined in security workflow file. Here are available results of security CI workflow .
During Dynamic Testing software is compiled and executed to test it.
Unit tests are mandatory in this project. It would be nice if you could write your code using TDD, but it is not required nor verified.
Unit tests decisions coverage is checked though and desired level is 100% so please have this in mind.
Unit tests are executed automatically on every commit according to Continous Integration workflow configuration.
Location of unit tests.
Test coverage assesment.
Integration tests are expected to be created for critical functionalities to make sure that the users always receive fully functional product without major defects.
Integration tests are taken into account during assessment of decisions coverage.
Integration tests are executed automatically on every commit according to Continous Integration workflow configuration.
Location of integration tests.
Test coverage assesment.
During system tests we make sure that our software cooperates with supported hardware.
Example system test would check that our package can be used to communicate over CAN using dedicated hardware for it (e.g. VN1600).
Location of system tests.
Acceptance Tests strategy is decided prior each release.