Test Specification

Types of testing:

##Unit tests

We are unsure of a unit testing framework in Unity, but if there is one, we will use it to test our modules. If there isn't one, we plan to try to build one that we can use.

##Automated tests We hope that it will be possible to run Android automated tests against the game, as this will significantly help our testing process. We would use it for 2 main types of problems:

-Unit functionality tests: We can use automated testing to verify that a puzzle can be solved or that a feature works properly by building a test module to handle the puzzle.
-Integration tests: We can use the test modules to play through levels, which allows us to do integration testing on all of the pieces.

We would have to make sure that the testing modules work across all devices, so there may be extra work in building solution modules for different versions and devices. We describe our compatibility testing below.

##Functionality tests

We have designed a test level that allows us to test each of the game play requirements. If we can beat the level, then each of these requirements is met (except death). If automation possible on Android device, create a script that will play through the test level.

• G1.1 - User controlled player - Inherent in G1.2
• G1.2 - Player can move left/right and jump - Simple puzzle that requires jump to the right, jump to the left
• G1.3 - Blocks that change gravity in all four directions - Path stops, gravity block up allows player to continue
• G1.4 - Blocks that break when shot with gun - Tested with G1.11
• G1.5 - Obtainable keys that unlock specific blocks - Key right before locked block, which is part of a wall
• G1.6 - Spikes that hurt/kill player - Automated? See below
• G1.7 - Blocks that lock screen - Tested with G1.8
• G1.8 - When screen is locked, player jumps from one side of lock to the other - Require locking and screen jumping in all 4 directions
• G1.9 - When player dies, he starts at beginning of level - With G1.6
• G1.10 - Player can finish level - Finish criteria
• G1.11 - Working gun - Wall of breakable blocks

Also negative testing:
Gun doesn't break all blocks, just the breakable ones - shoot unbreakables and try to go through, and if the script still finishes the level, it worked.

See Test and Introduction Level for a view of a preliminary design for the test level.

##Compatibility tests Does it work on all Android devices? Use emulator like Genymotion to allow testing of different types and versions of devices. Try single automated script on all, with allowed modification in case of strange differences.

##Soak tests Soak tests, or tests of running a program for an extended amount of time, are used to catch memory leaks. We plan to use them to test several different scenarios:

• Leave game running for hours on the start screen
• Character in a level not moving
• Character in a level holding a button
• Character in a level continually jumping
• Character in a level continually dying (Using automation)
• Character in a level falling

##Beta test After implementing the aforementioned testing plans and completing the second milestone (or the levels ported over from the original game), we will do a beta test of the game by allowing a small group of about 100 people install and play the game. We will follow a MapReduce model of these tests to prevent duplicates, as described below:

• Mappers: report bugs to their assigned reducer.
• Reducers: compare bugs and combine them into a smaller set of bugs that are then reported and tracked on the issue board; reducers assign bugs to specific developers.

We will assign 2 or 3 team members to the reducer role, as bug count requires.