Test Specification

Types of testing:

##Unit tests

We are unsure of a unit testing framework in Unity, but if there is one, we will use it to test our modules. If there isn't one, we plan to try to build one that we can use.

##Automated tests We hope that it will be possible to run Android automated tests against the game, as this will significantly help our testing process. We would use it for 2 main types of problems:

• Unit functionality tests: We can use automated testing to verify that a puzzle can be solved or that a feature works properly by building a test module to handle the puzzle.
• Integration tests: We can use the test modules to play through levels, which allows us to do integration testing on all of the pieces.

We would have to make sure that the testing modules work across all devices, so there may be extra work in building solution modules for different versions and devices. We describe our compatibility testing below.

Our team has some familiarity with Android's UIAutomator tool, which allows for automated interactions with the Android device. If this tool proves to be inadequate, we will consider other options, like Selendroid or robotium.

##Functionality tests

We have designed a test level that allows us to test each of the game play requirements. If we can beat the level, then each of these requirements is met (except death). If automation possible on Android device, create a script that will play through the test level.

• G1.1 - User controlled player - Inherent in G1.2
• G1.2 - Player can move left/right and jump - Simple puzzle that requires jump to the right, jump to the left
• G1.3 - Blocks that change gravity in all four directions - Path stops, gravity block up allows player to continue
• G1.4 - Blocks that break when shot with gun - Tested with G1.11
• G1.5 - Obtainable keys that unlock specific blocks - Key right before locked block, which is part of a wall
• G1.6 - Spikes that hurt/kill player - Automated? See below
• G1.7 - Blocks that lock screen - Tested with G1.8
• G1.8 - When screen is locked, player jumps from one side of lock to the other - Require locking and screen jumping in all 4 directions
• G1.9 - When player dies, he starts at beginning of level - With G1.6
• G1.10 - Player can finish level - Finish criteria
• G1.11 - Working gun - Wall of breakable blocks

Also negative testing:
Gun doesn't break all blocks, just the breakable ones - shoot unbreakables and try to go through, and if the script still finishes the level, it worked.

See Test and Introduction Level for a view of a preliminary design for the test level.

For milestone 2 game play requirements, we will create a second test level that tests the new requirements:

• G2.1 - Player can manually control vertical gravity - level will have no gravity blocks
• G2.2 - Player can manually control horizontal gravity - same as G2.1
• G2.3 - Player can manually lock screen scroll - level will require this feature to advance
• G2.4 - Mechanic for next region - at present, this is undecided, so we cannot test it as yet
• G2.5 - Player can progress through levels - finishing the first test level will advance to the second

A link to the design will be added here when it is available.

##Compatibility tests Does it work on all Android devices? Use emulator like Genymotion to allow testing of different types and versions of devices. Try single automated script on all, with allowed modification in case of strange differences.

##Soak tests Soak tests, or tests of running a program for an extended amount of time, are used to catch memory leaks. We plan to use them to test several different scenarios:

• Leave game running for hours on the start screen
• Character in a level not moving
• Character in a level holding a button
• Character in a level continually jumping
• Character in a level continually dying (Using automation)
• Character in a level falling

##Beta test After implementing the aforementioned testing plans and completing the second milestone (or the levels ported over from the original game), we will do a beta test of the game by allowing a small group of about 100 people install and play the game. We will follow a MapReduce model of these tests to prevent duplicates, as described below:

• Mappers: report bugs to their assigned reducer.
• Reducers: compare bugs and combine them into a smaller set of bugs that are then reported and tracked on the issue board; reducers assign bugs to specific developers.

We will assign 2 or 3 team members to the reducer role, as bug count requires.

##Requirements testing