Test Specification

Types of testing:

##Unit tests

We are unsure of a unit testing framework in Unity, but if there is one, we will use it to test our modules. If there isn't one, we plan to try to build one that we can use.

##Automated tests We hope that it will be possible to run Android automated tests against the game, as this will significantly help our testing process. We would use it for 2 main types of problems:

• Unit functionality tests: We can use automated testing to verify that a puzzle can be solved or that a feature works properly by building a test module to handle the puzzle.
• Integration tests: We can use the test modules to play through levels, which allows us to do integration testing on all of the pieces.

We would have to make sure that the testing modules work across all devices, so there may be extra work in building solution modules for different versions and devices. We describe our compatibility testing below.

Our team has some familiarity with Android's UIAutomator tool, which allows for automated interactions with the Android device. If this tool proves to be inadequate, we will consider other options, like Selendroid or robotium.

##Functionality tests

We have designed a test level that allows us to test each of the game play requirements. If we can beat the level, then each of these requirements is met (except death). If automation possible on Android device, create a script that will play through the test level.

• G1.1 - User controlled player - Inherent in G1.2
• G1.2 - Player can move left/right and jump - Simple puzzle that requires jump to the right, jump to the left
• G1.3 - Blocks that change gravity in all four directions - Path stops, gravity block up allows player to continue
• G1.4 - Blocks that break when shot with gun - Tested with G1.11
• G1.5 - Obtainable keys that unlock specific blocks - Key right before locked block, which is part of a wall
• G1.6 - Spikes that hurt/kill player - Automated? See below
• G1.7 - Blocks that lock screen - Tested with G1.8
• G1.8 - When screen is locked, player jumps from one side of lock to the other - Require locking and screen jumping in all 4 directions
• G1.9 - When player dies, he starts at beginning of level - With G1.6
• G1.10 - Player can finish level - Finish criteria
• G1.11 - Working gun - Wall of breakable blocks

Also negative testing:
Gun doesn't break all blocks, just the breakable ones - shoot unbreakables and try to go through, and if the script still finishes the level, it worked.

See Test and Introduction Level for a view of a preliminary design for the test level.

For milestone 2 game play requirements, we will create a second test level that tests the new requirements:

• G2.1 - Player can manually control vertical gravity - level will have no gravity blocks
• G2.2 - Player can manually control horizontal gravity - same as G2.1
• G2.3 - Player can manually lock screen scroll - level will require this feature to advance
• G2.4 - Mechanic for next region - at present, this is undecided, so we cannot test it as yet
• G2.5 - Player can progress through levels - finishing the first test level will advance to the second

A link to the design will be added here when it is available.

##Compatibility tests Does it work on all Android devices? Use emulator like Genymotion to allow testing of different types and versions of devices. Try single automated script on all, with allowed modification in case of strange differences.

##Soak tests Soak tests, or tests of running a program for an extended amount of time, are used to catch memory leaks. We plan to use them to test several different scenarios:

• Leave game running for hours on the start screen
• Character in a level not moving
• Character in a level holding a button
• Character in a level continually jumping
• Character in a level continually dying (Using automation)
• Character in a level falling

##Beta test After implementing the aforementioned testing plans and completing the second milestone (or the levels ported over from the original game), we will do a beta test of the game by allowing a small group of about 100 people install and play the game. We will follow a MapReduce model of these tests to prevent duplicates, as described below:

• Mappers: report bugs to their assigned reducer.
• Reducers: compare bugs and combine them into a smaller set of bugs that are then reported and tracked on the issue board; reducers assign bugs to specific developers.

We will assign 2 or 3 team members to the reducer role, as bug count requires.

##Requirements testing Some requirements cannot be tested using the automated tool, as they are more of a verification type of testing. For these, we will have a suite of manual test cases. Listed below are the requirements not listed above, with a description of how we will test them.

Level Design

• L1.1 - Create instructions/test level - Inherent in game play tests listed above
• L1.2 - Design puzzles using newly discussed possible mechanics - Once mechanic is decided, we will verify that they work using automation, as described above

• ---

• L2.1 - Design for next region using new mechanics, plugging in puzzles previously designed - We will verify that there are new levels that include the mechanics that we decide on

Storyline

• S1.1 - Create a story line - We will verify that a story line exists
• S1.2 - Use of existing levels makes sense - We will verify that the story line is in-world feasible
• S1.3 - Scaleable - Makes addition of new regions make sense - We will verify that the story line allows for new levels and regions in a way that is in-world feasible
• S1.4 - Write script for intro and initial 10 levels - We will verify that the script is written following the story line

• ---

• S2.1 - Write script for next region - We will verify that the script is written following the story line

Environment

• E2.1 - Port initial 10 levels from old version - We will attempt to use automation to run through each of the original levels and as such verify that they exist in the new version
• E2.2 - Decide on mechanic for next region - We will verify that a mechanic was decided
• E2.3 - Design new "regions" with several levels in each - We will verify that new regions' concept art exists
• E2.4 - Design upgrades that suit environment - We will verify that the upgrades suit the environment, meaning that they are in-game feasible to the environment

Art

• A1.1 - Art for intro screen - We will verify that the intro screen art exists
• A1.2 - Changes to original sprites for initial 10 levels - Discarded requirement
• A1.3 - Concept for each of the possible regions - Inherent in E2.3
• A1.4 - New basic sprites for new regions - We will verify that sprites exists for the new regions

• ---

• A2.1 - In-depth art for next region - We will verify that the next region has models and artwork with which the Unity team can develop