Skip to content

Commit

Permalink
added more documents
Browse files Browse the repository at this point in the history
  • Loading branch information
Dirk de Kok committed Jan 22, 2013
1 parent 457c0dd commit 40d6dbc
Show file tree
Hide file tree
Showing 4 changed files with 160 additions and 15 deletions.
3 changes: 3 additions & 0 deletions documents/README.md
View file
@@ -0,0 +1,3 @@
#README documents

These documents are meant to be used as checklist and template for testing plans
35 changes: 35 additions & 0 deletions documents/concept_testing_mobile_apps.md
View file
@@ -0,0 +1,35 @@
# Concept testing mobile apps

The goal of a concept test is to verify that there is a certain target market with a demand for your product. You are solving a problem for them, or fullfilling a need

##Setup
1. Get 10-50 respondents that fit in your target group
2. Prepare materials that indicate or explain the working of your new app
3. Decide whether you want to do the test remotely or in person. In person will give better results but is obviously more expensive

## Points of attention

### Get the right respondents
Think about who your target market is, and get respondents that represent that market.

### Avoid social desirable questions and subsequent answers
Being rewarded for participating in your test makes it very likely that respondents want to please

## Questions to ask
1. What is the goal of the prototype app?
2. What kind of functions does the app have?
3. How do you do this currently without this app?
4. How much time do you spend on average doing this?
5. How do you feel about the current way you do this?


#Copyright and license
Copyright 2013 Mobtest, Inc.

See [http://github.com/mobtest](http://github.com/mobtest) for most current version

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this work except in compliance with the License. You may obtain a copy of the License in the LICENSE file, or at:

[http://www.apache.org/licenses/LICENSE-2.0](http://www.apache.org/licenses/LICENSE-2.0)

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
68 changes: 53 additions & 15 deletions documents/manual_qa_ios_apps.md
View file
Expand Up @@ -22,14 +22,14 @@ Which of the following hardware models does the app support?

You might deem iOS 4.x obsolete

## Device Orientation and rotation
## Device orientation and rotation
1. Does the app support both landscape and portrait orientation?
2. Any particular views (UIViewControllers) that support different orientations?

## Network
How does the app behave under these network circumstances:

1. No network at all
1. No network at all/Airplane mode
2. Intermittent network
3. Cellular slow: 2G (Edge)
4. Cellular medium: 3G (UTMS, CDMA)
Expand All @@ -50,7 +50,7 @@ Other network questions:
1. Is confidential data sent over unsecured HTTP?
2. Are user credentials like username/password sent over unsecured HTTP?
3. Are sensitive files stored unencrypted on the device?
4. Would a pin code on the app (e.g. DropBox) be advisable?
4. Would a pin code on the app (e.g. like DropBox) be advisable?
5. What happens if a device is stolen? Should the app be remotely wiped?


Expand All @@ -62,6 +62,10 @@ How does the app behave under low memory circumstances:

Trigger memory warnings in the Simulator.

## Local data and caching
1. Is local data stored? Is it backed up to iCloud or not?
2. What happens when the OS deciced to whipe out temp files, e.g. when not enough disk space is available?

## Battery
1. Does the app drain the battery extensive?
2. Does the app use background processes like fine grained location services that require a lot of resources?
Expand All @@ -70,10 +74,13 @@ Trigger memory warnings in the Simulator.
1. Can a very large text be entered?
2. What happens on suspend and resume? Can I continue entering text where I left off?

## Context and environment
1. How is it to use the app outdoors with lots of light?
2. Does a busy/distracting surrounding make the app significantly harder to use?
3. Will the app be used in a nonoptimal way, like walking/running, attached to arm with an armband?
## Input
1. How do gestures work?
2. Is tilting one of the input ways? How does it work? Does it interfere with other input methods?

## Location
1. Does the app use location? What level is required?
2. What happens if the location is off, e.g. when a user is indoors with no GPS and unknown wifi?

## Accessability
1. Are controls accessible?
Expand All @@ -82,19 +89,20 @@ Trigger memory warnings in the Simulator.
## Crash logs
Use one of the following services to collect and analyse crash logs:

1. Crashlytics
2. Crittercism
3. iTunes Connect
1. [Crashlytics](http://www.crashlytics.com)
2. [Crittercism](http://www.crittercism.com/)
3. [iTunes Connect](https://developer.apple.com/library/ios/#technotes/tn2008/tn2151.html)

## Installation
1. What is the size of the initial download, is it too big for cellular downloads?
1. What is the size of the initial download, is it too big for cellular downloads (> 50mb)?

## Updates
1. What is the size of the updated app, is it too big for cellular downloads?
1. What is the size of the updated app, is it too big for cellular downloads (> 50mb)?
2. Are there database updates? Have they been tested with multiple previous releases?
3. Is CoreData used for database migrations?
4. Are there other persistent data/settings that should be tested with updates?
5. Did you update the version number in a consistent way?
6. Are updates forces to keep up with an updated server environment?

## Guidelines
1. Are the [Apple App Review Guidelines](https://developer.apple.com/appstore/guidelines.html) followed?
Expand All @@ -111,7 +119,6 @@ Use one of the following services to collect and analyse crash logs:
7. Can the process be hacked?
8. What happens if Apples sends you inconsistent info, is the data validated?


## Push notifications
1. Can a user turn off push notifications?
2. What happens when a user uninstalls the app?
Expand All @@ -122,15 +129,46 @@ Use one of the following services to collect and analyse crash logs:
7. How consistent is the user experience if after a time of no connectivity only the last push notification arrives at the phone?
8. How consistent is the user experience with delays on notifications (wifi only means a check every 15 minutes)
9. How do you deal with push notifications when users are behind a firewall that blocks port 5223, on which the push notification process runs?

10. What does the user see in Do-not-disturb mode?

## Consistent data
1. Do various ways of displaying information do this consistently? As in unread icon, and then the list of unread messages?
2. Is your app's badge updated consistently?
3. What happens on a crash while information is processed, is there a repair process or a consistency check?
4. What happens if the server returns inconsisstent data?
5. What happens if a user sets the time of their device incorrect?

## Interrupts
1. How does the app deal with interrupts like incoming calls, push notifications?
2. After returning to the app, is it in a consistent state?

##
## Animations & concurrency
1. Are all animations smooth?
2. Can multiple animations be triggered at the same time?
3. What happens if an animation is started and the user rotates the app?
4. Are buttons disabled after an initial touch and before results returned?
5. Does the app show activity indicators while a process is running, e.g. a network call lasting several seconds?
5. What happens if a user initiates a process, and then quickly navigates away before the process is finished?

## Localisation and time
1. Are multiple time formats (12-24h) supported?
2. What happens in other timezones?
3. What languages are supported?
4. Do all text snippets and labels fit in all languages
5. Is the translation correct?
6. Are graphics with text also translated?

## Custom Url schemes
1. What url schemes are supported?
2. Any conflict with an existing scheme? Check [handleOpenURL](http://handleopenurl.com/)

#Copyright and license
Copyright 2013 Mobtest, Inc.

See [http://github.com/mobtest](http://github.com/mobtest) for most current version

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this work except in compliance with the License. You may obtain a copy of the License in the LICENSE file, or at:

[http://www.apache.org/licenses/LICENSE-2.0](http://www.apache.org/licenses/LICENSE-2.0)

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
69 changes: 69 additions & 0 deletions documents/usability_testing_mobile_apps.md
View file
@@ -0,0 +1,69 @@
# Usability testing mobile apps

The goal of usability testing is to see how efficiently and succesfully someone can use your app, while having an enjoyable experience

## Testing plan
1. What are the main use cases? Describe them for yourself, and give the end state as a taks to users: buy a car, book a room etc.
2. For quantive research, track each step that a user takes with software, use heatmap technology to see what buttons and areas are touched

## Context and environment
1. In what context is the app used? Is this a peaceful environment (e.g. home, office) or busy (street, public transportation). With or without cell phone reception/wifi?
2. Is the app tested and optimised for the most common tasks & environments?
3. How does the app support the user flow?
4. How is it to use the app outdoors with lots of light?
5. Does a busy/distracting surrounding make the app significantly harder to use?
6. Will the app be used in a nonoptimal way, like walking/running, attached to arm with an armband, one hand iPad usage?

## Clarity
1. How many call to actions are there on a screen? Can this be limited?
2. Are the main buttons or texts clearly visible and outstanding?

## Discoverability
1. Is it easy for a user to figure out how the app works?
2. Are there hidden functionalities unlocked by swiping, tilting, tap-and-hold-down?
3. Would a walk through/tutorial be needed to explain the app? Preferably not but maybe unavoidable

## Speed
1. How much time does it cost the app to become interactive? Is this less than 10 seconds?
2. Is the number of steps, number of input fields to be filled out minimised?

## Conventions
1. Are there several apps that offer similar functionality? Think refresh of table data, browse through photo's or pages, etc.
2. Is this functionality implemented in the most standard way?
3. Are the [iOS HIG (Human Interface Guidelines)](http://developer.apple.com/library/ios/#documentation/UserExperience/Conceptual/MobileHIG/Introduction/Introduction.html) followed?

## Error messages
1. Are all error messages clear?
2. Is there no technical jargon involved?
3. In case of user errors, is the explanation what the user should do clear?
4. In case of no network, does the app retry by itself when network has been restored?

## Buttons
1. Are buttons big enough?
2. Do buttons have a highlight state?
3. Are buttons disabled when the started process takes a long time (say more than .2 seconds)?

## iPad-specific issues
1. Does the app work as well on an iPad as on an iPad Mini? Are buttons big enough
2. Are both portrait and landscape mode supported?
3. Is the app really optimised for the iPad, and not just a blown up version of the iPhone app?

## Pleasant experience
1. What happens if there is no network? Can the app still be used to browse offline content?
2. Is the interface responsive under all circumstances, including when long running processes are busy (e.g. network calls)
3. Does the app have personality that creates a memorable and pleasant memory?

## Sign in/registration
1. Sign up: is Facebook and Twitter supported? Also standard email/password?
2. If also Facebook is supported, how do you deal with iOS 5 users? Only iOS 6 has built in support for Facebook

#Copyright and license
Copyright 2013 Mobtest, Inc.

See [http://github.com/mobtest](http://github.com/mobtest) for most current version

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this work except in compliance with the License. You may obtain a copy of the License in the LICENSE file, or at:

[http://www.apache.org/licenses/LICENSE-2.0](http://www.apache.org/licenses/LICENSE-2.0)

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

0 comments on commit 40d6dbc

Please sign in to comment.