Testing and Errors

Testing

• There are many types of software testing
• Unless you write perfect code, testing must be used to find errors
• Use testing to make code as close to correct as possible
• Testing is used after code is compiled (free of syntax errors)
• Cannot detect the absence of all defects in complex program

Unit Testing

• Each class can have it's own main method which thoroughly tests that class (unit test)
• Unit test should call as many methods of the class with as many combinations of parameters as possible
• Full coverage is the goal of making sure every path through the code has been used at least once (see: Cobertura)

Integration Testing

• Tests how multiple components work together (not just testing a single class)
• Occurs after unit testing

White Box Testing

• In white box testing, the tester may study the code being tested
• The tester uses their programming skills to make sure every path through the code is tested
• Statement testing - ensure each statement is executed at least once
• Branch testing - ensures each choice of a branch (if, switch, loops) is taken
• Path testing - tests each path through a method
• Basically test data should ensure all combinations of paths through the code are executed
• Also called glass-box, open-box, or coverage testing

Advantages	Disadvantages
Knowledge of source code	Knowledge of source code
Ability for tester to remove defects in source	Not always possible to test every single condition
Programmer produces better source because they know it will be scrutinized

Black Box Testing

• Tester does not have access to source code
• Tester simply knows what the source code is supposed to do
• Varying input parameters over large range (and outside acceptable range) and comparing with independently calculated results
• Check all expected inputs as well as unanticipated data
• Acceptance testing

System Testing

• Tests the entire program in the context in which it will be used

Testing Strategies

• Print all method invocations, parameters, and return values
• Write code that makes sure all invariants are established and maintained
• Write test cases that not only provide full coverage, but also test boundary cases
• Decide how the software will be tested
• Decide when the tests will occur (built a little, test a little)
• Decide who will do the testing
• Decide what test data will be used
• Include debugging information in program (can be removed later)

private static final boolean TESTING = true;
if(TESTING) {
  // Code for output statements
}

Common boundary cases

• It's impossible to test infinite values, so we test boundary cases instead
• -1, 0, 1, list.size(), array.length
• First and last elements of array, collection, list, etc
• Items that are somewhere in the middle of the data
• Desired elements that are not in collection or in collection more than once
• Collection has size 0, 1
• Null values

Common errors

• OBOB (off by one bug)
• Not initializing data correctly (can cause Null Pointer Exception)
• Assumptions that don't turn out to be true
• Not checking things that need to be checked (null values)

Use of Stubs

• Stubs are method placeholders for methods called by other classes, but not yet implemented yet
• Stubs allow testing with classes that are still being developed
• Stubs can print out value of inputs, assign predictable values to outputs
• Change the state of variables