Assessment item 3 - Dynamic Testing
<div>
<strong>Value:</strong>
<span>
<span>40</span>%
</span>
</div>
<div>
<strong>Due Date:</strong>
<span>25-Sep-2020</span>
<!-- show the centrally administered content if applicable -->
</div>
<div>
<strong>Return Date:</strong>
<span>20-Oct-2020</span>
</div>
<div>
<div>
<strong>Submission method options:</strong>
<span>
<span>
<span>EASTS (online)</span>
</span>
</span>
</div>
</div>
<!-- iterate over each assessment item child section -->
<section id="section-6-2-3-1">
<div class="level-4">
<!-- display heading as a header -->
<div class="sectionHeader section-header">
<div class="sectionHeaderTitle section-header-title">
<header>
<h5>Task</h5>
</header>
</div>
<div class="pdf-hide">
<a class="anchor-link" href="#contentPanel">back to top</a>
</div>
</div>
<!-- display heading inline -->
<!-- display learning outcomes -->
<!-- display section value, if the section type is not a banner -->
<div class="fr-view section-value"><div class="word-content"><p style="margin:0in;margin-bottom:.0001pt;">Assignment 3 walks you through the full range of developer based dynamic software testing.</p><p style="margin:0in;margin-bottom:.0001pt;"><br></p><p style="margin:0in;margin-bottom:.0001pt;">In
Part A, you need to critically analyse the provided case study and design documents to determine the areas of highest product and technical risk. You then need to create a minimal ‘Master Test Plan’ in which you specify the amounts and types of testing you will apply to different aspects of the application.
In Part B, you are required to implement a full range of dynamic tests for a particular use case, ranging from low level unit tests, through at least two levels of integration testing, to system testing.
Part C asks you to engage in some critical reflection on the effectiveness and utility of different aspects of dynamic testing.
Please note that it is recognised that standard practice is to specify user acceptance tests before development activities occur, and to conduct unit testing as classes are initially coded. Unfortunately, the semester timetable makes this impractical.
Also note that the focus in this assignment is on the process and techniques applied, not the complexity of the code. It is recognised that the process applied is ‘way over the top’ for the size of problem addressed.
Finally, please note that the code supplied is deliberately NOT bug free. The purpose of your testing is to expose those bugs. You are specifically instructed NOT to modify the code to eliminate those bugs. You are ONLY to implement and report on the results of your testing.
Tasks
Part A – Master Test Plan (10 marks)
You are provided with a case study and a set of design documents and diagrams which support a use case. You are required to analyse the product and technical risks implied by these artefacts and develop a minimal ‘Master Test Plan’ which addresses these risks.
Quality criteria for a minimal master test plan include:
- Provides an overview of the project objectives
- Identifies testing objectives in terms of problem domain outcomes
- Identifies areas of product and technical risk
- Identifies a test strategy for each risk (test level at which risk is addressed, extent and coverage of testing, including static testing approaches)
- For each test level, identifies the test approach (test environment, test goals, entry criteria, exit criteria, defect response)
Part B – Dynamic Testing (80 marks)
You are provided with an implementation and a ‘master test plan’ which specifies the areas of concern, types of testing and coverage required for the application. You are required to implement a suite of dynamic tests to enact that plan using best practice testing techniques and procedures.
Unit Testing - 30 marks
Implement a suite of unit tests according to the master test plan using best practice unit test techniques and processes.
These include:
- Tests are well organised and well named
- Tests are well structured (Arrange, act, assert internal structure)
- Tests are isolated (Tests only one code unit)
- Tests are independent (Can be run in any order, don’t depend on external services or state)
- Tests for error conditions included
- Tests achieve appropriate level of coverage (as specified in the master test plan)
- Each test tests only one thing (No logic in test, in general one assert per test)
- Tests are automated (Do not require visual inspection to differentiate passing or failing)
- Makes effective use of a test framework and test substitute library as appropriate (such as Junit, and Mockito)
- Tests separated from production code
Integration Testing - 30 marks
Implement a suite of integration tests according to the master test plan using best practice integration test techniques and processes.
Use a ‘bottom up’ or ‘top down’ approach to check interaction between entities and between control and entity classes. Using your ‘master test plan’ as a guide, make sure you test the most critical and/or risky system operation. You are not required to test boundary (user interaction) classes.
Guidelines for best practice integration testing include:
- Tests are well organised and well named
- Tests are well structured (Arrange, act, assert internal structure)
- Tests are isolated to only those code units included in the intended level of integration.
- Tests are independent (Can be run in any order, don’t depend on external services or state)
- Tests for error conditions included
- Tests achieve appropriate level of coverage (as specified in the master test plan)
- Each test tests only one thing (no logic in test, in general one assert per test)
- Tests separated from production code
- All levels of integration are addressed
- Tests are automated (Do not require visual inspection to differentiate passing or failing)
- Makes effective use of a test framework and test substitute library as appropriate (such as Junit, and Mockito)
System Tests - 20 marks
On the basis of the requirement and design documentation identify some system test cases which can be used to check correct functionality in the implemented use case. Specify system test procedures, test data, and expected results. A template for a system test will be available in the Resources section of the subject's Interact site.
Carry out the system tests you specified. Use the test data and procedures specified and check that it meets stated acceptance criteria. Report on the success or otherwise of the test. Include screenshots as evidence of your results in your test reports.
System test quality criteria include:
- Tests are well organised (test cases are organised into test scenarios addressing specific functional areas.
- Test are well named
- Preconditions are completely and concretely specified.
- Test data are completely and concretely specified
- Postconditions are completely and concretely specified
- Test steps and expected results for each step are completely and concretely specified
- Tests are objectively evaluated and supporting evidence for test results is provided
Part C – Reflective Practice (10 marks)
Write a reflective report in which you critically evaluate the following questions:
- Which aspect of testing was the most effective? i.e. Which aspect of testing discovered the most defects for the effort expended?
- How relevant and useful are the guidelines for good unit testing practices?
- How easy is it to achieve comprehensive test coverage? What proportion of programming effort should be allocated to testing?
- How effective is dynamic testing compared to static code review in terms of discovering defects?
<!-- display graduate learning outcomes -->
<!-- display assessment items if this section contains them -->
<!-- iterate over each child section -->
</div>
</section>
<section id="section-6-2-3-2">
<div class="level-4">
<!-- display heading as a header -->
<div class="sectionHeader section-header">
<div class="sectionHeaderTitle section-header-title">
<header>
<h5>Rationale</h5>
</header>
</div>
<div class="pdf-hide">
<a class="anchor-link" href="#contentPanel">back to top</a>
</div>
</div>
<!-- display heading inline -->
<!-- display learning outcomes -->
<div class="learning-outcomes">
<h6 class="learning-outcomes-header">Subject learning outcomes</h6>
<!-- display learning outcome stem text -->
<div>
<p class="learning-outcomes-stem-paragraph">This assessment task will assess the following learning outcome/s:</p>
</div>
<!-- display the list of learning outcomes -->
<ul>
<li>
<span>be able to recognise and apply knowledge and skills to
effectively utilize version control as part of professional programming practice.
<!-- display section value, if the section type is not a banner -->
<div class="fr-view section-value"><p>The intent of the assignment is
to articulate taught concepts and skills in the context of developing a suite of unit, integration, and user acceptance tests for a small application.
<!-- display graduate learning outcomes -->
<div class="learning-outcomes">
<div>
<h6 class="learning-outcomes-header">Graduate learning outcomes</h6>
<p class="learning-outcomes-stem-paragraph">
This task also contributes to the assessment of the following <a href="https://student.csu.edu.au/study/glo" target="_blank">CSU Graduate Learning Outcome/s</a>:
</p>
<ul>
<li>
<span>Professional Practice (Skill) - CSU Graduates demonstrate
discipline-specific technical capabilities and self-appraisal required for a beginning practitioner or professional.
<!-- display assessment items if this section contains them -->
<!-- iterate over each child section -->
</div>
</section>
<section id="section-6-2-3-3" class=" landscape">
<div class="level-4">
<!-- display heading as a header -->
<div class="sectionHeader section-header">
<div class="sectionHeaderTitle section-header-title">
<header>
<h5>Marking criteria and standards</h5>
</header>
</div>
<div class="pdf-hide">
<a class="anchor-link" href="#contentPanel">back to top</a>
</div>
</div>
<!-- display heading inline -->
<!-- display learning outcomes -->
<!-- display section value, if the section type is not a banner -->
<div class="fr-view section-value"><div class="word-content"><p style="margin:0in;margin-bottom:.0001pt;"><strong>Master Test Plan (10 marks)</strong></p><p style="margin:0in;margin-bottom:.0001pt;"><br></p><p style="margin:0in;margin-bottom:.0001pt;">Quality criteria:</p><ol><li>Provides an overview of the project objectives </li><li>Identifies test objectives in terms of problem domain outcomes</li><li>Identifies areas of product and technical risk</li><li>Identifies
a test strategy for each risk (test level at which risk is addressed, extent and coverage of testing, including static testing approaches))
Master Test Plan | HD | DI | CR | PS |
8.5 – 10.0 | 7.5 – 8.4 | 6.5 – 7.4 | 5.0 – 6.4 | |
How well did the student demonstrate understanding and skills relating to software testing in the context of specifying a master test plan? | All criteria completely satisfied. | Completely achieved criteria 1, 2, and 3 Achieved criteria 4 and 5 to a large extent | Completely achieved criteria 1 Achieved criteria 2 and 3 to a large extent Achieved criteria 4 and 5 to a partial extent | Achieved criteria 1 and 2 to a large extent Achieved criteria 3, 4, and 5 to a partial extent |
Unit Testing (20 marks)
Quality criteria:
- Tests are well organised and well named
- Tests are well structured (Arrange, act, assert internal structure)
- Tests are isolated (Tests only one code unit)
- Tests are independent (Can be run in any order, don’t depend on external services or state)
- Tests for error conditions included
- Tests achieve appropriate level of coverage (as specified in the master test plan)
- Each test tests only one thing (no logic in test, in general one assert per test)
- Tests are automated (Do not require visual inspection to differentiate passing or failing)
- Makes effective use of a test framework and test substitute library as appropriate (such as Junit, and Mockito)
- Tests separated from production code
Unit Testing | HD | DI | CR | PS |
17.0 – 20.0 | 15.0 – 16.9 | 13.0 – 14.9 | 10.0 – 12.9 | |
How well did the student demonstrate understanding and skills relating to software testing in the context of implementing a set of unit tests? | All criteria completely satisfied. | Completely satisfied criteria 1, 2, 3, 4, and 5 Satisfied criteria 6, 7, 8, 9, and 10 to a large extent | Completely satisfied criteria 1, 2, 3, and 4 Satisfied criteria 5, 6, and 7 to a large extent Satisfied criteria 8 and 9 to a partial extent May not have satisfied criteria 10 | Satisfied criteria 1, 2, 3, and 4 to a large extent Satisfied criteria 5, 6, and 7 to a partial extent May not have satisfied criteria 8, 9, and 10 |
Integration Testing (20 marks)
Quality criteria:
- Tests are well organised and well named
- Tests are well structured (Arrange, act, assert internal structure)
- Tests are isolated to only those code units included in the intended level of integration.
- Tests are independent (Can be run in any order, don’t depend on external services or state)
- Tests for error conditions included
- All levels of integration are addressed
- Tests achieve appropriate level of coverage (as specified in the master test plan)
- Each test tests only one thing (no logic in test, in general one assert per test)
- Tests are automated (Do not require visual inspection to differentiate passing or failing)
- Makes effective use of a test framework and test substitute library as appropriate (such as Junit, and Mockito)
- Tests separated from production code
Integration Testing | HD | DI | CR | PS |
17.0 – 20.0 | 15.0 – 16.9 | 13.0 – 14.9 | 10.0 – 12.9 | |
How well did the student demonstrate understanding and skills relating to software testing in the context of implementing a set of integration tests? | All criteria completely satisfied. | Completely satisfied criteria 1, 2, 3, 4, 5, and 6 Satisfied criteria 7, 8, 9, 10, and 11 to a large extent | Completely satisfied criteria 1, 2, 3, and 4 Satisfied criteria 5, 6, 7, and 8 to a large extent Satisfied criteria 9 and 10 to a partial extent May not have satisfied criteria 11 | Satisfied criteria 1, 2, 3, and 4 to a large extent Satisfied criteria 5, 6, 7, and 8 to a partial extent May not have satisfied criteria 9, 10, and 11 |
System Tests - 20 marks
Quality criteria:
- Tests are well organised (test cases are organised into test scenarios addressing specific functional areas.
- Test are well named
- Preconditions are completely and concretely specified.
- Test data are completely and concretely specified
- Postconditions are completely and concretely specified
- Test steps and expected results for each step are completely and concretely specified
- Tests are objectively evaluated and supporting evidence for test results is provided
System Tests | HD | DI | CR | PS |
17.0 – 20.0 | 15.0 – 16.9 | 13.0 – 14.9 | 10.0 – 12.9 | |
How well did the student demonstrate understanding and skills relating to software testing in the context of implementing a set of system tests? | All criteria completely satisfied. | Completely satisfied criteria 1 and 2 Satisfied criteria 3, 4, 5, 6, and 7 to a large extent | Satisfied criteria 1 and 2 to a large extent Satisfied criteria 3, 4, 5, 6, and 7 to a partial extent | Satisfied all criteria to a partial extent |
Quality Criteria:
- Responses address all questions
- Responses are comprehensive, objective, and insightful
- Responses are consistent with an external view, and supported by available evidence
- Responses go beyond description and display critical analysis.
- Logical conclusions and implications for future practice are stated.
- Analysis and conclusions drawn are logically justified and/or correct in terms of subject material
- All content relates directly to critical reflection, analysis, and argument for conclusions reached
- Responses are concise, relevant, easy to read, and easy to understand.
Reflective Practice | HD | DI | CR | PS |
| 8.5 - 10 | 7.5 - 8.4 | 6.5 - 7.4 | 5.0 - 6.4 | |
How well does the reflective report demonstrate an ability to reflect on and critically evaluate issues encountered involving teamwork, version control, and static code review? | Completely achieved all criteria. | Completely achieved criteria 1, 2, 3 Achieved all other criteria to a large extent | Completely achieved criteria 1 and 3 Achieved criteria 2 to a large extent Achieved all other criteria to a partial extent | Achieved criteria 1, and 3 to a large extent Achieved criteria 2 and 4 to a partial extent May not achieve other criteria |
<!-- display graduate learning outcomes -->
<!-- display assessment items if this section contains them -->
<!-- iterate over each child section -->
</div>
</section>
<section id="section-6-2-3-4" class=" page-break">
<div class="level-4">
<!-- display heading as a header -->
<div class="sectionHeader section-header">
<div class="sectionHeaderTitle section-header-title">
<header>
<h5>Presentation</h5>
</header>
</div>
<div class="pdf-hide">
<a class="anchor-link" href="#contentPanel">back to top</a>
</div>
</div>
<!-- display heading inline -->
<!-- display learning outcomes -->
<!-- display section value, if the section type is not a banner -->
<div class="fr-view section-value"><p>You should submit a zip file through EASTS containing the following:</p><ol style="margin-left: 20px;"><li>A document containing your Master Test Plan</li><li>Your Development Directory containing all source and test code</li><li>A summary document explaining where test files for each section of the assignment can be found</li><li>A document containing your responses to the reflective practice task.</li></ol><p>If you are required to submit using TurnItIn, submit the following:</p><p style="margin-left: 20px;">A Word or PDF document containing the concatenated contents of :</p><ol style="margin-left: 40px;"><li>Your master plan</li><li>A screen shot showing the directory structure and contained files of your test development environment</li><li>The source code for all tests you developed, with each test file separately identified</li><li>Your responses to the reflective practice task.</li></ol></div>
<!-- display graduate learning outcomes -->
<!-- display assessment items if this section contains them -->
<!-- iterate over each child section -->
</div>