Domain TEST

Tests and field learning

Start here · Questions · Progress · People decide

Turn ideas into checkable results through demonstrations, measurements, exercises and field experience.

Question	What we are working out	AI-generated potential answers and research	Work state
TEST-01	What operating conditions and measurable results should the first version demonstrate?	AI-generated option: Agree acceptance measures around successful public tasks, information age, message handling, recovery and operator workload. Test ordinary use and disruption under stated conditions, recording actual observations and each limit that the next version should improve. AI-generated future concept: A twin could generate repeatable demand bursts while an AI evaluator compares logs with expected outcomes. XR trials could reveal crowding or confusing layouts; observed participant results and equipment measurements would remain the basis for performance claims. Research examples: Digital Performance Standard: measure service success · Managing Exercises Handbook	open
TEST-02	How should a candidate release demonstrate reliable local operation, safe updates and recovery from faults or a compromised account?	AI-generated option: Prove a candidate release can start offline, reject unsuitable updates and recover to a working edition. Rehearse delayed connections, uncertain time, frozen screens and lost credentials with test accounts, recording the observed result and recovery steps. AI-generated future concept: An AI/SI test team could search for failure combinations within a bounded laboratory and propose patches for review. A future device controller could detect a frozen screen, switch to an approved fallback and log exactly why it acted. Research examples: Managing Exercises Handbook · Australia's AI Ethics Principles	open
TEST-03	What evidence should support each endurance and continuity claim, from simulation through bench work to a site exercise?	AI-generated option: Support each endurance claim with its own evidence: model assumptions, bench measurements and site observations. In the practical exercise over two to three days, include changing energy, weather, staff, fuel, supplies, connectivity and recovery. Record actual test duration and participant time; battery capacity alone does not establish service continuity. AI-generated future concept: A living twin could compare measured behaviour with its forecast and show the resulting uncertainty. Advanced AI/SI could suggest the next high-value measurement, while qualified reviewers assess whether the evidence supports a particular installation or operating claim. Research examples: Shoalhaven: Community Information Hubs Launched · Managing Exercises Handbook	open
TEST-04	How should people demonstrate useful access, understanding and learning across ordinary, assisted and reduced-capability service?	AI-generated option: Ask varied participants to complete useful tasks through touch, text, audio, assistance and offline service. Observe understanding, successful completion, confidence and workload, then revise the design from what people actually do and explain. AI-generated future concept: Compare ordinary and XR or Aura-assisted versions of the same task, including a person declining personal AI. Future adaptive teaching could offer just-in-time practice, with users controlling context and observers checking learning rather than only engagement. Research examples: Digital Performance Standard: measure service success · Privacy and the use of commercially available AI products	open
TEST-05	What evidence and decisions should bring the service to a supervised field pilot?	AI-generated option: Prepare a supervised pilot packet connecting every readiness requirement to actual evidence and its responsible reviewer. Include host permission, engineering, staffing, information agreements, privacy, cultural conditions, maintenance and an agreed return to the previous service. AI-generated future concept: Let a shared twin and Aura-guided walkthrough explain readiness evidence to participants and reviewers. AI/SI could detect missing dependencies, while the actual qualified professionals, authorities, hosts and receiving services provide decisions within their respective responsibilities. Research examples: Managing Exercises Handbook · Australia's AI Ethics Principles	open
TEST-06	What results should guide the decision to grow, improve, pause or retire a capability?	AI-generated option: Review everyday usefulness, disaster tasks, reliability, access, privacy, learning, staffing and whole-service cost together. Compare results with agreed thresholds and record the reasons to grow, improve, pause or retire each capability and support people through the change. AI-generated future concept: An AI/SI learning system could identify trends across approved evidence and simulate improvement options. Auras could explain the alternatives to participants, with decisions preserving user choice, professional assessment and the history of why a capability changed. Research examples: Digital Performance Standard: measure service success · Managing Exercises Handbook	open