Future Testing Concept

Future testing of Simulation Models

The impending transition of our simulations to GitHub and easier access to cloud platforms presents challenges and opportunities for transforming the approach to regression testing, trunk progression and release cycle management. Here we develop a vision, requirements and potential solutions around this topic with the ultimate aim of using it as evidence towards commissioning a project.

The sub-headings below capture top level goals, requirements and possible solutions. Sub-pages contain more detail.

Top-level Goals

This is our vision of what we want to achieve by 202N

• Federate responsibility for advancing the trunks comprising the simulation model components of Momentum to the Momentum Partnership
• Provide equitable regression testing for any developer worldwide
• Leverage GitHub functionality to maximise the opportunities for Continuous Integration
• Leverage Cloud platforms to maximise portability through access to diverse hardware and software
• Minimise the barriers to using cloud and on-premise platforms for regression and compute performance testing

Historically, management of the Unified Model (UM) release cycle was Met Office-centric. This proved to by highly successful and provided strong assurance of the UM for its operational weather forecasting role. However, the UM ecosystem was not well-posed to evolve with trends in technology and software engineering. Its successor, based around the LFRic infrastructure, attempted to catch up with these trends but was limited in its ability to achieve this. With the planned migration of the codebase from the fac/svn-based Met Office Shared Repository Service (MOSRS) to GitHub, and changes to the effective access to cloud platforms, catching up is now a more tractable proposition.

Top-level Requirements

Top-level Potential Solutions