-
Notifications
You must be signed in to change notification settings - Fork 10
Validation_Plan
{{>toc}}
This summarizes the ways we have validated GUNNS in the past and their results. These tests have not been maintained and are not part of our current validation anymore.
Circa 2011 when GUNNS was in the “alpha” stage of development under the TS-21 project, we compared fluid and thermal aspect networks against equivalent models in the Thermal Desktop/SINDA/FLUINT COTS tool.
The thermal aspect network modelled a section of the NASA Orion capsule. The GUNNS network was developed partially from an automated export of the Thermal Desktop (TD) model, with simplifications made by hand. Validation compared temperature trends of all network nodes between the GUNNS & TD models over several hours in an example thermal environment. A presentation of the results: Early Thermal Validation Against Thermal Desktop
The fluid aspect network was a theoretical case of a high-pressure O2 tank blowing down to a large cabin volume through a pipe. The GUNNS network was tuned to match the TD model pipe flow rates, similar to how we normally tune fluid networks to match a desired system. Validation compared the pipe flow rate, O2 tank and cabin pressures & temperatures over the duration of the blow-down and documented the variance between the GUNNS & TD models. The variance was similar to the expected variance when comparing GUNNS fluid flows against more high-fidelity theory (see a discussion on this here). A presentation of the results and the modelled system: Early Fluid Validation Against Thermal Desktop
Circa 2013, the MMSEV project developed a fluid network modeling a real-world PCS design for the Orion/MPCV capsule. This fluid network was tuned to match data collected from an actual hardware build of the system in tests at JSC. This served as a validation of the GUNNS fluid aspect. A presentation of the results: MMSEV PCS Validation
GUNNS has been validated by the NASA customer in the following training simulation deliveries:
- TS-21’s Generic Visiting Vehicle (GVV) sim, which was initially delivered in 2012. It passed Operational Readiness Review (ORR) in TBD
These tests are actively maintained in ER’s current development environment. These are what we rely on to validate that GUNNS is currently “working”.
Our Continuous Integration environment runs a set of integration tests whenever we update our git next branch. It includes these test projects:
- GUNNS_SIM_test: this builds & runs the Trick sim in gunns/sims/SIM_test. It currently includes very simple fluid and basic networks. It verifies the basic functionality of the solver, network initialization & step, and solver/link and solver/node interfaces. It does not verify detailed signatures of GUNNS models (links).
- GUNNS_UT: this builds & runs the complete suite of CPPUnit tests as the gunns/test/make_all_ut.sh script.
This is a high-level summary of the GUNNS functionality that are verified in our current integration tests:
TODO
All delivered GUNNS/Core code is thoroughly tested at the unit level. See Unit_Testing. The gunns/ repo currently has roughly 13,000 separate assert statements in its various unit tests:
- gunns/core: ~3000
- gunns/aspects/fluid: ~5700
- gunns/aspects/thermal: ~700
- gunns/aspects/electrical: ~3000
- gunns/gunnshow: ~150
- gunns/ms-utils/(various): ~500
Validation of the GUNNS standard fluid flow equation performance against orifice and pipe flow theory is presented here.
These are additional validation efforts that we expect to occur in the future. We don’t have a timetable for these.
Several of the items listed below will eventually be included as automated regression tests in our Continuous Integration test suite. These will provide continuous that a larger portion of the GUNNS functionality is in working order.
We plan to continue our practice of maintaining complete code coverage in our unit tests for new development.
We will establish a more permanent test fixture comparing GUNNS thermal and fluid outputs to TD. It may be similar in scope to the effort described in 1.1 above, and set up as an automated regression test in our Continuous Integration.
The HESTIA project is producing GUNNS fluid networks that will be compared to data collected from the real hardware systems they are modelling.
We are currently developing models in all 3 aspects for the NExSyS project. We will create test versions of some of these models that will validate GUNNS adheres to the principles of conservation of mass and energy.
Mass conservation in the fluid aspect will be shown via a before & after comparison of the total mass balance of all fluid constituents in NExSyS EAM ECLSS models. This will validate conservation of mass in all modelled fluid processes: transport internal to and between networks, chemical reactions, absorption/desorption, molecular diffusion, thermal convection, conduction and direct heating, etc.
Conservation of energy through all 3 aspects will be shown by an end-to-end flow of power through a NExSyS EAM vehicle model. Power absorbed by solar panels in the electrical aspect is converted to waste heat added to the thermal aspect, transported through the fluid aspect via multiple coolant fluid types and across multiple heat exchanger devices, and finally radiated back to space. In steady-state, this model will radiate the same total power absorbed by the solar panels.
These tests will be added to our Continuous Integration test suite as automated regression tests.
GUNNS will pass a major validation milestone when the TS-21 ISS full-task trainer (SSTF) project passes its ORR, currently planned for December 2015.
GUNNS electrical networks are being used in testing of EAM power distribution system hardware. GUNNS is being used to simulate the solar array power generation and regulation of the EAM vehicle, and the simulated output supply voltage of the regulator is an input to the test hardware loads. We plan to later plug the recorded hardware loads back into our simulation and compare the overall system reponse to the hardware test. This will serve as a validation of the electrical aspect.