Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FMU Conformance to FMI 2.0 for CS #639

Open
CThuleHansen opened this issue Sep 30, 2019 · 4 comments

Comments

@CThuleHansen
Copy link

commented Sep 30, 2019

PLEASE ADD LUND LABEL IF CAPABLE

What

Systematic testing of the cross-check FMUs in context of FMI 2.0 for co-simulation has revealed several shortcomings of the FMU implementations and the static description files.
The conducted systematic testing consisted of:

  • A static check of the model description files
  • A model-based testing of the dynamic behaviour of FMUs

A static check of the model description files implemented with VDM-SL

Some examples
invalid InitialUnknowns (missing required unknowns, or too many unknowns)
ScalarVariable inconsistencies with causality, variability, initial and start attributes

Tool is available
https://github.com/INTO-CPS-Association/FMI2-VDM-Model/releases

To be presented at
The 17th Overture Workshop, October 7. (Nick Battle, Casper Thule, Cláudio Gomes, Hugo Daniel Macedo and Peter Gorm Larsen. “Towards Static Check of FMUs in VDM-SL”)
Pre-proceedings available: https://github.com/overturetool/overturetool.github.io/raw/master/workshops/17/overture-2019-pre-proceedings.pdf
Some Results
– There are 692 FMUs in the 2.0 branch of the repository (some are duplicated for different architectures and tools)
– 294 of them pass without any @OnFail messages (ie. isValidFMIModelDescription returns true)
– 123 of them have malformed structured ScalarVariable names
– 118 of them have invalid InitialUnknowns (missing required unknowns, or too many unknowns)
– 79 of them have ModelStructure Derivatives indexes that do no match Real/derivative variables
– 67 of them have malformed floating point variables (eg. "1." or "4.e-10")
– 56 of them have aliased variables that do not all have the same units
– 33 of them have ScalarVariable attribute inconsistencies
– 27 have a Derivatives section but no derivative variables
– 24 have the reinit flag set for co-simulation models
– 18 have derivative variables defined but no Derivatives section
– 14 have Real units that are not declared in UnitDefinitions
– 13 of them have ModelStructure Outputs that do not include every "output" variable
– 4 of them have InitialUnknowns that are not sorted in ascending order

A model-based testing of the dynamic behavior of FMUs

Some examples
image

Tool is available
https://msdl.uantwerpen.be/git/claudio/FMIMOBSTER

To be submitted to
American Modelica Conference 2020
(Cláudio Gomes, Romain Franceschini, Nick Battle, Casper Thule, Kenneth Lausdahl, Hans Vangheluwe, Peter Gorm Larsen. “Application of Model Based Testing to Dynamic Conformance Evaluation of Functional Mockup Units”)
The paper also considers feed-through (#623 (comment))

The publications have yet to be presented.

Goal

The goal of the research is to support the FMI community in resolving possible ambiguities and improving the conformance to the standard.

Why

It is of interest that FMUs adheres to the standard entirely and correctly to improve the adoption and usability of the standard.
“The most acknowledged challenge is related to practical aspects. These include: faulty/incomplete implementations of the FMI standard, ambiguities/omissions in the specification and documentation of the FMUs, …”
(Schweiger, Gerald, Cláudio Gomes, Georg Engel, Irene Hafner, Josef-Peter Schoeggl, Alfred Posch, and Thierry Nouidui. 2019. “An Empirical Survey on Co-Simulation: Promising Standards, Challenges and Research Needs.” Simulation Modelling Practice and Theory 95: 148–63. https://doi.org/10.1016/j.simpat.2019.05.001.)

To support this process it is ideal to provide tools that tests the FMUs compatibility with the FMI standard.

This issue has been discussed and approved by @clagms

@chrbertsch

This comment has been minimized.

Copy link
Collaborator

commented Oct 1, 2019

@CThuleHansen : Do you address with this issue shortcoming of the FMI standard or of tools exporting FMUs according to the standard? My understanding when realding it is the second.

@CThuleHansen

This comment has been minimized.

Copy link
Author

commented Oct 1, 2019

@chrbertsch We have discussed this quite a bit internally, and it might be both. Some due to imprecisions/ambiguities of the standard and some due to tools not fully implementing the standard

@chrbertsch

This comment has been minimized.

Copy link
Collaborator

commented Oct 1, 2019

@CThuleHansen will present this at the coffee breek of the FMI Design meeting today (at 15;30 in room "Micro".

@nickbattle

This comment has been minimized.

Copy link

commented Oct 1, 2019

I think it is both, as @CThuleHansen says. In order to produce the formal specification, we have to shake the ambiguities out of the standard (and we may have done so incorrectly, but the result is unambiguous). As a result, the formal spec will then identify problems in FMUs that tools have produced (which may or may not be valid, depending on how accurately we formalized the standard). But overall, the result is unambiguous; the challenge is to make the formal specification correct, given that we only have an informal basis and existing tools to go by.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.