Logging and plotting of data

[lochel]

• Selection of variables to store and plot
• Subsample simulation for logging
• Split up the result files into several smaller ones instead of one large when simulating sequentially
• Allow to skip writing a result file
• Make buffer size adjustable

[meek1]

Needed in Release v2.0.

[meek1]

Any examples in the test suite or in documentation on how to use setVariableFilter?

[lochel]

Not yet, but I will add one as soon as this feature is available.

[meek1]

OK, in the meantime, is it possible to simulate sequentially to split up the result files into several smaller ones instead of one large? I tried the following but it resulted in only one result file named "result1.mat":

-- simulation settings
oms2_setCommunicationInterval("AVSDemo", 1e-3)
-- *** 1st simulation sequence ***
oms2_setResultFile("AVSDemo", "result1.mat")
oms2_setStopTime("AVSDemo", 0.02)
oms2_initialize("AVSDemo")
oms2_simulate("AVSDemo")

-- *** 2nd simulation sequence ***
oms2_setResultFile("AVSDemo", "result2.mat")
oms2_setStartTime("AVSDemo", 0.02)
oms2_setStopTime("AVSDemo", 0.04)
oms2_simulate("AVSDemo")

[meek1]

A really nice feature would be to select which variable NOT to store, e.g. all concealed* for black-box FMUs. For several test FMUs, the majority of variables are concealed*.

[meek1]

@lochel, when can we expect a first functionality to specify plot interval/down sampling for FMI composite models? It is very needed. For the test cases I'm running, I need a communication interval of approx. 1e-3s for stability reasons but I'm only interested in results data of say 1Hz.

I don't want the spcified plot interval to force the master to take exact steps of certain lenght, rather data should be interpolated from available points (or as a first version take the point closest to the specified if this is significantly less implementation work).

[lochel]

@meek1 #237 will provide a new API to specify a logging interval. This approach doesn't use any interpolation. It just skips data point which are too close in time. It works already, but I have to test it a bit more before merging (probably later today).

[lochel]

Usage of the new functioanlity: oms2_setLoggingInterval("model", value)

value E (-inf; 0):                  logging disabled
value E [0; communicationInterval]: logging of all data
value > communicationInterval:      reducing the size of the result file by skipping some data points

[meek1]

@lochel, oms2_setLoggingInterval results in model errors during simulation. For the main part of the simulation I want data in approx. 1Hz. In the example below, where should I place oms2_setLoggingInterval("AVSDemo", 1)? Before or after oms2_initialize? Separate logging interval for each mat-file? What if oms2_stepUntil is smaller than specified logging interval?

-- *** initialization ***
oms2_setResultFile("AVSDemo","result_init.mat")
oms2_setStopTime("AVSDemo",5)
oms2_initialize("AVSDemo")
oms2_setCommunicationInterval("AVSDemo", 1e-5)
oms2_stepUntil("AVSDemo", 1e-4)
oms2_setCommunicationInterval("AVSDemo", 1e-4)
oms2_stepUntil("AVSDemo", 1e-1)

-- *** simulatation ***
oms2_setCommunicationInterval("AVSDemo", 1e-3)
oms2_setResultFile("AVSDemo","result_5s.mat")
oms2_simulate("AVSDemo")

-- *** end ***
oms2_terminate("AVSDemo")
oms2_unloadModel("AVSDemo")

[lochel]

@meek1 It shouldn’t matter when you set the logging interval. It is also possible to change it within a single simulation. The following works for me:

-- *** initialization ***
oms2_setResultFile("AVSDemo","result_init.csv")
oms2_setStopTime("AVSDemo",5)
oms2_initialize("AVSDemo")
oms2_setCommunicationInterval("AVSDemo", 1e-5)
oms2_stepUntil("AVSDemo", 1e-4)
oms2_setCommunicationInterval("AVSDemo", 1e-4)
oms2_stepUntil("AVSDemo", 1e-1)

-- *** simulatation ***
oms2_setCommunicationInterval("AVSDemo", 1e-3)
oms2_setLoggingInterval("AVSDemo", 1e-1)
oms2_simulate("AVSDemo")

[lochel]

What are the error messages that you get?

[meek1]

Simulating without oms2_setLoggingInterval works (using v0.1.0-476-gd4b69d4-win). Inserting oms2_setLoggingInterval("AVSDemo", 1e-1) in point A, B, C below gives:
A: Error:
AVSDemo.eCS_Generic_Export (): The following error was detected at time: 0.003747750426350725
AVSDemo.eCS_Generic_Export (): Model error - power: (ECS.pack.comp.Tc/ECS.pack.comp.Tis) ^ (3.5) = (-4331.92) ^ (3.5)
s) ^ (3.5) = (-4331.92) ^ (3.5)
B: See A.
C: Simulation is stuck at ~0.05s without warning.

-- *** initialization ***
A
oms2_setResultFile("AVSDemo","result_init.csv")
oms2_setStopTime("AVSDemo",5)
oms2_initialize("AVSDemo")
B
oms2_setCommunicationInterval("AVSDemo", 1e-5)
oms2_stepUntil("AVSDemo", 1e-4)
oms2_setCommunicationInterval("AVSDemo", 1e-4)
oms2_stepUntil("AVSDemo", 1e-1)

-- *** simulatation ***
oms2_setCommunicationInterval("AVSDemo", 1e-3)
C
oms2_simulate("AVSDemo")

[meek1]

@lochel, please have a look at the test case in OpenCPS repo: \WP6\T6.3\test_setLoggingInterval

[lochel]

Btw, I also used OMSimulator v0.1.0-476-gd4b69d4-win.

[lochel]

I investigated this a bit more and it seem to be in fact a problem with the FMU itself. If I do not fetch all internal real variables (e.g. for writing them to result file) then this model will crash.

[meek1]

OK, but normally for a simulation to run properly, is it not requred to do fmi2GetXXX for all variables before (or after?) each fmi2DoStep? See e.g. p107 or Table 2 at p99:

https://svn.modelica.org/fmi/branches/public/specifications/v2.0/FMI_for_ModelExchange_and_CoSimulation_v2.0.pdf

[lochel]

Exactly, therefore I believe that it must be a problem with the FMU itself.

[meek1]

OK, could it have anything to do with black-box FMUs? Most FMUs in the test case are black-box and have lots of conceald variables. In modelDescription.xml, the causality of the concealed varaibles is not defined, i.e. they are not input, not output, not parameters. Maybe these needs to be updated for each doStep (or something...)?

[lochel]

I don't think so. I think it is a bug inside the FMU. Was it generated by Dymola or some other tool? Maybe it would be a good idea to talk to the support of the tool that was used to generate the FMU.

[meek1]

Dymola 2018 FD01, good idea for long term FMU generation enhancements. Can you come up with some workaround enabling the master to handle FMUs with this (strange) behavior? E.g. perform doStep exactly as before and log everything as before at each communication point, but to a temporary mat/csv file of limited size. Then during simulation interpolate from the temporary file based on oms2_setLoggingInterval and write to a new result file including only interpolated values.

[meek1]

Don't know if it helps but FYI I also get this warning when using setLoggingInterval: AVSDemo.engine (): CVODE: CVode failed with CV_TOO_CLOSE: tout too close to t0 to start integration.

[meek1]

@lochel, we need a workaround so that setLoggingInterval works also for this kind of FMUs. Until the FMU generation has been investigated by the tool vendor, maybe there could be two options in OMSimulator, something like:

1. "always get all variables at each communication point" (as with no setLoggingInterval)
2. "only get outputs at each communication point".

For efficienfy, I guess such option should be set on individual FMU level. To start with, a quick fix is needed so the simulation runs with setLoggingInterval for current FMUs.

I've now tried the following FMU types:

• Black box Cvode: "Original" version, stuck with warning.
• White box Cvode: Similar behavior, stuck but no warning.
• Black box Dassl: Takes forever to initialize.
• Black box Radau: Takes forever to initialize.

Even though it seems like the problem is in the FMUs and the FMU generation should then be corrected, it is in fact a quality of the master to be able to cope with poor FMUs. Not tailoring the master to specific FMU generation tools, but providing general functionality to workaround various problems.

[lochel]

I think everything is now implemented and fixed. Please open a new issue if anything is not working as expected.

[meek1]

Is the workaround also implemented? How do I activate/deactivate?

[lochel]

No it is not. I'll let you know once it is in place.

[lochel]

You can try oms2_setFlags("Model.FMU", "+fetchAllVars") to enable the requested workaround as soon as #252 is merged.