Evaluation at translation time and compliance with Modelica Specification

[AntoineGautier]

We are trying to use the JSON interface of ExternData and have a few questions.

The first one pertains to the compliance with the Modelica Language Specification, see https://specification.modelica.org/maint/3.5/class-predefined-types-and-declarations.html#specialized-classes where it is stated

The components directly declared in a record may only be of specialized class record or type.

So we wonder whether the construction of Types.ExternJSONFile json within the record ExternData.JSONFile is valid or not (although Dymola and Optimica do not complain). We noticed that a related issue is left open at modelica-3rdparty/Modelica_DeviceDrivers#93.

The second one is how to trigger the evaluation of some parameters which are assigned through calls to ExternData functions and used during flattening or symbolic processing, such as array size, and min, max or nominal attributes.
We noticed that the class annotation annotation(__Dymola_translate=true) was added in recent versions of the library. However, the following model still fails to translate with Dymola (version 2022), unless a literal value is assigned to the parameter n.

model Unnamed
  parameter ExternData.JSONFile dat(
    fileName=Modelica.Utilities.Files.loadResource(
      "modelica://ExternData/Resources/Examples/test.json"));
  parameter Integer n = dat.getArraySize1D(varName="vector") annotation(Evaluate=true);
  parameter Real array[n] = dat.getRealArray1D(varName="vector", n=n);
end Unnamed;

We did some tests (see TestModel below) with a previous version of ExternData (2.5.0 which is compatible with MSL 3.2.3) and it seems that the only case where annotation(__Dymola_translate=true) effectively triggers the evaluation at translation is when the external object is constructed within the function, as in loadAndGetReal. Otherwise we get warnings with Dymola that non-literal values are used in nominal attribute for p and r.
So is there a recommended way to force the evaluation at translation time, without resorting to a construct such as loadAndGetReal which does not seem optimal in terms of resource utilization?

package ExternDataEvaluation
  model TestModel
    extends Modelica.Icons.Example;
    parameter String fileName = Modelica.Utilities.Files.loadResource(
      "modelica://ExternData/Resources/Examples/test.json");
    parameter ExternData.JSONFile dat(fileName=fileName);
    parameter Real p_nominal = dat.getReal(varName="set1.gain.k")
      annotation(Evaluate=true);
    parameter Real q_nominal = loadAndGetReal(varName="set1.gain.k", fileName=fileName)
      annotation(Evaluate=true);
    parameter Real r_nominal = getReal(varName="set1.gain.k", json=dat.json)
      annotation(Evaluate=true);
    Real p(nominal=p_nominal) = 1;
    Real q(nominal=q_nominal) = 1;
    Real r(nominal=r_nominal) = 1;
  end TestModel;
  /* 
  Function loadAndGetReal not supported by OCT despite
  https://specification.modelica.org/maint/3.5/functions.html#external-objects
  "External objects may be a protected component (or part of one) in a function."
  And potentially memory-greedy as it loads the external file at each function call.
  */
  function loadAndGetReal
   input String varName "Key";
   input String fileName;
   output Real var "Variable value";
  protected 
   ExternData.Types.ExternJSONFile extObj = ExternData.Types.ExternJSONFile(fileName)
     "External file object";
  algorithm 
   var := ExternData.Functions.JSON.getReal(varName, extObj);
  annotation(__Dymola_translate=true);
  end loadAndGetReal;
  /* 
  Function getReal supported by OCT but does not trigger evaluation at compile 
  time with Dymola.
  */
  function getReal
    input String varName "Key";
    input ExternData.Types.ExternJSONFile json;
    output Real var "Variable value";
  algorithm 
    var := ExternData.Functions.JSON.getReal(varName, json);
  annotation(__Dymola_translate=true);
  end getReal;
end ExternDataEvaluation;

[beutlich]

So we wonder whether the construction of Types.ExternJSONFile json within the record ExternData.JSONFile is valid or not (although Dymola and Optimica do not complain). We noticed that a related issue is left open at modelica-3rdparty/Modelica_DeviceDrivers#93.

See https://doi.org/10.3384/ecp21181141, section 2.1.1:

As of MLS 3.5 (Modelica Language Specification), it is not yet fully specified, if external objects may be used in records¹.

Footnotes

1. MLS issue #2399 ↩

[beutlich]

We noticed that the class annotation annotation(__Dymola_translate=true) was added in recent versions of the library.

As for your second inquiry. It is also dealt by recent publication https://doi.org/10.3384/ecp21181141, section 2.1.3:

Reading structural parameters from external data resources [...] by functions getArraySize1D or getArraySize2D is not generally supported¹. Of the tested Modelica tools, it only works in SimulationX.

For Dymola, you need to explicitly call the read functions (of ExternData ≥ v3.0.0) when determing array dimensions.

model 'Dymola Workaround'
  parameter ExternData.JSONFile dat(
    fileName=Modelica.Utilities.Files.loadResource(
      "modelica://ExternData/Resources/Examples/test.json"));
  parameter Integer n = ExternData.Functions.JSON.readArraySize1D(
    fileName=Modelica.Utilities.Files.loadResource(
      "modelica://ExternData/Resources/Examples/test.json"),
    varName="vector") "Cannot use the record member function here";
  parameter Real array[n] = dat.getRealArray1D(varName="vector", n=n);
end 'Dymola Workaround';

Footnotes

1. MLS issue #2425 ↩

[beutlich]

Closing as duplicates.