[codegen] Improved handling of records and record arrays in the code…

…gen.

  - Every record now gets
    - A dedicated "default" constructor function for allocating its members.
      - this is used when the variable is declared. It initializes members to
        their default values.
      - Note that this is different from default modelica record constructor.
        The modelica constructor is used explictly. The "default" one is used
        always on declaration.
    - A dedicated copy function for copying its members.
    - An array typedef to base_array.
    - A macro for creating an array of the record.
      - This uses the constructor function to create each element of the array.
      - This uses generic array function with the record size.
    - A macro for copying an array
      - This uses the copy function to copy each element of the array.
    - A macro getter to access elements of the array.
      - this uses the proper casting on the returned type.

  - We now keep bindings in TYPES_VARS in when creating a record declaration
    in SimCode structure. The bindings are used as default values by the "default"
    constructor.

  - Bindings of a derived record declaration are handled properly now.
    - If a record is a derived record with modifications, then bindings of the modifications
      are available in the declration scope.
           record A = B(k=exp); A r; //  'exp' needs to be passed to the constrcutor
        => A_construct(A* r, exp) {...; r->k = exp; ...}
      This means a default constrcutor will need those bindings passed to it explictly
      if it is to work correctly. Now every TYPES_VAR of a record has a boolean attribute
      specifiying whether it is bound from outside or not.

  - When collecting used functions in the BACKEND we traverse bindings of record members.
    - Right now this is needed because some functions get inlined and are not used by the
      equation system even though the model has instances of the record (since we generate
      constructors now we need them to be visiable even if the record is never used in a function).
    - The proper way to fix this would be to check if a record is used in a function and
      only then generate a constrcutor. OR another option is to also traverse record member
      bindings when doing inlining of functions (which is not done right now.)
      This seemed to be the simplest and probably quickest way even though we sometimes may
      generate a function even though it is not really used (very rarely).

  - Fixed copying of records in the simulation context.

    - If a record is being copied to simulation vars we need special handling.
      This is because there is no structure to simvars. The variables are scattered.
      Therefore there is now a copy function that writes a given record to the corresponding
      element in the simvars arrays.

      The generated function <RecName>_copy_to_vars(RecType* src, .../*simvars locations*/) is used to achive this.

  - Improved generation of component references in function context
    - We can now handle qualified crefs with subscripts anywhere in the cref.
    - This is done uniformly in functionContextCref template. Try to use this everywehre.

  - CONTEXT_FUNCTION now has a cref prefix
    - This is prepeneded to any cref generated in function context. This gives us some scope
      control. E.g. it is used in record constructor functions to make all local variable accesses
      prepended to a specific record name that is passed as input to the function.

   - Unknown size array handling.
    - Arrays of unknown size are arrays where at least one dimension is unknown.
    - If an unknown size array is declared with a default value, i.e. a binding,
        then it has fixed dimension sizes equal to the binding.
    - However if the unknown size array is declared with out a binding then it
        is considered flexible and can change sizes as needed.
    - The array representation in the runtime system ,i.e. base_array, now has a
        new memeber .flexible to signify this.

  - Handle array expressions of records i.e. {c1,c2,..} properly
    - Array expressions of records are handled a bit more cleanly now.
    - Do not sort record member variables in the record declaration. This is just absurd.

  - instDims in SimCodeFunction.VARIABLE is now list<DAE.dimension> not list<DAE.Expression>.
    This Helps to normalize handling od dimension expressions.
    - We really need to normalize the handling of dimension and subscript
    - Plus dimension and subscript related functions are not exchangable.
        Constructs are interpreted differntly as dimension compared to subscript
        and vice versa.
    - template dimension() now takes context as input. This is needed.

  - Some more minor fixes.
    - Casting of call expressions which return records is handled explictly now.
    - Change handling of casts.
      - Casts to modelica_integer are disabled for now.
      - Casts to records with different number of members are disabled for now.
    - Remove PARALLE_FUNCTION_CONTEXT
      - Use a boolean value in FUNCTION_CONTEXT instead. Most of what we do for these contexts is very similar.
    - Fix and rename sortIntN to countingSort.
      - The function can now handle non-unique lists.

OMCompiler/Compiler/BackEnd/BackendDAEOptimize.mo

@@ -1413,44 +1413,61 @@ algorithm
 end removeUnusedFunctions;
 
 public function copyRecordConstructorAndExternalObjConstructorDestructor
-  input DAE.FunctionTree inFunctions;
-  output DAE.FunctionTree outFunctions;
+  input DAE.FunctionTree inAllFunctionTree;
+  output DAE.FunctionTree outUsedFunctionTree;
 protected
-  list<DAE.Function> funcelems;
+  list<DAE.Function> allfuncs_list;
 algorithm
-  funcelems := DAEUtil.getFunctionList(inFunctions);
-  outFunctions := List.fold(funcelems,copyRecordConstructorAndExternalObjConstructorDestructorFold,DAE.AvlTreePathFunction.Tree.EMPTY());
-end copyRecordConstructorAndExternalObjConstructorDestructor;
+  outUsedFunctionTree := DAE.AvlTreePathFunction.Tree.EMPTY();
+  allfuncs_list := DAEUtil.getFunctionList(inAllFunctionTree);
 
-protected function copyRecordConstructorAndExternalObjConstructorDestructorFold
-  input DAE.Function inFunction;
-  input DAE.FunctionTree inFunctions;
-  output DAE.FunctionTree outFunctions;
-algorithm
-  outFunctions :=
-  matchcontinue (inFunction,inFunctions)
-    local
-      DAE.Function f;
-      DAE.FunctionTree funcs,funcs1;
-      Absyn.Path path;
-    // copy record constructors
-    case (f as DAE.RECORD_CONSTRUCTOR(path=path),funcs)
-      equation
-         funcs1 = DAE.AvlTreePathFunction.add(funcs, path, SOME(f));
-       then
-        funcs1;
-    // copy external objects constructors/destructors
-    case (f as DAE.FUNCTION(path = path),funcs)
-      equation
-         true = boolOr(
-                  stringEq(AbsynUtil.pathLastIdent(path), "constructor"),
-                  stringEq(AbsynUtil.pathLastIdent(path), "destructor"));
-         funcs1 = DAE.AvlTreePathFunction.add(funcs, path, SOME(f));
-       then
-        funcs1;
-    case (_,funcs) then funcs;
-  end matchcontinue;
-end copyRecordConstructorAndExternalObjConstructorDestructorFold;
+  for func in allfuncs_list loop
+    _ := match func
+      local
+        Absyn.Path path;
+        list<DAE.Var> var_list;
+        Option<DAE.Exp> obind;
+        DAE.Exp bind_exp;
+
+      case (DAE.RECORD_CONSTRUCTOR(path=path)) algorithm
+        // Add the constructor function.
+        outUsedFunctionTree := DAE.AvlTreePathFunction.add(outUsedFunctionTree, path, SOME(func));
+
+        // Now we traverse the bindings of the record members and look for function calls.
+        try
+          DAE.T_FUNCTION(funcResultType = DAE.T_COMPLEX(varLst=var_list)) := func.type_;
+        else
+          Error.addSourceMessage(Error.INTERNAL_ERROR,
+              {getInstanceName() + " got unxpected record constructor structure for  " + AbsynUtil.pathString(path)},
+              sourceInfo());
+          fail();
+        end try;
+
+        for var in var_list loop
+          obind := Types.getBindingExpOptional(var);
+          if isSome(obind) then
+            SOME(bind_exp) := obind;
+            (_, outUsedFunctionTree) := checkUnusedFunctions(bind_exp,inAllFunctionTree,outUsedFunctionTree);
+          end if;
+        end for;
+
+      then
+        ();
+
+      // copy external objects constructors/destructors
+      case DAE.FUNCTION(path = path) algorithm
+        if stringEq(AbsynUtil.pathLastIdent(path), "constructor") or
+           stringEq(AbsynUtil.pathLastIdent(path), "destructor") then
+          outUsedFunctionTree := DAE.AvlTreePathFunction.add(outUsedFunctionTree, path, SOME(func));
+        end if;
+      then
+        ();
+
+    end match;
+
+  end for;
+
+end copyRecordConstructorAndExternalObjConstructorDestructor;
 
 protected function removeUnusedFunctionsSymJacs
   input BackendDAE.SymbolicJacobians inSymJacs;

OMCompiler/Compiler/FFrontEnd/FBuiltin.mo

@@ -237,28 +237,28 @@ protected constant SCode.Element clockType = SCode.CLASS("Clock",commonPrefixes,
 // The builtin variable time. See also variableIsBuiltin
 protected constant DAE.Var timeVar = DAE.TYPES_VAR("time",
           DAE.dummyAttrInput,
-          DAE.T_REAL_DEFAULT,DAE.UNBOUND(),NONE());
+          DAE.T_REAL_DEFAULT,DAE.UNBOUND(),false,NONE());
 
 /* Optimica Extensions. Theses variables are considered builtin for Optimica: startTime, finalTime, objectiveIntegrand and objective */
 /* Optimica Extensions. The builtin variable startTime. */
 protected constant DAE.Var startTimeVar = DAE.TYPES_VAR("startTime",
           DAE.dummyAttrInput,
-          DAE.T_REAL_DEFAULT,DAE.UNBOUND(),NONE()) "- The `startTime\' variable" ;
+          DAE.T_REAL_DEFAULT,DAE.UNBOUND(),false,NONE()) "- The `startTime\' variable" ;
 
 /* Optimica Extensions. The builtin variable finalTime. */
 protected constant DAE.Var finalTimeVar = DAE.TYPES_VAR("finalTime",
           DAE.dummyAttrInput,
-          DAE.T_REAL_DEFAULT,DAE.UNBOUND(),NONE()) "- The `finalTime\' variable" ;
+          DAE.T_REAL_DEFAULT,DAE.UNBOUND(),false,NONE()) "- The `finalTime\' variable" ;
 
 /* Optimica Extensions. The builtin variable objectiveIntegrand. */
 protected constant DAE.Var objectiveIntegrandVar = DAE.TYPES_VAR("objectiveIntegrand",
           DAE.dummyAttrInput,
-          DAE.T_REAL_DEFAULT,DAE.UNBOUND(),NONE()) "- The `objectiveIntegrand\' variable" ;
+          DAE.T_REAL_DEFAULT,DAE.UNBOUND(),false,NONE()) "- The `objectiveIntegrand\' variable" ;
 
 /* Optimica Extensions. The builtin variable objective. */
 protected constant DAE.Var objectiveVar = DAE.TYPES_VAR("objective",
           DAE.dummyAttrInput,
-          DAE.T_REAL_DEFAULT,DAE.UNBOUND(),NONE()) "- The `objective\' variable" ;
+          DAE.T_REAL_DEFAULT,DAE.UNBOUND(),false,NONE()) "- The `objective\' variable" ;
 
 protected constant DAE.FuncArg argRealX = DAE.FUNCARG("x",DAE.T_REAL_DEFAULT,DAE.C_VAR(),DAE.NON_PARALLEL(),NONE());
 protected constant DAE.FuncArg argRealY = DAE.FUNCARG("y",DAE.T_REAL_DEFAULT,DAE.C_VAR(),DAE.NON_PARALLEL(),NONE());

OMCompiler/Compiler/FFrontEnd/FGraph.mo

@@ -561,6 +561,7 @@ algorithm
               DAE.ATTR(DAE.NON_CONNECTOR(), SCode.NON_PARALLEL(), variability, Absyn.BIDIR(), Absyn.NOT_INNER_OUTER(), SCode.PUBLIC()),
               ty,
               binding,
+              false,
               constOfForIteratorRange);
         r = lastScopeRef(g);
         g = FGraphBuildEnv.mkCompNode(c, r, FCore.BUILTIN(), g);

OMCompiler/Compiler/FFrontEnd/FNode.mo

@@ -610,7 +610,7 @@ algorithm
                   n,
                   DAE.ATTR(DAEUtil.toConnectorTypeNoState(ct),prl,var,dir,io,vis),
                   DAE.T_UNKNOWN_DEFAULT,
-                  DAE.UNBOUND(),NONE());
+                  DAE.UNBOUND(),false,NONE());
       then
         (nd, i);
 

OMCompiler/Compiler/FrontEnd/CevalFunction.mo

@@ -1720,7 +1720,7 @@ protected function makeFunctionVariable
   output DAE.Var outVar;
   annotation(__OpenModelica_EarlyInline = true);
 algorithm
-  outVar := DAE.TYPES_VAR(inName, DAE.dummyAttrVar, inType, inBinding, NONE());
+  outVar := DAE.TYPES_VAR(inName, DAE.dummyAttrVar, inType, inBinding, false, NONE());
 end makeFunctionVariable;
 
 protected function getBinding
@@ -2284,9 +2284,8 @@ protected
   DAE.Type ty;
   Option<DAE.Const> c;
 algorithm
-  DAE.TYPES_VAR(name, attr, ty, _, c) := inVar;
-  outVar := DAE.TYPES_VAR(name, attr, ty,
-    DAE.VALBOUND(inValue, DAE.BINDING_FROM_DEFAULT_VALUE()), c);
+  outVar := inVar;
+  outVar.binding := DAE.VALBOUND(inValue, DAE.BINDING_FROM_DEFAULT_VALUE());
 end updateRecordBinding;
 
 protected function updateRecordComponentBinding
@@ -2296,18 +2295,12 @@ protected function updateRecordComponentBinding
   input Values.Value inValue;
   output DAE.Var outVar;
 protected
-  DAE.Ident name;
-  DAE.Attributes attr;
-  DAE.Type ty;
-  DAE.Binding binding;
-  Option<DAE.Const> c;
   Values.Value val;
 algorithm
-  DAE.TYPES_VAR(name, attr, ty, binding, c) := inVar;
-  val := getBindingOrDefault(binding, ty);
+  outVar := inVar;
+  val := getBindingOrDefault(outVar.binding, outVar.ty);
   val := updateRecordComponentValue(inComponentId, inValue, val);
-  binding := DAE.VALBOUND(val, DAE.BINDING_FROM_DEFAULT_VALUE());
-  outVar := DAE.TYPES_VAR(name, attr, ty, binding, c);
+  outVar.binding := DAE.VALBOUND(val, DAE.BINDING_FROM_DEFAULT_VALUE());
 end updateRecordComponentBinding;
 
 protected function updateRecordComponentValue

OMCompiler/Compiler/FrontEnd/DAE.mo

@@ -813,6 +813,10 @@ uniontype Var "- Variables"
     Attributes attributes "attributes";
     Type ty "type";
     Binding binding "equation modification";
+    Boolean bind_from_outside "true if the binding has come from out of scope. This happens for derived record classes.
+                                   e.g. record A = B(k=exp). here the modification 'exp' is a binding from outside. We need
+                                   this infor to correctly generate default constructors at codegen time. This binding exp
+                                   will have to be supplied from outside for a default constructor of the owner record type";
     Option<Const> constOfForIteratorRange "the constant-ness of the range if this is a for iterator, NONE() if is NOT a for iterator";
   end TYPES_VAR;
 end Var;
@@ -884,13 +888,13 @@ constant Type T_COMPLEX_DEFAULT     = T_COMPLEX(ClassInf.UNKNOWN(Absyn.IDENT("")
 constant Type T_COMPLEX_DEFAULT_RECORD = T_COMPLEX(ClassInf.RECORD(Absyn.IDENT("")), {}, NONE()) "default complex with record CiState";
 
 constant Type T_SOURCEINFO_DEFAULT_METARECORD = T_METARECORD(Absyn.QUALIFIED("SourceInfo",Absyn.IDENT("SOURCEINFO")), Absyn.IDENT("SourceInfo"), {}, 1, {
-    TYPES_VAR("fileName", dummyAttrVar, T_STRING_DEFAULT, UNBOUND(), NONE()),
-    TYPES_VAR("isReadOnly", dummyAttrVar, T_BOOL_DEFAULT, UNBOUND(), NONE()),
-    TYPES_VAR("lineNumberStart", dummyAttrVar, T_INTEGER_DEFAULT, UNBOUND(), NONE()),
-    TYPES_VAR("columnNumberStart", dummyAttrVar, T_INTEGER_DEFAULT, UNBOUND(), NONE()),
-    TYPES_VAR("lineNumberEnd", dummyAttrVar, T_INTEGER_DEFAULT, UNBOUND(), NONE()),
-    TYPES_VAR("columnNumberEnd", dummyAttrVar, T_INTEGER_DEFAULT, UNBOUND(), NONE()),
-    TYPES_VAR("lastModification", dummyAttrVar, T_REAL_DEFAULT, UNBOUND(), NONE())
+    TYPES_VAR("fileName", dummyAttrVar, T_STRING_DEFAULT, UNBOUND(), false, NONE()),
+    TYPES_VAR("isReadOnly", dummyAttrVar, T_BOOL_DEFAULT, UNBOUND(), false, NONE()),
+    TYPES_VAR("lineNumberStart", dummyAttrVar, T_INTEGER_DEFAULT, UNBOUND(), false, NONE()),
+    TYPES_VAR("columnNumberStart", dummyAttrVar, T_INTEGER_DEFAULT, UNBOUND(), false, NONE()),
+    TYPES_VAR("lineNumberEnd", dummyAttrVar, T_INTEGER_DEFAULT, UNBOUND(), false, NONE()),
+    TYPES_VAR("columnNumberEnd", dummyAttrVar, T_INTEGER_DEFAULT, UNBOUND(), false, NONE()),
+    TYPES_VAR("lastModification", dummyAttrVar, T_REAL_DEFAULT, UNBOUND(), false, NONE())
   }, true);
 constant Type T_SOURCEINFO_DEFAULT  = T_METAUNIONTYPE({Absyn.QUALIFIED("SourceInfo",Absyn.IDENT("SOURCEINFO"))},{},true,EVAL_SINGLETON_KNOWN_TYPE(T_SOURCEINFO_DEFAULT_METARECORD),Absyn.IDENT("SourceInfo"));
 

OMCompiler/Compiler/FrontEnd/DAEUtil.mo

@@ -5958,6 +5958,7 @@ algorithm
               DAE.dummyAttrVar,
               DAE.T_UNKNOWN_DEFAULT,
               DAE.UNBOUND(),
+              false,
               NONE());
 end mkEmptyVar;
 

OMCompiler/Compiler/FrontEnd/Expression.mo

@@ -4839,7 +4839,7 @@ public function makeVar "Creates a Var given a name and Type"
   output DAE.Var v;
   annotation(__OpenModelica_EarlyInline = true);
 algorithm
-  v := DAE.TYPES_VAR(name, DAE.dummyAttrVar, tp, DAE.UNBOUND(), NONE());
+  v := DAE.TYPES_VAR(name, DAE.dummyAttrVar, tp, DAE.UNBOUND(), false, NONE());
 end makeVar;
 
 public function dimensionsMult
@@ -9902,7 +9902,7 @@ algorithm
 end dimensionsKnownAndEqual;
 
 public function dimensionKnown
-  "Checks whether a dimensions is known or not."
+  "Checks whether a dimension is known or not."
   input DAE.Dimension dim;
   output Boolean known;
 algorithm
@@ -9929,8 +9929,7 @@ algorithm
 end dimensionKnownAndNonZero;
 
 public function dimensionsKnownAndNonZero
-  "Checks whether all dimensions are known or not.
-  TODO: mahge: imprive this for speed"
+  "Checks whether all dimensions are known or not."
   input list<DAE.Dimension> dims;
   output Boolean allKnown;
 algorithm
@@ -9959,6 +9958,13 @@ algorithm
   end match;
 end dimensionUnknown;
 
+public function hasUnknownDims
+  input list<DAE.Dimension> dims;
+  output Boolean hasUnkown;
+algorithm
+  hasUnkown := List.mapBoolOr(dims, dimensionUnknown);
+end hasUnknownDims;
+
 public function subscriptEqual
 "Returns true if two subscript lists are equal."
   input list<DAE.Subscript> inSubscriptLst1;

OMCompiler/Compiler/FrontEnd/Inline.mo

@@ -1773,7 +1773,7 @@ algorithm
     case (_,_,_) then VarTransform.getReplacement(repl,cr);
     case (_,_,DAE.T_COMPLEX(complexClassType=ClassInf.RECORD(path),varLst=vars))
       equation
-        crs = List.map1(List.map(vars,Types.varName),ComponentReference.appendStringCref,cr);
+        crs = List.map1(List.map(vars,Types.getVarName),ComponentReference.appendStringCref,cr);
         exps = List.map1r(crs, VarTransform.getReplacement, repl);
       then DAE.CALL(path,exps,DAE.CALL_ATTR(ty,false,false,false,false,DAE.NO_INLINE(),DAE.NO_TAIL()));
   end matchcontinue;

OMCompiler/Compiler/FrontEnd/InnerOuter.mo

@@ -1264,6 +1264,7 @@ algorithm
       SCode.Visibility visibility;
       DAE.Type ty;
       DAE.Binding binding;
+      Boolean bndsrc;
 
       DAE.ConnectorType ct;
       SCode.Parallelism parallelism "parallelism";
@@ -1276,11 +1277,11 @@ algorithm
       equation
         // get the instance child
         r = FNode.childFromNode(node, FNode.itNodeName);
-        FCore.IT(DAE.TYPES_VAR(name, attributes, ty, binding, cnstForRange)) = FNode.refData(r);
+        FCore.IT(DAE.TYPES_VAR(name, attributes, ty, binding, bndsrc, cnstForRange)) = FNode.refData(r);
         DAE.ATTR(ct, parallelism, variability, direction, Absyn.INNER(), visibility) = attributes;
         attributes = DAE.ATTR(ct, parallelism, variability, direction, Absyn.OUTER(), visibility);
         // update the ref
-        r = FNode.updateRef(r, FNode.setData(FNode.fromRef(r),FCore.IT(DAE.TYPES_VAR(name, attributes, ty, binding, cnstForRange))));
+        r = FNode.updateRef(r, FNode.setData(FNode.fromRef(r),FCore.IT(DAE.TYPES_VAR(name, attributes, ty, binding, bndsrc, cnstForRange))));
         // env = switchInnerToOuterInGraph(env, inCr);
       then
         node;
@@ -1290,11 +1291,11 @@ algorithm
       equation
         // get the instance child
         r = FNode.childFromNode(node, FNode.itNodeName);
-        FCore.IT(DAE.TYPES_VAR(name, attributes, ty, binding, cnstForRange)) = FNode.refData(r);
+        FCore.IT(DAE.TYPES_VAR(name, attributes, ty, binding, bndsrc, cnstForRange)) = FNode.refData(r);
         DAE.ATTR(ct, parallelism, variability, direction, Absyn.INNER_OUTER(), visibility) = attributes;
         attributes = DAE.ATTR(ct, parallelism, variability, direction, Absyn.OUTER(), visibility);
         // update the ref
-        r = FNode.updateRef(r, FNode.setData(FNode.fromRef(r),FCore.IT(DAE.TYPES_VAR(name, attributes, ty, binding, cnstForRange))));
+        r = FNode.updateRef(r, FNode.setData(FNode.fromRef(r),FCore.IT(DAE.TYPES_VAR(name, attributes, ty, binding, bndsrc, cnstForRange))));
         // env = switchInnerToOuterInGraph(env, inCr);
       then
         node;