Mcp/0031 extend road map with prototyping plan

RationaleMCP/0031/99thDesignMeetingNotes.md

@@ -0,0 +1,41 @@
+# From the 99th Modelica Design Meeting minutes
+
+Agreement to proceed, but details unclear and need rapid progression.
+Tools should be able to handle both – either proper subset or major overlap.
+
+A proper subset might possible – in particular the drawbacks of quoting seem acceptable.
+
+Add the following to Modelica (seems possible in Dymola, OpenModelica, MapleSim, Wolfram SystemModeler)
+```
+foo()[i]  (…)[i]
+foo().r   (…).r
+```
+
+Separate tasks:
+* Name mapping (quoting etc)
+  - Separate name-space for automatically generated variables (perhaps not quoted).
+* Subset of grammar
+  - No extends.
+  - No package structure – or have user-defined functions with “.” in their name.
+  - Have user-defined function at start; but tight time-constraint – maybe in 2nd implementation phase in Emphysis
+* Annotating the source with source-location (hopefully a general solution for Modelica; also for Buildings-models generated by Python-script, CATIA kinematics, …). Not standardize too much since depends on tool-chain, but might suffice with container – saying that it is a source location, but leave details to the tool. The location should be usable by the producer of the flat Modelica (URI – or file name+line+column). LLVM have some smart variant. Unclear if we want locations on sub-expressions, and/or multiple source locations for one equation (e.g. for flow-equation based on connect-statements). Many structural parameters must be constant-folded away (e.g. conditions for unbalanced if-then-else).
+* How similar is FlatModelica to Modelica:
+  - Shall parameter/constant be removed in this format – since they are evaluated in other places, or kept since part of the result?
+  - Can a FlatModelica model be used as base-class? Should it behave the same in that case? Easiest to first forbid.
+  - Alias-elimination etc: Allowed, but not required.
+  - Are we losing some information, e.g. start-value priority
+  - Have “Flat Modelica” as a separate “class”?
+Reorganization of Modelica Specification – combined or separate step?
+
+How to separate FlatModelica and Modelica:
+* One possibility is a special annotation; and in that case un-mangle names etc.
+* Another possibility is to say “flatmodel” instead of “model” – and thus have a different grammar.
+
+Prioritize these steps with responsible:
+1.  Subset of grammar [M. Sjölund; working in Emphysis] (We already have algorithm part in Emphysis – but that is a different sub-set); might be best to have separate grammar in case it is not a full subset. Modelica grammar (current one is about LL(k)) with some removed productions – prefer to have in similar format. Also good to have proper Modelica-parser – based on exact grammar in specification.
+2.  Name mapping [H. Tidefelt]. Use quoted identifiers with standardized mangling scheme.
+3.  Subset of annotations [H. Olsson]. List sub-set of Modelica-annotations with specified semantics – especially the ones relevant for simulation. Others may be kept by tools. (Inline annotations for functions, vendor annotations? choices for values of components and Dialog? Experiment, HideResult, description-strings) – add source-location.
+4.  How to keep additional information, like start-value priority – that otherwise might be lost in FlatModelica format. What to preserve of Modelica_Services? How to handle ModelicaStandardTables reading from files when you don’t have files?
+How to work on these in a transparent way?
+* Different markdown files in MCP? Separate discussion from design in separate files (only summary)? Use labels for them? Have draft pull-request for branch with discussion? Have an official fork for each MCP? Comments on files
+  - Lennart to summarize what is possible for Github projects

RationaleMCP/0031/Flat-Modelica-requirements.md

@@ -0,0 +1,141 @@
+# Requirements on Flat Modelica
+This document should contain a first draft of the requirements on what Flat Modelica should and shouldn't be, to be used as a starting point for discussions at the upcoming design meeting in Regensburg on March 7-8, 2019.  It also contains a rough separation of things belonging to the three main parts of the Modelica Language Specification (MLS) that would come out of this MCP.
+
+## Flat Modelica features
+
+Things that the Flat Modelica format should support:
+- Basic scalar types with the same attributes as in Modelica.
+- Record and enumeration types.
+- Arrays.
+- Component declarations (with both public and protected visibility).
+  - Comment (@mtiller): It doesn't seem very flat if we preserve (potentially deeply nested) components instances
+  - Comment (@harmanpa): This should be changed to Component declarations of the scalar types, record and enumeration types, and arrays of each; not hierarchical. Component declarations have dot-notation names.
+  - Remove (that is, expand) records?
+- Equations and algorithms, both scalar, array-valued, and record-valued.
+- Optional source locations for use in error messages.
+   - Add this to the formal grammar?  If so, this will require some thought in order to be flexible and precise enough without interfering with too much with the rest of the grammar.
+   - Comment (@mtiller): Add this as standard annotations?  Already supported by current grammar.
+   - Comment (@harmanpa): Agreed as that also allows other meta-data. Unfortunately having `annotation(__something_LineNumber=12)` on every statement could become quite verbose.
+- Documentation strings.
+- Vendor-specific annotations.
+  - Comment (@mtiller): So annotations are present in the grammar but all annotations except vendor annotations are stripped.  If the concern is bloat from annotations that are no longer relevant in the flattened form (*e.g.,* graphical annotations), why not identify what gets excluded vs. what gets included? (see comment about source locations above)
+  - Comment (@harmanpa): I agree, I'd like to be able to put arbitary meta-data in.
+- All variabilities, but constant evaluation of parameters is not allowed.  (For example, this guarantees that all parameters will remain parameters if a Flat Modelica model is exported to an FMU for Model Exchange.)
+  - Comment (@mtiller): The semantics of this are unclear.  When you say constant evaluation, do you mean constant folding?  What about `final` parameters?  Should those be parameters in an FMU? I wouldn't think so.  It seems like they should just be `output`s.
+- List of all `parameter` variables that were treated as `constant` due to use in _parameter expressions_, the `Evaluate=true` annotation, or subject to constant evaluation during flattening for other reasons.
+  - `final` can probably be replaced by using normal equation instead of binding equation. meaning that binding eqations can always be treated as non-final.
+  - Comment (@mtiller): Why not just add an `Evaluate=true` annotation to indicate these.
+- Values for all constants, even those that have been inlined everywhere, since the values should be part of the simulation result.
+- Less restricted forms of record field access and array subscripting?
+  - Comment (@mtiller): Wouldn't we expect to restrict forms of record field access and array subscripting?  If so, then I would suggest saying "Restrict some forms of record field access and array subscripting" or "A more restrictive subset of record field access and array subscripting".
+ - Comment (@mtiller): Factor out constant subexpressions to unique variables?  Perhaps filter all literals (scalars and arrays) completely out of the flattened form and provide them in a separate file?
+  - Needs Standardized naming for introduced intermediate variables.
+- Expressions for all variables that were treated as aliases during flattening, specifying the variable that it is an alias of and the sign of the relationship
+  - Comment (@mtiller): Doesn't the expression already tell us all that (*e.g.,* `b = -a`...`-a` is an expression and it tells us that `b` is an alias of `a` with the opposite sign?  Or did you want something more explicit?  Should there be a special form of equation (perhaps in the form of an assignment statement) that can be used to indicate solved equations in general, and alias relations in particular?
+  - Comment (@harmanpa): Yes syntactically it is just that expression, however that isn't necessarily the expression that the alias came from. e.g. the model might contain `a = c` and `a = -b` but we decide to keep `b`, so in our aliases section we store `b = -a` and `b = -c`.
+- Function declarations that are utilised in the model.
+  - Comment (@mtiller): Do we flatten the functions?  I say that because functions can use features like `extends` or `redeclare` in their definitions.  Presumably we want all that removed in a flattened form, no?  Functions should probably be allowed to have arguments of array and record type.
+  - Comment (@harmanpa): Yes. Additionally we might have multiple versions of functions, because we might call a function with different dimension inputs. We need a naming convention for this.
+
+Examples of things that should be gone after flattening and shouldn't exist in Flat Modelica:
+- Complex classes that may contain equations.
+- Connectors.
+- Conditional components.
+- Un-balanced `if`-equations.
+- Non-literal array dimensions (except in functions).
+- Packages.
+- Connect equations.
+- Graphical and documentation annotations.
+- All redeclarations
+- Anything overloaded
+- Imports
+- Extends clauses?
+- Hierarchical modifications?
+- Package structure?
+  - If you keep components, restrict class definitions to a flat global namespace to avoid having to apply lookup rules.
+
+## New organization of MLS
+This MCP proposes separation of the MLS into the three parts outlined below.
+
+### Shared features: Expressions and functions
+Content of that should go into this section:
+- Built-in functions.
+- Expressions.
+- External functions.
+- User-defined functions.
+- Restricted forms of record field access and array subscripting (some or all of the restrictions may be lifted for Flat Modelica).
+- Unit expressions.
+- Syntax for synchronous language elements and state machines.
+
+### Flat Modelica
+Content of that should go into this section:
+- Overview of execution model for hybrid differential-algebraic equations.
+- Simulation initialization and priority of non-fixed start attributes.
+- Event generation, event iteration, and other discrete-time behavior.
+- Semantics of synchronous language elements and state machines.
+- Solving of mixed systems.
+- Hints for inlining, state selection, etc.
+
+### Modelica
+Content of that should go into this section:
+- All sorts of restricted classes and rules for how they may be used.
+- Inheritance, modification, and redeclaration.
+- Connectors, including stream connectors.
+- Connect equations.
+- Overloaded operators.
+- Global and local balancing of equations and variables.
+- Instantiation.
+- Flattening.
+
+## An alternative view of grouping Modelica semantics (@mtiller)
+
+One advantage to this approach would be to organize the semantics of Modelica by when they apply.  In this sense, I'd like to see a description of:
+
+- Semantics needed in order to create a Modelica editor
+  - Package structure
+  - Name lookup
+  - Graphical appearance
+  - Connection semantics
+  - Inheritance
+- Semantics needed to represent mathematical structure and behavior
+  - Variables
+  - Equations
+  - Functions
+
+From this perspective, expressions can almost be treated as "pass through".  The editor doesn't really need to
+interpret them in any way. For the most part they just pass through to the flattened form (but with some
+potential simplifications or restrictions, as described above).
+
+## Comments from regensburg design meeting (added by @HansOlsson)
+
+Need to support start-value priority (source location or special built-in operator,
+either complete source location or just needed hiearachy level).
+Do after Link�ping meeting?
+Need to design grammar: Sub-set of current Modelica or some additions?
+
+One major issue is variable declaration as we have e.g. `a[1].b` as a variable,
+and also need record variables.
+One possibility would be quoted identifiers - but can clash.
+  Can we always quote a quoted one without ambiguity? Just use anything - even base64.
+Another possibility is extended grammar - but looks ambiguous.
+E.g. `a[1].b` could be either a variable, or access to member of array of records.
+
+For declarations we need to know if array dimensions are part of variable or not.
+
+Might need accessing record elements using `.` and `[` for non record variables,
+  e.g., `f(x)[1]` and `f(x).x[1]`
+
+Array of components. Repeated equations (using quoted identifiers to be clear here):
+  `'a[1].x'='a[1].y';`
+  `'a[2].x'='a[2].y';`
+But should be possible to preserve as array equations in some way (see new frontend for OM)?
+
+Need to test-implement? Current Flat-Modelica in e.g. Dymola is different:
+e.g., records are not done like this (record values are missing), functions are missing.
+
+Pull-requests:
+- Variable naming
+- Source location?
+- Handling modifier level for start-attribute, e.g.: Real p(start=modifierLevel(3, 5)); or extension
+
+Some minor comments.

RationaleMCP/0031/MoM/2019-10-01_FlatModelica_100thDesignMeetingNotes.md

@@ -0,0 +1,112 @@
+# Session on Flat Modelica at the 100th Modelica Design Meeting
+
+## Agenda
+* Introduction of eFMI Equation Code
+* Align goals of eFMI with Flat Modelica MCP of MAP_Lang
+* Discuss the prepared simple example: PID controller
+* Next steps
+
+
+## Introduction of eFMI EqCode
+
+See ppt slides (https://emphysis.org/svn/EMPHYSIS/trunk/30_Work_Documents/WP3_eFMI-Standardization/WP3_1_Working_group_Eq-Code/eFMI_EqCode-Summary.pptx).
+
+## Align goals of eFMI with Flat Modelica MCP of MAP_Lang
+Comments:
+Francesco: functions are crucial
+Martin: Could be realized as call to C-function or inline.
+Gerd: Flattened Modelica might need few extension, e.g. annotations for index reduction and tearing.
+
+* EqCode is not the same as Flat Modelica, but we're heading in the same direction.
+* The scope of EqCode is more narrow, never-the-less it makes sense to work on this in a joint effort considering EqCode as a MVP (minimal viable product) towards Flat Modelica.
+
+## Discuss the prepared simple EqCode example: PID controller
+Below only the first lines of code discussed during the meeting are listed. The file will be made available once we have decided how to manage this work on github.
+
+```
+flat_model PID
+
+  type _Time Real (final quantity="Time", final unit="s");
+
+  parameter _Time 'period' = 0.1 "Period of clock (defined as Real number)";
+  constant Boolean 'enbDiscretize' = true "discretize system?";
+  input Real 'r' "Connector of Real input signal";
+  input Real 'y' "Connector of Real input signal";
+  parameter _Time 'periodicClock.period' = period "Period of clock (defined as Real number)";
+```
+
+### flat_model
+* agreed to have a new kind of class
+
+### type
+enumerations require type declarations, but types could be limited to enumerations.
+Having types requires handling of modifiers, but could be avoided by allowing modifications only for build-in attributes.
+
+* Type declarations should be supported to avoid code repetition and improve readability.
+* Support predefined scalar types: Real, Integer, Boolean.
+
+### type names
+* Should have leading underscore ("_ ") to avoid confusion with variable names and keywords, see name-mapping and related discussion pull 2389.
+
+### parameter
+How to treat a parameter without a value?
+In this case the start value will be used, which is zero for Real.
+Tools could restrict parameters having to have a value.
+What is the meaning of a start value for parameters in the context of eFMI?
+
+Side note (added after the meeting):
+On ECU there are different kinds of initializations:
+- After power on the device is launched (start_up) all memory is brought into a state that is always the same no matter which state other devices are in. This means all inputs must be ignored.
+- After the launch the device can be initialized.
+- Anytime during operation the device can be re-initialized, e.g. after a certain event.
+
+If the semantics of Moldelica cannot be mapped in reasonable way parameters could be handled by applying a restriction that requires parameters to have values for EqCode to avoid deviation of Flat Modelica from Modelica.
+* topic should be revisited.
+
+### variable names
+* Always use quoted names according to name-mapping
+Example has been adopted accordingly.
+
+## Requirements derived from the discussion
+* Shall be a self-contained definition in a single file.
+* Shall not be limited to scalars to enable efficient handling of multibody and similar.
+* Flat Modelica shall be fully compatible with Modelica, no exceptions.
+* EqCode can apply additional restrictions to Flat Modelica.
+
+## Issues from github and emails
+
+### [Michael Tiller] from github
+Although I must say I disagree with the premise that Flat Modelica should be a subset of Modelica. It certainly doesn't need to be. You could reuse a huge amount of the grammar between them without having to prescribe this requirement.
+* Having Flat Modelica as a separate language will sooner or later lead to compatibility problems. Therefore we see good reasons to consider Flat Modelica as a subset of Modelica.
+
+
+### [Hans] from github
+I would say that there are two alternatives - either we standardize such a mangling syntax for names, or we allow the declaration of variables with hierarchical names in Flat Modelica.
+* see above variable names
+
+### [Henrik] from email
+1) Flat Modelica needs to fulfill the needs to process full Modelica.  The Equation Code Language should then impose restrictions on what Flat Modelica it allows.  As an obvious example of this, Flat Modelica needs to allow functions with algorithmic bodies and side effects.  Other things that come to mind here include algorithms, looping constructs, if-equations, when-equations, when-statements, etc. 
+* Agreed, see above Requirements.
+
+2) There are many Modelica constructs that are not deeply involved in the complicated flattening process, and that need to be allowed also in Flat Modelica in order to preserve structure that is necessary for efficient handling of the equations.  These include record and array types. 
+* Agreed, see above Requirements.
+
+3) The "Identifier" section should probably be merged into the thing I did here:
+https://github.com/modelica/ModelicaSpecification/pull/2389
+I think the main topic here is the relation to array and record types.  Then, to avoid confusion, we need to make sure that example code isn't using any other form of identifier.  (To me, the examples under "Identifier" and "Variable name" seem to contradict each other.)
+
+Questions
+a) What is an "explicit declaration"?  What would a non-explicit ("reference") look like?
+An explicit variable declaration is one that has the type declaration inline, vs. using referring to type name declared elsewhere.
+
+b) Isn't "statically evaluatable" just the same as a constant expression?  (No need to introduce new terms.)
+Good point. Will be adopted.
+
+c) Why would "declare before use" be needed in a declarative setting such as Flat Modelica?
+To be discussed.
+
+d) Why remove "each"?  This would remove structural information that would just be tedious to recover for the tool processing Flat Modelica.
+To be discussed.
+
+e) Why restrict "der" to only operate on a "free variable"? 
+To be discussed with Kai.