RegPot Generators
In the Eclipse Free BIRD Tool project, we have the concept of a RegSeed , which contains the information of regulatory data models and transformations in an implementation-neutral language, and a RegPot which knows about technical implementation details such as particular languages(Java/Python/.Net) Frameworks (Ecore/XCore/Python Django/Java Spring) or even specific databases (Oracle/SQLServer) or custom frameworks. When we combine a RegSeed and a RegPot we get a usable implementation, where we can store data such as bonds and loans and execute transformations and report cell allocations.
The key part of a RegPot is its ‘generator’. A generator takes the data model and transformations written in the RegDNA standard and translates it into suitable elements for the target implementation. We call the target implementation a ‘Platform’.
RegDNA is a simple language, and like all languages it has grammar, and can be split into parts in a tree structure. For example we can think of the grammar of English and splitting sentences into ‘clauses’ and such as a noun clause.
We use a mature open source technology called Eclipse Xtext to define a grammar for the RegDNA language, and this also declares how we store the tree structure behind RegDNA text as a set of software ‘objects’.
To be a bit more tech specific we call the tree structure an "Abstract Syntax Tree (AST) " and in XText we store the parts of the AST using the Java Objects. We use different types(classes) of Objects to describe different parts of the sentences. These software objects and their classes are described using the Java Ecore implementation in the Eclipse Modelling Framework.
The grammar definition for RegDNA and its link to Ecore classes is defined in one file with the .xtext extension. The grammar definition is variant of EBNF which is a popular way to store language grammars.
The Ecore model referred to in this xtext file is available in a file here , we describe that file and the model and how we use it in more detail below. Amazingly these 2 files, the xtext and ecore file, describe pretty much all of RegDNA, and we can autogenerate language features and visualisation/navigation tools and data just from those 2 files using the XText framework and the Eclipse Modelling Framework.
In XText we can learn a lot about the language, and particularly how to use it to generate new artefacts, by looking at the fixed structure and relationships of the classes of the Java Objects. We can look at these relationships in an Ecore diagram. If you are not familiar with Ecore diagrams, you may be familiar with UML class diagrams which are 99% identical due to having a similar root (to be specific they they both come from OMG standards with a common root called ‘Meta Object Facility or MOF’)
RegDNA has 3 related parts, one for describing data models, one for describing transformations, and one for describing reports. The Ecore classes used for the part of RegDNA that describes data models are basically the same Ecore classes used for the meta-model of the mature Ecore standard itself, with anything we don’t need removed. It is a bit mind blowing that the Ecore meta-model is defined using Ecore, but this becomes quite demystified in practice when we start using it. There is lots of re-usable documentation on Ecore, its metamodel and how to use it with the Eclipse Modelling Framework (or the Python PYEcore framework). Hereis the Ecore model behind the part of RegDNA that is responsible for describing data models and we call it Ecore Lite. Where as Ecore meta-model has classes like EClass and EAttribute, Ecore Lite has ELClass and ELAttribute.
The image of the Ecore class diagram describing Ecore_lite is stored here and show below.
For the grammar of Reg DNA for representing data-models we just re-use a slightly simplified version of the mature Xcore XCore language , Xcore is designed to be a useful alternative to the XML/XMI format that is often used to store Ecore models. RegDNA does not include the XBase expression language used in XCore, because we want RegDNA to be simple, and we want users to be free to use a general purpose language of their choice to describe detailed logic of regulatory calculations (e.g Python or C#). Xcore’s grammar is described in an open source XText file, and we adapted that slightly for RegDNA.
The Ecore file describing Ecore Lite is here
In the same way that the Ecore metamodel can be described with Ecore, the Ecore Lite model can be described with Ecore Lite, and therefore by RegDNA…again mind-blowing, but don’t think about it too much unless you want a sore head!
Here is the RegDNA file that describes the Ecore Lite model is stored here
To describe generation transformation rules and regulatory reports we extend the Ecore_lite model with 3 other models, and amend the grammar to work with these also. For reference we show the models here, including a small model called Modules which helps us to split all RegDNA artefacts into smaller modules with clear dependencies. Note that the Module class is referred to in the generation_transformations, and reports models. Note that generation transformations, and reports models refer to items in the Ecore Lite model.
Images, ecore files, regdna files representing the models, and autogenerated documentation are all available here here are the diagrams
Generation Transformations, note the reference to the Module class in the bottom left
Reports model, note the reference to the Module class in the bottom left
These models are stored separately for display purposes and to ease understanding, and then they are merged into 1 model because this is a better technical artefact, that is usable in different technologies including the Python implementation of Ecore (PYEcore), as noted before it is stored here and this is what is referenced by the .xtext file.
The Generator code follows the same approach as all generators that work with the XText framework. Generators optionally makes uses of a Java templating technology called Xtend, which allows us to traverse a tree of Java objects (representing the Abstract Syntax Tree of a language text, such as RegDNA text) and output a text format. In the Desktop RegPot Example Generator ( available here) we generate text content in the Xcore standard, for another RegPot we might output something else (like Python Django models, or Apache Spark data frames) Here is the generator code for translating the RegDNA datamodel to Xcore.
Note that the code knows about the models, and so it knows that a package has classes, and classes have attributes and references, and that those have types. In fact, it allows code-complete when creating these templates because of its knowledge of the model. Note that this code is only about 40 lines. In this example we are creating XCore code from RegDNA ( which is largely the same as Xcore), but the creation other representations would probably not be much longer since we would be doing the same kind of traversal of the RegDNA Abstract Syntax Tree.
The generator reads in RegDNA text, typically in a directory called src, uses the Xtext grammar definition and its links to Ecore models to convert into a treee of Java objects, then traverses this tree using an XTend template to generate an alternative format (in our example case text bases Xcore files) and then stores the result , usually in a folder called src-gen.