CGP Tutorial

This page proposes a five-step methodology for how to develop a code generator based on the Overture Code Generation Platform (CGP). The methodology was originally proposed in the paper "Extending the Overture code generator towards Isabelle syntax", which is available from the proceedings of the 13th Overture workshop

TOC

1. Introduction

• Set up the CGP extension
• Add new nodes
• Transform the IR
• Generate syntax
• Validate the translation

Introduction

Below we describe the steps used to develop a new code generator.

The first step in the process is only necessary once. The remaining steps are done in an iterative manner. The approach is to start with a very small VDM example and go through the steps until the example is completely translated. Afterwards, the example should be expanded as little as possible and the steps repeated. This is done iteratively until the new CGP extension is complete.

Setup

Step 1 - Set up the CGP extension: Broadly speaking, the setting up of the CGP extension consists of subclassing the base code generator class – CodeGenBase – that is the common extension point of the CGP. The base code generator is responsible for driving the code generation and providing access to the Intermediate Representation (IR) and various settings. It is also responsible for storing data used and generated throughout the code generation process.

Next, it is necessary to construct a new template manager for the extension. This can be done by subclassing the base TemplateManager. This will provide access to the basic CGP TemplateStructure class which manages an initial collection of template locations. If additional template locations are necessary, the template manager can be used to configure them.

Finally, it is worth setting up a basic test infrastructure to drive the development process. This test infrastructure is responsible for processing a VDM source, passing the respective AST to the code generator and validating the translation outcome. In Overture there is a Test Framework you should use to set up your test.

New Nodes

Step 2 - Add new nodes: If the target language construct being translated is sufficiently different from those of the base IR, then it is likely that a code generator needs extra nodes. If necessary, these can be provided by extending the IR as described in AST Extension. Once the extension is defined, the AstCreator tool must be invoked in order to generate the extension nodes.

Transform

Step 3 - Transform the IR: Constructs that are not supported by the code generator need to be transformed away, using either base IR nodes or extended nodes generated in the previous step. This is done by implementing one or more necessary transformations. It is recommended that transformations be as small as possible so that each transformation only changes the IR in terms of one concept such as removing comprehensions or reordering definitions.

There are two kinds of transformations: A partial transformation is easier to apply but can only change the internal structure of a node, and a total transformation can change the node itself.

The following code snippet shows how to apply transformations.

List<ExtIrClassDeclStatus> transformed = new Vector<>();

for (IRClassDeclStatus status : untransformed)
{
  // Partial transformation applied directly
  PartialTransformationFoo t1 = new PartialTransformationFoo();
  generator.applyPartialTransformation(status, t1);

  // Total transformations need wrapping
  ExtIrClassDeclStatus eStatus = new ExtIrClassDeclStatus(status);
  TotalTransformationBar t2 = new TotalTransformationBar();
  generator.applyTotalTransformation(eStatus, t2);
  transformed.add(eStatus);
}

Generate

Step 4 - Generate syntax: Once the IR is in a form suitable for code generation, syntax can be generated using the syntax generation framework of the CGP. The CGP will automatically traverse the IR and generate code according to defined templates. All that needs to be done is to write those templates. This is done by creating the Apache Velocity template files for each of the nodes that is to be translated and updating the TemplateManager accordingly.

The following code snippets show you how to trigger the syntax generation

MergeVisitor pp = new MergeVisitor(...);
for (ExtIrClassDeclStatus status : transformed)
{
  AExtClassDeclCG irClass = status.getEClassCg();
  StringWriter sw = new StringWriter();
  irClass.apply(pp, sw);

  if (pp.hasMergeErrors())
  {
    result.add(new GeneratedModule(irClass.getBaseClass().getName(), irClass, pp.getMergeErrors()));
  } else if (pp.hasUnsupportedTargLangNodes())
  {
    result.add(new GeneratedModule(irClass.getBaseClass().getName(), new HashSet<VdmNodeInfo>(),   pp.getUnsupportedInTargLang()));
  } else
  {
    GeneratedModule generatedModule = new GeneratedModule(irClass.getBaseClass().getName(), irClass, sw.toString());
    generatedModule.setTransformationWarnings(status.getTransformationWarnings());
    result.add(generatedModule);
  }
}

Validate

Step 5 - Validate the translation: Validation of the translation should be done by means of the test infrastructure by comparing the translation output to a reference. This test should contain the minimum amount of VDM to exercise the latest bit of translation and a reference syntax for the result. The reference and actual result can be compared using string comparison or something more sophisticated, if necessary.