Skip to content

MOGLiCC Tutorial Part D

Jump to bottom
iks github user edited this page Jun 21, 2016 · 56 revisions

D) Clean Coding for both template and generation code: using subtemplates and white space

Currently, we have two Main.tpl files which contain code that is quite identical. From the Clean Code point of view we violated the DRY principle: Don't repeat yourself. We did so by copying template code. Now let's see how we can get clean. But was does clean mean? Well, see Clean Coding Cosmos. In short: Clean Code means code that is simple to understand, quick to change and easy to test. Or even shorter: clean means efficient. By avoiding template code duplication we will achieve more efficiency and also by formatting SQL statements (which can be very long and ugly to read) in a human friendly way. The latter is achieved by setting white space in the right places. This is bit tricky, since Velocity simply cuts all white space.

1 - At first, we will extract code from the FileMaker's main template which we like to use for the LineInserter. We create the file .input/VelocityClassBasedFileMaker/BundledInsertSQLScripts/createClassSpecificInsertStatements.tpl. As content we take the whole body of .input/VelocityClassBasedFileMaker/BundledInsertSQLScripts/Main.tpl. Thereafter replace the body of the Main.tpl by

#parse("createClassSpecificInsertStatements.tpl")

Delete the InsertSQL result directory, restart MOGLiCC and check that all result files are generated without errors. If so, our refactoring has been ok.

2 - Now, we create the folder _./input/VelocityClassBasedFileMaker/commonSubTemplates. If you start MOGLiCC now, you will get an error report, that no main template is found in _commonSubTemplates. MOGLiCC interprets this folder as input artefact and we have to configure the FileMaker that this is not the case. Do so by assuring that the file ./input/VelocityClassBasedFileMaker/generator.properties exists containing the following case sensitive line:

_commonSubTemplates=ignore

Move createClassSpecificInsertStatements.tpl into the new folder, delete the InsertSQL result directory, restart MOGLiCC and check that all result files are generated. But wait, there is an error: the subtemplate couldn't be found from the VelocityClassBasedFileMaker/BundledInsertSQLScripts/Main.tpl template, because its reference is not yet adapted to the new location. Ah ok, the new location of the subtemplate must be defined in the Velocity parse-instruction. Modify the template body of Main.tpl to:

#parse("_commonSubTemplates/createClassSpecificInsertStatements.tpl")

Again, delete the InsertSQL result directory, restart MOGLiCC and check that all result files are generated without errors.

3 - With its new location the subtemplate could be used now from other artefacts of the FileMaker. However, we want to use it in the LineInserter's main template. For this purpose we have to move the directory _./input/VelocityClassBasedFileMaker/commonSubTemplates one folder upwards. There (in the input root directory), it is accessible for templates of all generators. Note, that the reference of the subtemplate in the template file remains unchanged this time, because both the input directory itself and all containing plugin subdirs have been registered at the Velocity Engine in the same way. Now, the subtemplate can be introduced to the LineInserter's main template, by replacing the same dublicated code. The body of .\input\VelocityModelBasedLineInserter\SQLInsertScript\Main.tpl should look like that:

#set( $classDescriptorList = $model.getClassDescriptorList() )
#foreach( $classDescriptor in $classDescriptorList )

    #if( ! $classDescriptor.doesHaveAnyMetaInfosWithName("skip") )

		#parse("_commonSubTemplates/createClassSpecificInsertStatements.tpl")
		
	#end	
#end

Once again, delete the InsertSQL result directory, restart MOGLiCC and check that all result files are generated without errors. If so, our refactoring has been once again ok.

4 - Keeping the DRY principle is one reason for using subtemplates. Another one is to make the complex template code more understandable. The code of our subtemplate is really hard to read. To increase the maintainablity, you should keep templates simple. We will now extract a subtask from our subtemplate into a "subsubtemplate". We extract the code of the inner foreach loop into a new file called fillArrayOfCommaSeparatedListOfTableValues.tpl. We create this file in the same directory _input/commonSubTemplates. The subtemplate createClassSpecificInsertStatements.tpl is now reduced to:

#set( $attributeDescriptorList = $classDescriptor.getAttributeDescriptorList() )
#set( $commaSeparatedListOfAttributes = "" )
#set( $arrayOfCommaSeparatedListOfTableValues = [])

#foreach ($attributeDescriptor in $attributeDescriptorList)
	
	#set( $columnName = $attributeDescriptor.name)	
	#set( $fieldType = $attributeDescriptor.getMetaInfoValueFor("fieldType") )
	#set( $metaInfoList = $attributeDescriptor.getMetaInfosWithNameStartingWith("tableValue") )
	#set( $indexOfInsertStatements = -1 )

	#parse("_commonSubTemplates/fillArrayOfCommaSeparatedListOfTableValues.tpl")
		
	#set( $commaSeparatedListOfAttributes = $commaSeparatedListOfAttributes + $columnName + ", ")
		
#end

# Cut the last comma from the lists
#set( $numberOfChars = $commaSeparatedListOfAttributes.length() - 2 )
#set( $commaSeparatedListOfAttributes = $commaSeparatedListOfAttributes.substring(0, $numberOfChars) )

# loop that generates for each insert statement one output line
#foreach ($element in $arrayOfCommaSeparatedListOfTableValues)
	#set( $numberOfChars = $element.length() - 2 )
	#set( $element = $element.substring(0, $numberOfChars) )
	INSERT INTO $classDescriptor.name ($commaSeparatedListOfAttributes) VALUES ($element);  
#end

Please note, that although the two template are located in the same folder, the parent folder must be provided when the subsubtemplate is referenced! It starts to get boring, but delete the InsertSQL result directory, restart MOGLiCC and check that all result files are generated without errors. If so, our refactoring has been ok.

5 - Ok, both subtemplate looks passably now. However, subsubtemplate createClassSpecificInsertStatements.tpl is supposed to grow with each new fieldType used in the model data. Hence, we will simplify the template code a bit more. We create the new subsubsubtemplate _input/commonSubTemplates/addTableValueToInsertStatement.tpl with the following content:

# the following velocity instructions find the insertStatement to which the current tableValue belongs
#set( $indexOfInsertStatements = $indexOfInsertStatements + 1 )
#if ( $arrayOfCommaSeparatedListOfTableValues.size() <= $indexOfInsertStatements)
	# a new element for an additional insert statement has to be added to the array 
	#set( $forgetThisValue = $arrayOfCommaSeparatedListOfTableValues.add("") )
#end
#set( $commaSeparatedListOfTableValues = $arrayOfCommaSeparatedListOfTableValues.get($indexOfInsertStatements) )

# the following velocity instructions add the tableValue to the insertStatement where it belongs to
#set( $commaSeparatedListOfTableValues = $commaSeparatedListOfTableValues + $tableValue + ", ")
#set( $forgetThisValue = $arrayOfCommaSeparatedListOfTableValues.set($indexOfInsertStatements, $commaSeparatedListOfTableValues) )

To apply this new subsubsubtemplate, we modify our subsubtemplate to

# loop for each insert Statement to build for this class
#foreach ($metaInfo in $metaInfoList)

	#set( $tableValue = $metaInfo.value )
	
	#if( $fieldType.equals("alphanumeric") )
		#set( $tableValue = "'" + $tableValue + "'" )
	#elseif( $fieldType.equals("number") )
		# do nothing
	#else
		Unkown fieldtype ($fieldType) used for attribute $attributeDescriptor.name of class $classDescriptor.name !
	#end

	#parse("_commonSubTemplates/addTableValueToInsertStatement.tpl")
	
#end

Check that this refactoring work. You know what do: delete, restart, read...

6 - Ok, now let's focus on the readability of the generated SQL files. If the number of attributes (i.g. number of columns in the database table) increases, the insert statements will become hard to read. To avoid this, we firstly create the new subtemplate ./input/commonSubTemplates/addSpacesToMeetStringLength.tpl with the following content:

# This template takes the two String values 'string1' and 'string2'
# and adds spaces to the shorter one so that both become equal in length

#set( $charCounter = 0 )

#if( $string1.length() > $string2.length() )

    #set( $charList = $string1.toCharArray() )

    #foreach( $char in $charList )

        #set( $charCounter = $charCounter + 1 )

        #if( $charCounter > $string2.length() )

            #set( $string2 = $string2 + " " )

        #end

    #end

#end

#if( $string1.length() < $string2.length() )

    #set( $charList = $string2.toCharArray() )

    #foreach( $char in $charList )

        #set( $charCounter = $charCounter + 1 )

        #if( $charCounter > $string1.length() )

            #set( $string1 = $string1 + " " )

        #end

    #end

#end

Furthermore, we create the new subtemplate ./input/commonSubTemplates/createInsertStatement.tpl

#set( $valuesText = "VALUES" )
#set( $tableName = $classDescriptor.name )

#set( $string1 = $valuesText )
#set( $string2 = $tableName )
#parse("_commonSubTemplates/addSpacesToMeetStringLength.tpl")
#set( $valuesText = $string1 )
#set( $tableName = $string2 )

INSERT INTO $tableName ($commaSeparatedListOfAttributes) 
            $valuesText ($commaSeparatedListOfTableValues);

Finally, we change in the output generating line in the subtemplate createClassSpecificInsertStatements.tpl to

#parse("_commonSubTemplates/createInsertStatement.tpl")

Restart MOGLiCC and considering the generation result: we observe that all leading spaces in the insert line of the template has been removed. This is a feature (like it or not) of Velocity. To allow white space formatting, MOGLiCC comes with the white space marker '. This marker makes Velocity to preserve spaces that follows it, but cutting the marker itself. Thus, you can use it to move the visible start of a line to the right. We do so by modifing the two template lines of the insert statements to

 INSERT INTO $tableName ($commaSeparatedListOfAttributes) 
'            $valuesText ($commaSeparatedListOfTableValues);

Restart MOGLiCC and considering the generation result again: the space in the template file before INSERT INTO did not reach the output result as well as the space marker in the following line, but the spaces behing the space marker has been preserved. This results in a more readable form of the generated input statements.

7 - The next step is to apply the template addSpacesToMeetStringLength.tpl not only to the pair of strings "

/ VALUES", but also to the pairs "column names / table values". This will increase the complexity of our template code, because the variable $commaSeparatedListOfAttributes is currently identical for all insert statements. This must change to reach our goal. Unfortunately a bigger refactoring is needed. Firstly, we change the content of our subtemplate createClassSpecificInsertStatements.tpl to 
#set( $attributeDescriptorList = $classDescriptor.getAttributeDescriptorList() )
#set( $arrayOfCommaSeparatedListOfTableValues = [])
#set( $arrayOfCommaSeparatedListOfAttributeNames = [])  # needed due to space formatting which may be different for each insert statement

# loop that collect all attribute names and tableValues
#foreach ($attributeDescriptor in $attributeDescriptorList)
	
	#set( $columnName = $attributeDescriptor.name)	
	#set( $fieldType = $attributeDescriptor.getMetaInfoValueFor("fieldType") )
	#set( $metaInfoList = $attributeDescriptor.getMetaInfosWithNameStartingWith("tableValue") )
	#set( $indexOfInsertStatements = -1 )

	#parse("_commonSubTemplates/fillArraysOfCommaSeparatedLists.tpl")
		
#end

# loop that generates for each insert statement one output line
#set( $index = 0 )
#foreach ($commaSeparatedListOfTableValues in $arrayOfCommaSeparatedListOfTableValues)

	#set( $numberOfChars = $commaSeparatedListOfTableValues.length() - 2 )
	#set( $commaSeparatedListOfTableValues = $commaSeparatedListOfTableValues.substring(0, $numberOfChars) )    # Cut the last comma from the list
	
	#set( $commaSeparatedListOfAttributeNames = $arrayOfCommaSeparatedListOfAttributeNames.get($index) )
	#set( $numberOfChars = $commaSeparatedListOfAttributeNames.length() - 2 )
	#set( $commaSeparatedListOfAttributeNames = $commaSeparatedListOfAttributeNames.substring(0, $numberOfChars) )   # Cut the last comma from the list

	#parse("_commonSubTemplates/createInsertStatement.tpl")
	
    #set( $index = $index + 1 )

#end

Please note several issues: 1. The new array arrayOfCommaSeparatedListOfAttributeNames has been introduced. 2. The old subsubtemplate fillArraysOfCommaSeparatedListsOfTableValues.tpl has to be renamed to fillArraysOfCommaSeparatedLists.tpl, because it fills both arrays now. 3. When creating the insert statement, the commaSeparatedListOfAttributeNames is read from the array.

Furthermore, we adapt our subsubtemplate, now named fillArraysOfCommaSeparatedLists.tpl:

# loop for each insert Statement to build for this class
#foreach ($metaInfo in $metaInfoList)

    #set( $tableValue = $metaInfo.value )
	
    #if( $fieldType.equals("alphanumeric") )
        #set( $tableValue = "'" + $tableValue + "'" )
    #elseif( $fieldType.equals("number") )
        # do nothing
    #else
        Unkown fieldtype ($fieldType) used for attribute $attributeDescriptor.name of class $classDescriptor.name !
    #end
	
    # make strings of table value and column name equal in length
    #set( $string1 = $columnName )
    #set( $string2 = $tableValue )
    #parse("_commonSubTemplates/addSpacesToMeetStringLength.tpl")
    #set( $columnName = $string1 )
    #set( $tableValue = $string2 )  

    #parse("_commonSubTemplates/addAttributeNameAndTableValueToInsertStatement.tpl")

#end

Please note two issues: 1. Here the string length adaptation for attribute names and table values is implemented. 2. The subsubsubtemplate addTableValueToInsertStatement.tpl has been renamed to addAttributeNameAndTableValueToInsertStatement.tpl because it handles both attribute names and table values now.

Finally, we adapt our subsubsubtemplate, now named addAttributeNameAndTableValueToInsertStatement.tpl:

# The following velocity instructions find the insertStatement to which the current tableValue and columnName belong.
# An insert statement is here represented by an element in the first array and the element of the same index in the second array!

#set( $indexOfInsertStatements = $indexOfInsertStatements + 1 )

#if ( $arrayOfCommaSeparatedListOfTableValues.size() <= $indexOfInsertStatements)
	# a new element for an additional insert statement has to be added to the array 
	#set( $forgetThisValue = $arrayOfCommaSeparatedListOfTableValues.add("") )     # tricky: declaring the Velocity variable forgetThisValue avoids an unwanted output !
	#set( $forgetThisValue = $arrayOfCommaSeparatedListOfAttributeNames.add("") )  # tricky: declaring the Velocity variable forgetThisValue avoids an unwanted output !
#end

# the following velocity instructions add the tableValue to the insertStatement where it belongs to
#set( $commaSeparatedListOfTableValues = $arrayOfCommaSeparatedListOfTableValues.get($indexOfInsertStatements) )
#set( $commaSeparatedListOfTableValues = $commaSeparatedListOfTableValues + $tableValue + ", ")
#set( $forgetThisValue = $arrayOfCommaSeparatedListOfTableValues.set($indexOfInsertStatements, $commaSeparatedListOfTableValues) )

# the following velocity instructions add the attribute names to the insertStatement where it belongs to
#set( $commaSeparatedListOfAttributeNames = $arrayOfCommaSeparatedListOfAttributeNames.get($indexOfInsertStatements) )
#set( $commaSeparatedListOfAttributeNames = $commaSeparatedListOfAttributeNames + $columnName + ", ")
#set( $forgetThisValue = $arrayOfCommaSeparatedListOfAttributeNames.set($indexOfInsertStatements, $commaSeparatedListOfAttributeNames) )

You already know what has to be done: restart MOGLiCC and check that all result files are generated without errors. If so, our refactoring has been ok and the generated insert statements look very readable.

8 - The last step to go in this tutorial part is to use empty lines, in order to separate blocks of code in the big insert.sql file. This is simply done, by setting the space marker in the .input\VelocityModelBasedLineInserter\SQLInsertScript\Main.tpl into a separate line. Modify the body of this template to

#set( $classDescriptorList = $model.getClassDescriptorList() )
#set( $isFirstClass = true )
#set( $classCounter = 0 )
#set( $statementCounter = 0 )
#foreach( $classDescriptor in $classDescriptorList )

    #if( ! $classDescriptor.doesHaveAnyMetaInfosWithName("skip") )

        #set( $classCounter = $classCounter + 1 )
	
        #if( isFirstClass == true )

            #set( $isFirstClass = false )
            
        #else

	        # add two empty lines as block separator
            ' 
            '
			
		#end
		
		#set( $attributeDescriptorList = $classDescriptor.getAttributeDescriptorList() )
		#set( $attributeDescriptor = $attributeDescriptorList.get(0) )
		#set( $metaInfoList = $attributeDescriptor.getMetaInfosWithNameStartingWith("tableValue") )
		#set( $numberInsertStatementsForTable = $metaInfoList.size())
		#set( $statementCounter = $statementCounter + $numberInsertStatementsForTable )
		
		PRINT Inserting $numberInsertStatementsForTable datasets for table $classDescriptor.name...		
		#parse("_commonSubTemplates/createClassSpecificInsertStatements.tpl")
		
	#end	
#end

'
'
PRINT Done with $statementCounter statements for $classCounter data tables.

Restart MOGLiCC and you find - an error, ARRG. What the hell is wrong now? Can you see it? No? Then study the error report. It points to "isFirstClass" at Main.tpl[line 18, column 14]. Well, the Velocity symbol $ that indicates a variable is missing. Add it, restart and find the following result:

PRINT Inserting 2 datasets for table REMITTER...
INSERT INTO REMITTER (ID , NAME          )
            VALUES   (123, 'Peter Pepper');
INSERT INTO REMITTER (ID  , NAME          )
            VALUES   (2345, 'Sally Salt'  );


PRINT Inserting 3 datasets for table ORDER...
INSERT INTO ORDER  (ORDERNUMBER, REMITTERID)
            VALUES (667788     , 123       );
INSERT INTO ORDER  (ORDERNUMBER, REMITTERID)
            VALUES (991166     , 2345      );
INSERT INTO ORDER  (ORDERNUMBER, REMITTERID)
            VALUES (229911     , 123       );


PRINT Done with 5 statements for 2 data tables.

Some final notes to this tutorial part. You may see, that using subtemplates has two beneficial effects:

  1. It does help to make the template code easier to understand (although this needs some experience - doing it wrong makes it worse). Good practice is that each template is such simple that its content is understood easily. When breaking down the complexity, pay much attention to give subtemplates a good name that tells what happens in there.
  2. Reusing template code allows to do changes much effectively because the DRY principle is not violated. See the subtemplates of the MOGLiCC JavaBeans for a bigger example how to use subtemplates and for more examples using the MOGLiCC white space marker .

Next tutorial part ->

<- Back to the tutorial main page

Clone this wiki locally