Skip to content
apryamostanov edited this page Aug 19, 2019 · 41 revisions

Table of contents generated with markdown-toc

Infinite Technology ∞ Carburetor 🌀

Carburetor is a device that mixes air and fuel for internal combustion engines in the proper air–fuel ratio for combustion.
@Carburetor-based annotations are mixing the auto-generated "infrastructural" code with the user code - thus its name.

Purpose

Carburetor is not an end-user solution. It is a basis for other end-user solutions (such as Blackbox) providing compile-time code transformation implementation and a runtime API.

Carburetor provides a foundation for other libraries to automatically generate Groovy Semantic handling code based on Carburetor configuration and inject it into User code during the Compilation stage resulting in a possibility to intercept and interact with application run-time events including their corresponding compile-time metadata (class name, method name, line start, line end, column start, column end, ASTNode class name):

  • Method/constructor Start
  • Method Result return
  • Method/constructor End
  • Method/constructor Exception
  • Statement Start
  • Statement Start
  • Statement End
  • Control Statements (return, break, continue, throw)
  • Expression evaluation Start
  • Expression evaluation Result
  • Expression evaluation Exception
  • Expression evaluation End

In short

Carburetor-based annotations inject a lot of implementation-specific infrastructure code (such as logging, profiling, security, etc) without affecting the user program logic.

Granularity of injected code can be defined by the user (programmer) up to:

  • Method Exceptions handling (exception and causing method arguments are handled)
  • Method invocation handling (method arguments and result are handled)
  • Statement-level handling
  • Expression-level handling

Features

Code writing phase

Carburetor-based annotations

Carburetor-based annotations are applicable to:

  • Methods
  • Constructors
  • Classes (has same effect as when all methods and constructors in the class are annotated with same annotation)

Carburetor levels

Specify explicitly the needed level of code injection granularity by providing level parameter to Carburetor-based annotations:

@CarburetorBasedAnnotation(level = CarburetorLevel.EXPRESSION)
String foo() {
    return "bar"
}

There are 5 possible values for the carburetorLevel annotation parameter:

  • CarburetorLevel.NONE - method is unmodified
  • CarburetorLevel.ERROR - corresponds to Exception handling transformation
  • CarburetorLevel.METHOD - corresponds to Method transformation
  • CarburetorLevel.STATEMENT - enables Statement transformation in addition to CarburetorLevel.METHOD effect
  • CarburetorLevel.EXPRESSION - enables Expression transformation in addition to CarburetorLevel.METHOD and CarburetorLevel.STATEMENT effects

For description of the above transformations please refer to:

Compilation phase

Default Carburetor Level

When @Carburetor-based annotation omits specific value of level parameter, default value ERROR is used.

However it is possible to externalize configuration of this global default level, by placing Carburetor.json file into the home directory of the compiler.

Carburetor.json has the following structure:

  • defaultLevel - global default level for all Carburetor-based annotations within the build
  • levelsByImplementingClass - it is possible to individually specify default level for specific class implementing Carburetor transformation (e.g. io.infinite.carburetor.TestTransformation).
Sample Carburetor.json
{
    "defaultLevel": "EXPRESSION"
}

Class declarations

Any class annotated with @Carburetor-based annotation (or having annotated methods/constructors) gets 3 additional declared fields:

  • final transient public <Class Type> thisInstance = this - reference to instance object
  • final static transient public Class thisClass = this - reference to class object in static context
  • final static transient public CarburetorEngine testEngine = new CarburetorEngine().getEngine() - Carburetor engine implementation (class and variable name will vary based on implementation).

Transformation rules

During the compilation phase Carburetor performs transformation of methods and constructors having the @Carburetor-based annotation. As the result Method/Constructor code is modified having the additional statements added, while the method signatures and the actual functionality are preserved.

Carburetor follows a clearly defined set of code transformation rules. All Groovy AST Statements and Expressions are subject for transformation using these rules, depending on Carburetor level.

Error-level transformation

Method code is enclosed into Try/Catch statement and in case exception happens during execution of method code - it is passed to corresponding Carburetor engine handling method, along with the method arguments that caused the exception.

Example

Before:

    @TestCarburetor(level = CarburetorLevel.ERROR)
    def test(String bar) {
        bar
    }

After:

try {
    bar (1)
} 
catch (java.lang.Exception automaticException) {
    (2) testEngine.methodException(new io.infinite.supplies.ast.metadata.MetaDataMethodNode(8, 11, 5, 6, 'Foo', 'test'), ['bar': bar ], automaticException)
    throw automaticException 
} 
finally { 
} 
  1. Unmodified original method code
  2. Method Exception handling, capturing Method meta-data, exception and method invocation arguments. Method meta-data includes:
    • Method line and column start and end positions
    • Method declaring class name and method name

This is one of the key functionalities of Carburetor - in this scenario there is practically no performance impact on normal application execution - however if unhandled exception occurs - we are now automatically aware of the Method Arguments which caused the exception.

visit method is not called on method code and the AST traversing/transformation terminates at this stage

Method-level transformation

Method arguments, result and exceptions are handled. Method code is not modified.

Void methods example

Before:

    @TestCarburetor(level = CarburetorLevel.METHOD)
    void test(String bar) {
        bar
    }

After:

(2) testEngine.methodStart(new io.infinite.supplies.ast.metadata.MetaDataMethodNode(8, 11, 5, 6, 'Foo', 'test'), ['bar': bar ])
try {
    bar (1)
} 
catch (java.lang.Exception automaticException) {
    (4) testEngine.methodException(new io.infinite.supplies.ast.metadata.MetaDataMethodNode(8, 11, 5, 6, 'Foo', 'test'), ['bar': bar ], automaticException)
    throw automaticException 
} 
finally { 
    (5) testEngine.methodEnd(new io.infinite.supplies.ast.metadata.MetaDataMethodNode(8, 11, 5, 6, 'Foo', 'test'))} 
Non-void methods example

Before:

    @TestCarburetor(level = CarburetorLevel.METHOD)
    def test(String bar) {
        bar
    }

After:

(2) testEngine.methodStart(new io.infinite.supplies.ast.metadata.MetaDataMethodNode(8, 11, 5, 6, 'Foo', 'test'), ['bar': bar ])
try {
    testEngine.executeMethod({ java.lang.Object itVariableReplacement0 ->
        bar (1)
    }, thisInstance) (3)
} 
catch (java.lang.Exception automaticException) {
    (4) testEngine.methodException(new io.infinite.supplies.ast.metadata.MetaDataMethodNode(8, 11, 5, 6, 'Foo', 'test'), ['bar': bar ], automaticException)
    throw automaticException 
} 
finally { 
    (5) testEngine.methodEnd(new io.infinite.supplies.ast.metadata.MetaDataMethodNode(8, 11, 5, 6, 'Foo', 'test'))} 
  1. Method code is enclosed with Closure (only in case of non-void methods) and Try/Catch statement
  2. Method code execution is preceded with handling of method arguments by Carburetor Engine
  3. Method code closure is passed to "executeMethod" which handles method result (only in case of non-void methods)
  4. Any exception is handled by Carburetor Engine
  5. Finally method execution completion is handled (by Carburetor Engine regardless of whether it is due to unhandled exception or Return statement)

visit method is NOT called on method code and the AST traversing/transformation terminates at this stage

Constructors transformation

Constructors are transformed in similar way to normal methods, with the below exception:

❗ Due to Java platform rules, those Constructors having "super" constructor call should have it as a 1st statement/expression in the code of such Constructors. There are no exceptions possible for this rule and therefore it is impossible to inject any logging/other code before the "super" constructor is being called. Due to this reason, Constructors having "super" call are treated differently in Carburetor: the "super" call is extracted and placed as first statement of transformed constructor code. The code subsequent to the "super" call is transformed using standard Method transformation.

Statement-level transformation
  • Carburetor level "STATEMENT" has same effect as "METHOD", however it additionally transforms the method code.
  • The statements within the method are transformed according to Carburetor transformation rules, adding handling to method execution flow (statement-wise) while preserving the original functionality of the method code.
Normal statements (non-control statements)

Before:

    @TestCarburetor(level = CarburetorLevel.STATEMENT)
    void test(String bar) {
        if (true) {}
    }

After:

testEngine.methodStart(new io.infinite.supplies.ast.metadata.MetaDataMethodNode(10, 13, 5, 6, 'Foo', 'test'), ['bar': bar ])
try {
    testEngine.statementStart(new io.infinite.supplies.ast.metadata.MetaDataStatement('IfStatement', 12, 12, 9, 21, 'test', 'Foo'))
    if (true) {
    } else {
    }
    testEngine.statementEnd(new io.infinite.supplies.ast.metadata.MetaDataStatement('IfStatement', 12, 12, 9, 21, 'test', 'Foo'))
} 
catch (java.lang.Exception automaticException) {
    testEngine.methodException(new io.infinite.supplies.ast.metadata.MetaDataMethodNode(10, 13, 5, 6, 'Foo', 'test'), ['bar': bar ], automaticException)
    throw automaticException 
} 
finally { 
    testEngine.methodEnd(new io.infinite.supplies.ast.metadata.MetaDataMethodNode(10, 13, 5, 6, 'Foo', 'test'))} 
Control statements

Control statements include:

  • return
  • continue
  • break
  • throw

Control statements are handled differently from normal statements, because they directly affect the flow of execution stack, therefore such statements should be fully handled by Carburetor engine before executing the control statement itself, as there is no guarantee that any subsequent code (including any Carburetor handling code) will be reached.

Before:

    @TestCarburetor(level = CarburetorLevel.STATEMENT)
    def test(String bar) {
        return bar
    }

After:

testEngine.methodStart(new io.infinite.supplies.ast.metadata.MetaDataMethodNode(10, 13, 5, 6, 'Foo', 'test'), ['bar': bar ])
try {
    testEngine.executeMethod({ java.lang.Object itVariableReplacement0 ->
        testEngine.preprocessControlStatement(new io.infinite.supplies.ast.metadata.MetaDataStatement('ReturnStatement', 12, 12, 9, 19, 'test', 'Foo'))
        return bar 
    }, thisInstance)
} 
catch (java.lang.Exception automaticException) {
    testEngine.methodException(new io.infinite.supplies.ast.metadata.MetaDataMethodNode(10, 13, 5, 6, 'Foo', 'test'), ['bar': bar ], automaticException)
    throw automaticException 
} 
finally { 
    testEngine.methodEnd(new io.infinite.supplies.ast.metadata.MetaDataMethodNode(10, 13, 5, 6, 'Foo', 'test'))} 
Statement transformation rules
Statement Class Transformed? Children transformation Comments
BlockStatement statements
ForStatement collectionExpression
loopBlock
WhileStatement booleanExpression
loopBlock
DoWhileStatement booleanExpression
loopBlock
IfStatement booleanExpression
ifBlock
elseBlock
ExpressionStatement expression
ReturnStatement expression Control statement transformation; Child Expression is transformed only non-void methods
AssertStatement booleanExpression
messageExpression
TryCatchStatement tryStatement
code (catchStatements)
finallyStatement
All catch statements are transformed
EmptyStatement
SwitchStatement expression
code (switchStatements)
defaultStatement
CaseStatement expression
code
All case statements are transformed
BreakStatement Control statement transformation
ContinueStatement Control statement transformation
SynchronizedStatement expression
code
ThrowStatement expression Control statement transformation
CatchStatement code
Expression-level transformation
  • Carburetor level "EXPRESSION" has same effect as "STATEMENT", however it additionally transforms the expressions within the method
  • The expressions within the method are transformed according to Carburetor transformation rules, adding handling to method execution flow (expression-wise) while preserving the original functionality of the method code.

This helps to handle the expression evaluation results and have an exhaustive runtime data for methods having Carburetor annotation.

Non-declaration expressions

Before:

    @TestCarburetor(level = CarburetorLevel.EXPRESSION)
    void test(String bar2) {
        def foo = bar()
    }

After:

testEngine.methodStart(new io.infinite.supplies.ast.metadata.MetaDataMethodNode(8, 11, 5, 6, 'Foo', 'test'), ['bar2': bar2 ])
try {
    testEngine.expressionStart(new io.infinite.supplies.ast.metadata.MetaDataExpression('DeclarationExpression', 'java.lang.Object foo = this.bar()', 10, 10, 9, 24, 'test', 'Foo'))java.lang.Object foo = testEngine.expressionEvaluation(new io.infinite.supplies.ast.metadata.MetaDataExpression('MethodCallExpression', 'this.bar()', 10, 10, 19, 24, 'test', 'Foo'), { java.lang.Object itVariableReplacement1 ->
        return testEngine.expressionEvaluation(new io.infinite.supplies.ast.metadata.MetaDataExpression('VariableExpression', 'this ', -1, -1, -1, -1, 'test', 'Foo'), { java.lang.Object itVariableReplacement0 ->
            return this 
        }, thisInstance).bar()
    }, thisInstance)testEngine.expressionEnd(new io.infinite.supplies.ast.metadata.MetaDataExpression('DeclarationExpression', 'java.lang.Object foo = this.bar()', 10, 10, 9, 24, 'test', 'Foo'))
} 
catch (java.lang.Exception automaticException) {
    testEngine.methodException(new io.infinite.supplies.ast.metadata.MetaDataMethodNode(8, 11, 5, 6, 'Foo', 'test'), ['bar2': bar2 ], automaticException)
    throw automaticException 
} 
finally { 
    testEngine.methodEnd(new io.infinite.supplies.ast.metadata.MetaDataMethodNode(8, 11, 5, 6, 'Foo', 'test'))} 
Declaration expressions

Declaration expressions are handled differently from normal expressions, as they affect the variable scope of the code, and thus can't be enclosed into closure like normal expressions - this will make the variable invisible and invalidate the original code.

DeclarationExpression is not wrapped into MethodCallExpression - it is rather transformed into ListOfExpressionsExpression having the below expressions:

  • Injected code (MethodCallExpression)
  • Self expression after child nodes transformations
  • Injected code (MethodCallExpression)
    • leftExpression of declaration is never transformed

❗ Furthermore - leftExpression of DeclarationExpression is not visited. This is the only case when BlackBox AST traversing is terminated prematurely.

  • Right expression is transformed
  • Usage of ListOfExpressionsExpression is a hack
    • it works in similar way to BlockStatement - however such usage is "undocumented feature" of Groovy AST

Q: Why do we inject code only for DeclarationExpression (by adding expressions into ListOfExpressionsExpression) and replace other Expressions with MethodCall expression?
A: Declaration expression does not evaluate itself to object and can’t be replaced with MethodCall expression.

❗ VariableScopeVisitor must be able to declare the variable after BlackBox transformation into the same branch of variable scopes. Thus we have to surround DeclarationExpression with injected code (MethodCall expressions) and replace it with ListOfExpressionsExpression.

Before:

    @TestCarburetor(level = CarburetorLevel.EXPRESSION)
    void test(String bar) {
        def foo
    }

After:

testEngine.methodStart(new io.infinite.supplies.ast.metadata.MetaDataMethodNode(8, 11, 5, 6, 'Foo', 'test'), ['bar': bar ])
try {
    testEngine.expressionStart(new io.infinite.supplies.ast.metadata.MetaDataExpression('DeclarationExpression', 'java.lang.Object foo ', 10, 10, 9, 16, 'test', 'Foo'))java.lang.Object foo testEngine.expressionEnd(new io.infinite.supplies.ast.metadata.MetaDataExpression('DeclarationExpression', 'java.lang.Object foo ', 10, 10, 9, 16, 'test', 'Foo'))
} 
catch (java.lang.Exception automaticException) {
    testEngine.methodException(new io.infinite.supplies.ast.metadata.MetaDataMethodNode(8, 11, 5, 6, 'Foo', 'test'), ['bar': bar ], automaticException)
    throw automaticException 
} 
finally { 
    testEngine.methodEnd(new io.infinite.supplies.ast.metadata.MetaDataMethodNode(8, 11, 5, 6, 'Foo', 'test'))} 
Expression transformation rules
Expression Class Is transformed? Children transformation Comments
EmptyExpression
MapEntryExpression keyExpression
keyExpression
ArgumentListExpression
DeclarationExpression rightExpression
BinaryExpression rightExpression
leftExpression (only when operation is not "Assignment" ("="))
BitwiseNegationExpression expression
NotExpression expression
BooleanExpression expression
CastExpression expression
ConstructorCallExpression arguments Special ConstructorCallExpressions (isSpecialCall - e.g. super, this) are excluded from standard transformation into a MethodCallExpression as they do not have a method target.
MethodPointerExpression expression
methodName
AttributeExpression objectExpression
property
PropertyExpression objectExpression
property
RangeExpression from
to
SpreadExpression expression
SpreadMapExpression expression
StaticMethodCallExpression arguments
ElvisOperatorExpression trueExpression
falseExpression
TernaryExpression booleanExpression
trueExpression
falseExpression
UnaryMinusExpression expression
UnaryPlusExpression expression
ConstantExpression n/a
ClassExpression n/a
VariableExpression n/a "Super" variable expression are not transformed
FieldExpression n/a
GStringExpression Values
ClosureListExpression Expressions

Runtime

The injected/transformed code is taking it’s effect during runtime (execution) of user program.

Both compile-time (line numbers, statement/expression names and other meta-data) and runtime (expression evaluation results, method results, their timestamps, exceptions, etc) are passed to Carburetor-based engine for subsequent handling.

Carburetor Runtime Events

Carburetor engine supports runtime events as per the below matrix:

Events\Levels Error Method Statement Statement Expression
Method Exception
Method Start
Method End
Handle method result (except void methods)
Statement Start
Statement End
Expression Start
Expression End
Handle expression result (except declaration expressions)
Handle expression exception (except declaration expressions)
Method Start

Allows to capture Method meta-data as well as runtime method invocation arguments.

Carburetor level Supports this runtime event?
Error
Method
Statement
Expression
Method End

Captures only Method meta-data.

Carburetor level Supports this runtime event?
Error
Method
Statement
Expression
Handle method result

Captures only Object returned by the method.

Only for non-void methods

Carburetor level Supports this runtime event?
Error
Method
Statement
Expression
Method Exception

Captures Method meta-data, runtime method invocation arguments and the actual exception thrown within the method.

Carburetor level Supports this runtime event?
Error
Method
Statement
Expression
Statement Start

Captures Statement meta-data.

Carburetor level Supports this runtime event?
Error
Method
Statement
Expression
Statement End

Captures statement meta-data.

Carburetor level Supports this runtime event?
Error
Method
Statement
Expression
Expression Start

Captures Expression meta-data.

Carburetor level Supports this runtime event?
Error
Method
Statement
Expression
Expression End

Captures Expression meta-data.

Carburetor level Supports this runtime event?
Error
Method
Statement
Expression
Handle expression result

Captures an object to which the expression has been evaluated.

Only for non-declaration expressions

Carburetor level Supports this runtime event?
Error
Method
Statement
Expression
Handle expression exception

Captures exception thrown during expression evaluation, as well as Expression meta-data.

Only for non-declaration expressions

Carburetor level Supports this runtime event?
Error
Method
Statement
Expression

Principles

Carburetor-based end-user solutions must follow the below principles:

Code consistency

Code is being transformed by Carburetor-based Annotations during SEMANTIC_ANALYSIS Groovy compiler phase. The resulting transformed code should be semantically and syntactically valid and it should successfully pass the subsequent compilation phases.

In simple words - Carburetor-based transformations should not produce code which won’t compile.

Code equivalency

Since Carburetor-based Annotations are performing transformation of user code, as a base principle of transformation it needs to ensure that the transformed code is equivalent to the initial user code in terms of its actual functionality excluding the additional injected code. This principle should guarantee that @Carburetor-based annotation can be safely added to any existing or new Groovy methods and constructors, minimizing risk of regression issues.

Risks

Performance

Closure usage

Non-void methods and expressions are enclosed into Groovy Closures and executed using "call()" method.

Therefore Carburetor performance is dependent on Closure execution performance in Groovy.

Code volume burst

Carburetor levels Statement and Expression add a significant amount of additional code.

The code base volume tends to grow exponentially in this situation.

Therefore it slows down the execution of business-functionality and is recommended only to be used in Test builds/environments and for debugging.

Security

Sensitive data exposure

Sensitive data such as Cardholder data, passwords and personal data becomes exposed to Carburetor Engine API.

Public injected fields

Due to the reasons of maintaining compatibility with Spring Boot, fields injected into Classes using Carburetor annotations are public.

Test coverage and code disruption

Carburetor is traversing and transforming whole AST of User code.

Any violation of Code equivalency principle due to bug in Carburetor can cause catastrophic damage.

This is especially actual since the Carburetor annotation addition is subtle in terms of regression impact analysis.

Limitations

@CompileStatic support

Groovy static compilation is supported not in all Carburetor levels as shown at the below matrix:

Carburetor level Is CompileStatic supported?
Error
Method
Statement
Expression

Usage

Carburetor is hosted in JCenter via Bintray:

Download

End-user Carburetor implementations should follow the below conventions.

Annotation

Target: Method, Constructor, Type.

Field: mandatory to have field level with type CarburetorLevel.

Transformation class

Should extend and implement abstract methods of io.infinite.carburetor.CarburetorTransformation.

Recommended phase: SEMANTIC_ANALYSIS

Engine class

Should extend and implement abstract methods of io.infinite.carburetor.CarburetorEngine.

Should declare getInstance() method (with arguments as per Transformation Class implementation).

Clone this wiki locally
You can’t perform that action at this time.