KIGen: Inline templating for source code (or whatever you want)

KIGen (KeyMe Inline Generator) is a tool for inline (and in-place) automatic generation of source files. It is, more or less, a wrapper script and formalized method for organizing jinja2 snippets and using them to render portions of arbitrary files

Suggested use cases:

• Generating python objects to serialize/deserialize to a json schema for use in message passing systems
• Generating set/get methods for large objects in languages that lack templating/metaprogramming (C comes to mind...)
• Keeping multiple pieces of source code in-sync when interfaces change (e.g. defining function signatures in a file and autofilling them in your code)
• Automatically generating APIs from a json/yaml spec

Pretty much anywhere you need to generate a lot of code or text from manicured data, KIGen can be used to ease the burdon.

Requirements

• Python >= 3.5
• jinja2

Installation

TODO

Use

KIGen is easy to use and the only requirement is that the target file supports inline comment markers (i.e. # or //, but not /**/...Sorry pedantic C89 folks)

The end user must provide two things:

1. A file with autogen blocks

2. An autogen module, which consists of a pair of files (.py content and .jinja2 template) with the same base name (i.e. foo.py + foo.jinja2).

Autogen Blocks

To make use of KIGen, one must include autogen blocks in the target files.

(Note: For all examples, the comment marker # will be used, this could be any inline comment marker in your use case)

An autogen block consists of a start and end segment.

The start segment should follow the format: # KIGEN_start foo arg1:val1 The end segment should follow the format: # KIGEN_end

Lets break down the start segment:

KIGEN_start is the marker itself and is what the library looks for to identify markers.

foo is the name of the autogen module. This should typically be lower case and must match the name of the autogen content and template files

arg1:val1: In additon to a module name, keyword arguments may also be supplied to the autogen module. These are useful when you need to parameterize a single autogen module for slightly different usecases (i.e maybe you want to generate two different languages in the same module). Any number of key value pairs may be provided.

The end segment is simply there to denote the end of a block and takes no additional information.

NOTE: Currently due to the way that the comment marker and start/end markers are extracted, the text KIGEN_start and KIGEN_end should not appear in autogenerated files outside of actual start/end lines. This may change in the future.

Autogen Modules

An autogen module consists of two files:

• A python content generator (.py)
• A jinja2 template (.jinja2)

The jinja2 template supports all of the normal jinja2 template features. For more information about jinja templates, see http://jinja.pocoo.org.

The python content generator can be as simple or complex as the end user requires, but must follow the following rules:

• The generator MUST provide a get_content(**kwargs) method
• The generator MUST return a dictionary

A simple usecase may just return a dictionary of hardcoded values that are maintained in the content generator file. More advanced usecases may use the content generator to read some external file (a large JSON/YAML/CSV file), parse it, and depending on the supplied keyword arguments, generate snippets of multiple programming languages.

Hello World Example

Assume three files:

A file bar.py to generate that contains the text:

An autogenrated file
# KIGEN_start foo
# KIGEN_end

A template file (foo.jinja2) containing:

Hello {{ name }}

And a content file (foo.py) containing:

def get_content(**kwargs):
    return {'name': 'World'}

Once autogenerated, the autogen file will contain:

An autogenrated file
# KIGEN_start foo
Hello World
# KIGEN_end

CLI

kigen (the CLI tool) takes three main arguments:

• i/input-files
• m/module-path
• o/output-dir (optional)

Multiple input-files and module-path arguments can be specified for batch rendering.

Each input file specified will have all autogenerated blocks rendered by moules present in module-paths. If the module does not exist, an exception will be raised. If no ouput-dir is specified, the file will be autogenerated in-place. If one is specified, the directory will be created (if it doesn't already exist) and the rendered files will be written in that directory.

ex: kigen -i path/to/file.py -i /path/to/file2.py -m mod/dir/one -m /mod/dir/2