Plugins

Software should not be a mess of hardcoded features. In fact, it grows and changes over time. On top of that, some users of the software might ought it important for very specific features to be part of a bigger whole.

This is usually done with plugins or addons.

Because not everyone might be on board with the Graphviz rendering engine (there are many others out there), or people want to provide easy support for their custom domain specific language (DSL) in GraphDonkey, the decision was made to embed a plugin system.

The goal was to make this as extensive and expandable as possible, while maintaining a good base for future releases. If you have any issues or thoughts on the plugin system, please leave it here, so it can be reviewed by other users and developers.

What can be done with plugins?

Within the scope of the GraphDonkey app, a plugin is either a rendering engine (complete with possible preferences), a file reference (including syntax and semantic checking) or both. In fact, for generality, it contains a set of file references and a set of engines.

General System Workflow

To demonstrate how plugins are used and how all information that's mentioned below will be combined together, take a look at this schematic representation:

workflow

The boxes indicate functions that are to be defined by the creator of the plugin, with the dotted shapes being optional. The standalone texts represent some instance types:

text: The text from the editor.
tree: The AST parse tree of the given text.
X: Any type that is accepted as single input for the Converter. Whenever X is a string that can also be interpreted as another language, it may be added to the Transform menu.
graph: The graph that is to be drawn. This can be an SVG-definition, binary data (that can be interpreted as an image), or a QImage object.

The Parser takes the text and creates an AST ("tree" in the figure). It also does a syntactic analysis of the tree. The AST cannot be created when there are syntax errors. Optionally, the AST is further analyzed by a Visitor, which does a semantic analysis. When this analysis results in at least one error, the process ends here. When no errors occur, both the text and the tree are inputted into the Transformer, which outputs an object of an unspecified type, X. X can than be interpreted by the Converter in order to create a graph.

When the Converter is a wrapper around an external tool (like Graphviz), it usually has a builtin parser. Hence, the Transformer may simply return the text as "transformed" data.

Plugin File Structure

A plugin has a root folder that must be located in the vendor/plugins directory. Each such folder contains a number of documents that are necessary for the plugin to work. There must be at least an __init__.py file that contains the plugin details as defined below. Note that you don't have to have a python interpreter for your plugin to work with the GraphDonkey executable! Isn't that amazing?

Experienced programmers might be aware that python allows you to execute any command. This makes it so the plugins don't necessarily have to be written in python, as long as they're linked correctly. On the other hand, this can be a massive security risk. Do not use plugins that come from untrusted sources! I am not responsible for any malicious plugins that exist out there. When in doubt ask an experienced user from the community to take a look at it.

requirements.txt

Provides your plugin with additional information on extra python modules/packages that need to be installed. This is conveniant for users when installing this plugin. Packages that are automatically bundled with GraphDonkey (denoted on the installation page) do not have to be listed, seeing as they are required for the app to run.

This file also collects all requirements and allows users to install them before enabling using your plugin. This is incredibly useful and bypasses any additional overhead in sharing your plugin with others. The syntax of this file must be the Python PIP Requirements syntax in order for this to work correctly.

This is the only way to allow for additional packages to be installed, due to security reasons. GraphDonkey must never be ran as administrator and should not require such access. All plugins and requirements will be installed locally to prevent misuse of this system. If other packages are required, it is essential to denote this in the README/documentation of the plugin. This way, your users have the fate of their systems in their own hands.

init.py

This file is the only required file in a plugin folder. Theoretically, it is not necessary for other files to exist, but may help in clarity. This file has the following parts (usually denoted in the order they are listed here):

doc

The docstring at the beginning of your file allows you to name your plugin, give a description and denote an additional set of attributes.

The text on the first line is seen as the plugin name, which is followed by any number of empty lines, before the description starts. Afterwards, you can add a few key-value pairs (in the form of key: value) to associate with your plugin. These can mainly be used for adding copyright, authorship, version numbers, documentation references, websites... Finally, you may end with as many empty lines as you desire.

The documentation itself can be formatted with John Gruber's markdown syntax. As for the key-value pairs, these will be rendered with the keys in bold and the values w.r.t. the same markdown syntax as mentioned above. If your value is too long to fit on the same line, you must add a colon (:) at the beginning of the next line, before continuing your explanation. If your value is a url, it will become a clickable link without additional styling requirements. All whitespace surrounding the key-value pairs is trimmed.

To show the author and the version of your plugin in the plugin browser (in the Preferences window), their corresponding keys (author and version, case-insensitive) must be set. Otherwise, they will not be rendered.

The existance of such a docstring is required for the plugins to flawlessly work together. Please be aware that the name of a plugin cannot be empty and must be unique between all (enabled) plugins in the system.

For instance, an adapted version of the docstring for the bundled Graphviz plugin is given below.

"""Graphviz

Render graphs in Graphviz/Dot format.

This plugin includes:

*   Syntax Highlighting and (LALR) Parsing for Graphviz-Files
*   A rendering engine for Graphviz
    + Rendering in `dot`, `neato`, `twopi`, ...
    + Exporting to numerous filetypes
    + Rendering AST structures to be shown
*   Highly customizable

Author:     Randy Paredis
Website:    https://www.graphviz.org/
Note:      '_Author_' refers to the creator of this plugin, not the software
:           used behind the scenes.
"""

Imports

Often, one may require additional information for a plugin to work. Maybe, a set of predefined constants might help in developing your plugin.

All import statements used in your plugin must assume the project root is the root of the repo. For instance:

# Imports the constants from the main project
from main.extra import Constants

# Import the all objects from the Engine module in the myplugin plugin
from vendor.plugins.myplugin.Engine import *

ICON

The __init__.py file allows for an optional ICON variable, that allows plugin creators to give their plugins an icon. Starting from the current plugin directory, the ICON variable expects a filename referring to an image for your plugin.

Below is an annotated example, based on the Graphviz plugin.

ICON = "graphviz.png"   # Will look in the current directory for this image.

TYPES

With an optional TYPES variable, a set of DSLs can be identified. This includes syntax highlighting, syntax checking, semantics checking and much more.

TYPES is a dictionary of dictionaries. Each internal dictionary has a unique file type name and a set of values w.r.t. the DSL defined by the given file type name. Let's call such an internal dictionary a DSL description. There are multiple DSL descriptions possible for each plugin.

A DSL description consists of the following keys:

extensions: A list of extensions that can be opened with this DSL. If multiple DSL use the same extensions, the first loaded DSL will be selected upon opening a file of that type. Technically, this field is optional, but it's encouraged to use it anyways.
grammar: Optionally provide a [lark][lark] grammar file to use for syntax analysis of the file and building of an AST. The base path for this file is the plugin's root directory.
parser: The [lark][lark] parser to use (as a string), can be one of early, lalr or cyk. Defaults to early.
semantics: A classname that can be used to do semantic analysis of a parse tree. This class must ideally inherit from main.editor.Parser.CheckVisitor. We refer to this class' documentation, the bundled vendor.plugins.graphviz.CheckDot class and sections "Autocompletion" and "Smart Line Indentation" at the bottom of this page for more information.
highlighting: A list of highlighting rules, ordened to their importance (i.e. rule 1 will be applied first, next rule 2, afterwards rule 3...). Such a rule is identified with another dictionary containing the following keys.
- format: The highlighting format to use, which are linked to the user preferences. This can be one of the following (see also the "Theme and Colors" preferences in the app):
  - keyword: For showing keywords in your language.
  - attribute: For specifying attributes or sub-keywords.
  - number: For identifying numbers in your code.
  - string: For indicating strings in your DSL.
  - html: For indicating HTML-like strings.
  - comment: For comments.
  - hash: For preprocessing macro's and/or hashes used in code. Can also be used as a special kind of comment.
  - error: For underlining a certain word with an error line. While possible, this value should be considered bad practice.
- global: Optional boolean value. Corresponds to the /g modifier in Perl regular expressions. When True, the pattern will be matched against the full text, otherwise the text is matched by line. This is incredibly useful for multiline patterns, but is a more expensive operation. In fact, it is discouraged to use this for single line matches. Defaults to False.
- regex: Optional, but when omitted the start and end keys need to be present. Indicates the regex to use. This is either a pattern string (see below), or yet another dictionary.
  - pattern: The pattern string to use. This is a PCRE pattern or a list of words that need to match on word bounds (\b). The latter option is rather useful for adding keywords and other lists.
  - insensitive: Optional boolean value. When True, the pattern will be checked case-insensitively. Corresponds to the /i modifier in Perl regular expressions. Defaults to False.
  - single: Optional boolean value. Corresponds to the /s modifier in Perl regular expressions. When True, the dot (.) in the pattern string is allowed to match any character in the subject string, otherwise it will not match newlines. Defaults to False.
  - multiline: Optional boolean value. Applies the /m modifier in Perl regular expressions. When True, it ensures the caret (^) and dollar sign ($) match the beginning and end of a line (and a string), respectively. Only makes sense when global (see above) is True. Otherwise, this key does not do anything. Defaults to False.
  - extended: Optional boolean value. Corresponds to the /x modifier in Perl regular expressions. When True, any non-escaped whitespace in the pattern string is ignored. Additionally, line comments can be added with an unescaped hash (#). This allows you to increase the readability of your pattern. Defaults to False.
  - unicode: Optional boolean value. Corresponds to the /u modifier in Perl regular expressions. When True, matching with full unicode is enabled. Defaults to False.
  - ungreedy: Optional boolean value. Inverts the greediness of the qualifiers when True. Zero or more (*), one or more (+), zero or one (?) and any number between a and b ({a, b}) will become lazy, while their lazy counterparts become greedy. Defaults to False.
transformer: A mapping (dictionary) of an engine name (see below) to a function representing the input for the engine. This allows multiple engines to be used for the same DSL. The function has two parameters text and tree and must return an object that can be used as an input for the engine. text represents the text inside the editor window and tree is the AST thereof.
(Hint: the AST can be seen in the editor via "View > View Parse Tree")
If your DSL has its own rendering engine that accepts the language as-is (without looking through the AST), lambda x, T: x is enough for this function. When this value is undefined, or the function returns None, no rendering is performed.
Furthermore, when the "engine name" is an extisting file type, the transformation will be added to the Transform menu.
snippets: Optional dictionary that allows you to set a group of predefined snippets with their names that become available for the given file type. They are not added to the Snippets window, but are added behind the scenes. User-defined snippets will take precedence over plugin-defined ones. For instance, if a user has defined a snippet with name My Snippet and an enabled plugin for this file type also uses this name in defining a snippet, the user-defined one will be chosen over the plugin-defined snippet.
Nevertheless, it is discouraged to add snippets for the sake of adding snippets. Only add them if they allow for a better user experience, for instance when certain sentences are commonly used in your language.
paired: Optional list of which groups to pair together (will only work for single-character pairs). Pairing implies that when the first character of a pair is typed, the second one is inserted as well, a technique you see very often in code editors. On top of that, the bracket highlighting in the code editor is also linked to this value list. The list consists of individual pairs that work accordingly.
By default, this list pairs all common brackets (i.e. parentheses ((...)), braces ({...}) and square brackets ([...])) and single ('...') and double ("...") quotes. The default value for this key is therefore given as follows:

`[("(", ")"), ("{", "}"), ("[", "]"), ('"', '"'), ("'", "'")]`
To remove pairing, set this value to the empty list (`[]` or `list()`).
_**Hint:** You can use multiple characters for each opening and closing sequence! This becomes useful for matching `if ... end` groups and the likes._

Below, you can find a working example, based on the Graphviz plugin:

keywords = ["strict", "graph", "digraph", "node", "edge", "subgraph"]
TYPES = {
    "Graphviz": {
        "extensions": ["canon", "dot", "gv", "xdot", "xdot1.2", "xdot1.4"],
        "grammar": "graphviz.lark",         # Defined in the current directory
        "parser": "lalr",
        "semantics": CheckDotVisitor,       # This name is imported
        "highlighting": [
            {
                "regex": {
                    "pattern": keywords,    # Any of the above-defined keywords
                    "insensitive": True
                },
                "format": "keyword"
            },
            ...
            {
                "regex": "\\b-?(\\.[0-9]+|[0-9]+(\\.[0-9]*)?)\\b",
                "format": "number"
            },
            ...
            {
                "regex": {
                    "pattern": "/\\*.*?\\*/",
                    "single": True
                },
                "format": "comment",
                "global": True
            }
        ],
        "transformer": {
            "Graphviz": lambda x, T: x
        },
        "snippets": {
            "FSA Start Node": "start [label=\"\", shape=none, width=0];",
            "FSA End Node": "end [shape=doublecircle];"
        },
        "paired": [
            ("{", "}"), ("[", "]"), ("<", ">"), ('"', '"')
        ]
    }
}

ENGINES

Engines allow users to render in a specific way. Without this property, all DSLs must be converted to a single uniform file reference, which in its turn can be rendered flawlessly. Some users swear by graphviz, while others praise PlantUML instead. Long story short, this adds more flexibility to the overall system.

This property is not required and therefore it can be ignored if you have no desire for a custom rendering engine.

Let's take a quick look at an example, adapted from the Graphviz plugin.

ENGINES = {
    "Graphviz": {
        "convert": convert,
        "preferences": {
            "file": "preferences.ui",
            "class": GraphvizSettings
        },
        "AST": AST,
        "filetypes": {"My file type": ["my", "mine", "notyours"]},
        "export": {
            "extensions": ['fig', 'jpeg', 'pdf', 'tk', 'eps', 'cmapx_np', 'jpe', 'ps', 'ismap', 'x11', 'dot_json', 'gd',
                           'plain', 'vmlz', 'xlib', 'pic', 'plain-ext', 'pov', 'vml', 'json0', 'cmapx', 'jpg', 'svg',
                           'wbmp', 'vrml', 'xdot_json', 'gd2', 'png', 'gif', 'imap_np', 'svgz', 'ps2', 'cmap', 'json',
                           'mp', 'imap'],
            "exporter": export
        }
    }
}

As you can see, ENGINES is constructed in a similar fashion to the TYPES object. This allows for users to assign multiple engines within the same plugin. Each engine is identified by a unique name that is refered to by the converter key in the DSL descriptions defined in the TYPES object.

Each engine itself is specified with the following keys:

convert: A function that takes a string as input and returns binary data as a result. If this binary data is a valid svg file, it is rendered as such. If it's another image type that's recognized by Qt, it is rendered as a plain image. This key is required.
preferences: If you desire to have custom user-defineable settings in your engine, you may use this key to indicate this. More information is given below.
AST: A handy feature of the editor is that it can display the current AST for the file type you're using. If you so desire to use your own rendering mechanism for the tree, you can define this key. The value is a function that takes a lark Tree as input and produces binary output similar to the convert key. If unspecified, the default Graphviz renderer is used, as long as the plugin is enabled and specified.
export: Your engine can allow for the exporting to numerous other file types that can't necessarily be rendered, or to images if you want your user to be able to save the images that your engine generates. This field allows for such export possibilities.

Note that this method is not foolproof/perfect, especially if the rendering of your images require some additional settings. Therefore this might be something that gets a massive overhaul in a future version. If you have any thoughts on the matter, please leave them here.

Nevertheless, the export key is a dictionary that contains the following items:
- extensions: A list containing all the file extensions you support exporting to. At the moment, a user can only export w.r.t. the currently selected engine.
- exporter: A required function that takes text and extension as attributes and returns the binary data to write to a file, or None. The latter can be used as a shorthand to identify you cannot export to that file under the current circumstances. It will not throw an error (unless you tell it to) and instead fail silently. If it returns empty (binary) data, it will be assumed that None was returned. Otherwise, the core will successfully write away your file.
filetypes: Of course, not all filetypes will be recognized by the system. Should you decide to export to a filetype that the system does not recognize, it will state that the type name is the extension in capitals. With this key, you can alter this behaviour via specifically assigning which extensions belong to which file type. They can therefore be grouped together and be overwritten this way. The value is a dictionary with as keys the new names assigned to the extensions. Each of these keys is assigned to a list of extensions.

Preferences / Configuration

You may access the values from the Preferences window for your own purpose. It is, however discouraged to update and adapt them during execution. The set of preferences, or (more specifically) the configuration list, can be obtained via the following code:

from main.extra.IOHandler import IOHandler
Config = IOHandler.get_preferences()

The Config object is a singleton of type QSettings (and therefore it has the same limitations/features provided by that class).

If you desire to have your own preferences as an option in the plugin, you can do so. Each engine section can hold the optional preferences key, which must hold a class and a file key. The file refers to the ui file that's used in the UI (assuming the plugin directory is the root directory) and describes a QGroupBox. The class refers to a specific class, inheriting from the main.plugins.Settings class and takes a pathname and parent as constructor arguments.

This subclass has a preferences member that refers to the Config singleton, as described above and two member functions: rectify and apply. rectify needs to set values from the preferences to the UI and apply does the opposite; it takes values from the UI and stores them in the preferences. The preferences to be added are automatically added to the plugin/<plugin-name> group (to prevent overriding of exisiting values; changing this will inevoquably cause unexpected behaviour), where <plugin-name> is your plugin name (all spaces converted to dashes (-) and <key> is the key you're actually assigning (see also the QSettings page to make note of keylengths and platform-specific features).

Remember to add default values when accessing the preferences from the rectify method!

YOU MAY ONLY READ FROM / WRITE TO THE preferences FROM THE rectify AND apply METHODS! ANY OTHER ACCESS IS DISCOURAGED AND CAN BREAK YOUR SETUP OF GraphDonkey!

For an example, take a look at the Settings class in the bundled vendor.plugins.graphviz.Settings module.

Autocompletion

Most texteditors these days give the user more flexibility by implementing some sort of autocompletion. The most common autocompleter is called intellisense.

Because GraphDonkey allows for full customizability of languages and renderers, the choice was made to allow each plugin to fully customize their own set of completion functions (possibly context-dependent). This can be achieved by a flexible customization of the class assigned to the semantics key in the TYPES variable (see above).

If you inherit from the main.editor.Parser.CheckVisitor class, you may notice you have a few functions and members available. One of these is the completer member. This field is an instance of the main.editor.Intellisense.CompletionStorage class and provides a clean and straightforward way of adding (add method), obtaining (get method) and clearing (clear method) the list of words and definitions that need to be autocompleted.

As you can see in the vendor.plugins.Graphviz.CheckDot.CheckDotVisitor class, you may use these methods to define a set of context-specific autocompletion attributes.

By default, the displayed list will be sorted alphabetically.

Smart Line Indentation

Most languages experience an increase in readability when you use indentation every time a new scope opens. These days, almost every single editor automates this principle by implementing an auto-indent feature. And GraphDonkey is not falling behind!

Alas, most indentation specifics are language specific (i.e. Python prefers indentation after a colon (:), but Java, C++ and most similar languages prefer this to be behind an opening brace ({)). Seeing as GraphDonkey allows numerous languages, it makes more than sense that the logic of such indentation can be set in the file type specification.

Similar as the autocompletion (see above), the class that handles the semantic analysis has an indent and an obtain function. The former allows you to set a specific indentation for a scope, while the latter helps obtaining the indentation level for any line.

See the vendor.plugins.Graphviz.CheckDot.CheckDotVisitor and the main.editor.Parser.CheckVisitor classes for more info on how to do this.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Plugins

What can be done with plugins?

General System Workflow

Plugin File Structure

requirements.txt

init.py

doc

Imports

ICON

TYPES

ENGINES

Preferences / Configuration

Autocompletion

Smart Line Indentation

Clone this wiki locally

Plugins

What can be done with plugins?

General System Workflow

Plugin File Structure

requirements.txt

__init__.py

__doc__

Imports

ICON

TYPES

ENGINES

Preferences / Configuration

Autocompletion

Smart Line Indentation

Clone this wiki locally

init.py

doc