This file describes the interface between rsyslog and external output plugins.
Rsyslog uses stdin and stdout to communicate with the external plugin. This is an established mode of interprocess communication and well supported by all decent languages. Parameters, if any, will be passed in via command line arguments, which should also be easy to obtain in (almost) all languages. Where this is not the case, it is suggested to either use an external config file or hardcode the parameters inside the plugin code.
Rsyslog pushes messages via stdin. Each message is terminated by a LF. So a plugin can obtain a full messages simply by reading a line.
This can cause problems with multi-line messages. There are some cures for this. The recommended one is to use JSON message format (more on message formats below). This will encode LF as "\n" (by JSON RFC requirements) and thus will ensure there are no embedded LFs even in multiline messages. An alternative is to use a message format which contains some other delimiter and program the plugin to watch for this delimiter. With the near-universal availability of JSON libraries in languages these days, we strongly think that the JSON approach is superior.
The message format is generated by standard rsyslog methods, that is via a template. This gives full flexibility over what the plugin is fed. Unfortuanly this also means the necessary template definitions are needed. See the rsyslog doc for how to do that (in the future, this file will also contain some samples).
The plugin may want to convey error information to rsyslog. Unfortunately, the current interface is very weak in this regard. All a plugin currently can do is terminate. In the future, there will be a capability to send feedback via stdout or stderr. For that reason, make sure that no data is written to stdout or stderr by your plugin.
Plugins that write data to stdout or stderr are not compliant to the current interface specification and will not continue to work in the future.
Write your plugin as you would do in a single threaded environment. Rsyslog automatically detects when it is time to spawn additional threads. If it decides so, it will also spawn another instance of your script and feed it concurrently. Note that rsyslog will also terminate instances that it knows are no longer needed.
If your plugin for some reason cannot be run in multiple instances, there are ways to tell rsyslog to work with a single instance. But it is strongly suggested to not restrict rsyslog to do that. Multiple instances in almost all cases do NOT mean any burden to the plugin developer. Just think of two (or more) independent instances of your program running in different console windows. If that is no problem, rsyslog running multiple instances of it is also no problem.
The external output plugin interface will be enhanced within the next future. Most importantly, it will provide support for conveying back status information to rsyslog. Note that all existing plugins will continue to work, even when the interface is evolved. So there is no need to wait for a new interface version.
Interfaces for external input, filter and message modification plugins are planned. Most probably, they will become available in the order mentioned in the last sentence.
The external plugin will use stdin to receive the message that it potentially can modify. The message will be LF-terminated, and no LF must be present within the message itself. By default, the MSG part of the message is provided as input. The "interface.input" parameter can be used to modify this. It may has the following values:
- "msg" (the default)
- "rawmsg", which is the complete message (including header) as received by rsyslog
- "json", which is the complete message object (with all properties broken out) as a JSON object string. This is the "jsonmesg" dynamic message property.
Note: if multi-line messages are to be processed, JSON representation must be used, otherwise errors will happen.
The ability to use non-JSON representations is primarily a performance enhancement. Building the JSON representation causes some overhead, and very often access to either msg or rawmsg is fully sufficient.
The plugin will emit a JSON representation of those properties that need to be modified and their new values to stdout. Again, this is delmited by LF, with no LF permitted inside the JSON representation. Only properties that are to be changed must be included in the response. Unchanged properties should NOT be included in the response, as this would increase processing cost. If no property is to be modified, an empty JSON representation is to be provided.
The plugin must emit one response line for each message (line) received, and must do so in the same order in which the messages were put in stdin. Note that like the output module interface, multiple instances of the plugin may be activated. See above for more information.
Most message properties can be modified. Modifieable are:
- rawmsg
- msg
- syslogtag
- syslogfacility
- syslogseverity
- msgid
- procid
- structured-data
- hostname (aliased "source")
- fromhost
- fromhost-ip
- all message variable ("$!" tree)
If the message variable tree is modified, new variables may also be added. Deletion of message variables is not directly supported. If this is desired, it is suggested to set the variable in question to the empty string ("").
The plugin interface is implemented via the "mmexternal" native plugin. See it's documentation on how to tie your plugin into rsyslog's procesing flow.