This module was first published as crayfishx/rsyslog
. It has now moved to puppet/rsyslog
and is managed by the community group Vox Pupuli.
This module manages the rsyslog server and client configuration. It supports rsyslog v8 and defaults to configuring most things in the newer rainerscript configuration style. Where possible, common configuration patterns have been abstracted so they can be defined in a structured way from hiera. Though there are a lot of customization options with the configuration, highly complex rsyslog configurations are not easily represented in simple data structures and in these circumstances you may have to provide raw rainerscript code to acheive what you need. However, the aim of this module is to abstract as much as possible.
This module is only compatible with Puppet 4.0.0+
- Installs the Rsyslog package, service, and configuration.
- Install ancillary plugin packages.
This declaration will get you basic configuration for Rsyslog on your system:
include rsyslog::config
By default, everything is configured in a single file under $confdir
called 50_rsyslog.conf. This means that packages and other OS specific configurations can also be included (see purge_config_files above). The default file can be changed using the rsyslog::target_file
directive and is relative to the confdir.
eg:
rsyslog::target_file: 50_rsyslog.conf
You can, however, define custom confdirs and/or custom paths for configuration files. All configuration options have the following global options you can add to their hiera keys:
priority
- Order in the file to place the config value relative to the other config options in the file. Takes an integer. Defaults to the priority set for the configuration type. See Ordering for more.target
- Target file to place the config values in. Defaults to 50_rsyslog.conf in the default$confdir
.confdir
- Target configuration directory. Defaults to/etc/rsyslog.d
.
The following configuration parameters are defaults for the order of configuration object types within the configuration file. They can be overriden for individual object definitions (see configuring objects below)
## Default object type priorities (can be overridden)
rsyslog::global_config_priority: 10
rsyslog::module_load_priority: 20
rsyslog::input_priority: 30
rsyslog::main_queue_priority: 40
rsyslog::parser_priority: 45
rsyslog::template_priority: 50
rsyslog::filter_priority: 55
rsyslog::action_priority: 60
rsyslog::ruleset_priority: 65
rsyslog::lookup_table_priority: 70
rsyslog::legacy_config_priority: 80
rsyslog::custom_priority: 90
Ordering is done numerically. I.E. 111 is after 110 is after 99.
Configuration objects are written to the configuration file in rainerscript format and can be configured in a more abstract way directly from Hiera. The following configuration object types are supported
- Modules
- Global configuration
- Main queue options
- Templates
- Actions
- Inputs
- Lookup_tables
- Parser
- Rulesets
- Filters
- legacy_config
A hash of hashes, hash key represents the module name and accepts a hash with values or an empty hash as its value. The hash accepts the following three values:
type
: values can beexternal or builtin
the default value is external and need not be specified explicitly.config
: its a hash which provides optional parameters to the module loaded.priority
: The module load order can be prioritised based on the optionalpriority
value.
Puppet example:
class { 'rsyslog::config':
'modules' => {
'imuxsock' => {},
'imudp' => {
'config' => {
'threads' => '2',
'TimeRequery' => '8',
'batchSize' => '128',
},
},
'omusrmsg' => {
'type' => 'builtin',
},
'omfile' => {
'type' => 'builtin',
'config' => {
'fileOwner' => 'syslog',
'fileGroup' => 'adm',
'dirGroup' => 'adm',
'fileCreateMode' => '0640',
'dirCreateMode' => '0755',
},
},
'impstats' => {
'type' => 'external',
'priority' => 29,
'config' => {
'interval' => '60',
'severity' => '7',
'log.syslog' => 'off',
'log.file' => '/var/log/rsyslog/logs/stats/stats.log',
'Ruleset' => 'remote',
},
},
},
}
Hiera example:
rsyslog::config::modules:
imuxsock: {}
imudp:
config:
threads: "2"
TimeRequery: "8"
batchSize: "128"
omusrmsg:
type: "builtin"
omfile:
type: "builtin"
config:
fileOwner: "syslog"
fileGroup: "adm"
dirGroup: "adm"
fileCreateMode: "0640"
dirCreateMode: "0755"
impstats:
type: "external"
priority: 29
config:
interval: "60"
severity: "7"
log.syslog: "off"
log.file: "/var/log/rsyslog/logs/stats/stats.log"
Ruleset: "remote"
will produce
module (load="imuxsock")
module (load="imudp"
threads="2"
TimeRequery="8"
batchSize="128"
)
module (load="builtin:omusrmsg")
module (load="builtin:omfile"
fileOwner="syslog"
fileGroup="adm"
dirGroup="adm"
fileCreateMode="0640"
dirCreateMode="0755"
)
module (load="impstats"
interval="60"
severity="7"
log.syslog="off"
log.file="/var/log/rsyslog/logs/stats/stats.log"
Ruleset="remote"
)
A hash of hashes, they key represents the configuration setting and the value is a hash with the following keys:
value
: the value of the settingtype
: the type of format to use (legacy or rainerscript), if omitted rainerscript is used.
Puppet example:
class { 'rsyslog::config':
'global_config' => {
'umask' => {
'value' => '0000',
'type' => 'legacy',
'priority' => 01,
},
'RepeatedMsgReduction' => {
'value' => 'on',
'type' => 'legacy',
},
'PrivDropToUser' => {
'value' => 'syslog',
'type' => 'legacy',
},
'PrivDropToGroup' => {
'value' => 'syslog',
'type' => 'legacy',
},
'parser.escapeControlCharactersOnReceive' => {
'value' => 'on',
},
'workDirectory' => {
'value' => '/var/spool/rsyslog',
},
'maxMessageSize' => {
'value' => '64k',
},
},
}
Hiera example:
rsyslog::config::global_config:
umask:
value: '0000'
type: legacy
priority: 01
RepeatedMsgReduction:
value: 'on'
type: legacy
PrivDropToUser:
value: 'syslog'
type: legacy
PrivDropToGroup:
value: 'syslog'
type: legacy
parser.escapeControlCharactersOnReceive:
value: 'on'
workDirectory:
value: '/var/spool/rsyslog'
maxMessageSize:
value: '64k'
will produce
$umask 0000
$PrivDropToGroup syslog
$PrivDropToUser syslog
$RepeatedMsgReduction on
global (
parser.escapeControlCharactersOnReceive="on"
workDirectory="/var/spool/rsyslog"
maxMessageSize="64k"
)
Configures the main_queue
object in rsyslog as a hash. eg:
Puppet Example:
class { 'rsyslog::config':
'main_queue_opts' => {
'queue.maxdiskspace' => '1000G',
'queue.dequeuebatchsize' => 1000,
}
}
Hiera Example:
rsyslog::config::main_queue_opts:
queue.maxdiskspace: 1000G
queue.dequeuebatchsize: 1000
will produce
main_queue(
queue.maxdiskspace="1000G"
queue.dequeuebatchsize="1000"
)
Configures template
objects in rsyslog. Each element is a hash containing the name of the template, the type and the template data. The type parameter can be one of string
, subtree
, plugin
or list
Puppet Example:
class { 'rsyslog::config':
'templates' => {
'remote' => {
'type' => 'string',
'string' => '/var/log/rsyslog/logs/%fromhost-ip%.log',
},
'tpl2' => {
'type' => 'subtree',
'subtree' => '$1!$usr',
},
'someplug' => {
'type' => 'plugin',
'plugin' => 'foobar',
},
}
}
Hiera Example:
rsyslog::config::templates:
remote:
type: string
string: "/var/log/rsyslog/logs/%fromhost-ip%/%fromhost-ip%.log"
tpl2:
type: subtree
subtree: "$1!$usr"
someplug:
type: plugin
plugin: foobar
will produce
template (name="remote" type="string"
string="/var/log/rsyslog/logs/%fromhost-ip%/%fromhost-ip%.log"
)
When using list
, the list_descriptions
hash should contain an array of single element hashes, the key should be constant
or property
with their corresponding parameters in a sub hash.
Puppet example:
class { 'rsyslog::config':
'templates' => {
'plain-syslog' => {
'type' => 'list',
'list_descriptions' => [
{
'constant' => {
'value' => '{',
}
},
{
'constant' => {
'value' => '\"@timestamp\":\"',
}
},
{
'propery' => {
'name' => 'timereported',
'dateFormat' => 'rfc3339',
}
},
{
'constant' => {
'value' => '\",\"host\":\"'
}
},
{
'property' => {
'name' => 'hostname'
}
},
{
'constant' => {
'value' => '\",\"severity\":\"'
}
},
{
'property' => {
'name' => 'syslogseverity-text',
}
},
{
'constant' => {
'value' => '\",\"facility\":\"'
}
},
{
'property' => {
'name' => 'syslogfacility-text'
}
},
{
'constant' => {
'value' => '\",\"host\":\"'
}
},
{
'property' => {
'name' => 'syslogtag',
'format' => 'json',
}
},
{
'constant' => {
'value' => '\",\"message\":\"'
}
},
{
'property' => {
'name' => 'msg',
'format' => 'json'
}
},
{
'constant' => {
'value' => '\"}'
}
}
]
}
}
}
Hiera example:
plain-syslog:
type: list
list_descriptions:
- constant:
value: '{'
- constant:
value: '\"@timestamp\":\"'
- property:
name: timereported
dateFormat: rfc3339
- constant:
value: '\",\"host\":\"'
- property:
name: hostname
- constant:
value: '\",\"severity\":\"'
- property:
name: syslogseverity-text
- constant:
value: '\",\"facility\":\"'
- property:
name: syslogfacility-text
- constant:
value: '\",\"tag\":\"'
- property:
name: syslogtag
format: json
- constant:
value: '\",\"message\":\"'
- property:
name: msg
format: json
- constant:
value: '\"}'
will produce
template (name="plain-syslog" type="list"
)
{
constant(value="{" )
constant(value="\"@timestamp\":\"" )
property(name="timereported" dateFormat="rfc3339" )
constant(value="\",\"host\":\"" )
property(name="hostname" )
constant(value="\",\"severity\":\"" )
property(name="syslogseverity-text" )
constant(value="\",\"facility\":\"" )
property(name="syslogfacility-text" )
constant(value="\",\"tag\":\"" )
property(name="syslogtag" format="json" )
constant(value="\",\"message\":\"" )
property(name="msg" format="json" )
constant(value="\"}" )
}
Configures action objects in rainerscript. Each element of the hash contains the type of action, followed by a hash of configuration options. It also accepts an optional facility parameter and the content is formatted based on the no of config options passed and if the facility option is present.
Puppet example:
class { 'rsyslog::config':
'actions' => {
'all_logs' => {
'type' => 'omfile',
'facility' => '*.*;auth,authpriv.none',
'config' => {
'dynaFile' => 'remoteSyslog',
'specifics' => '/var/log/test',
}
},
'kern_logs' => {
'type' => 'omfile',
'facility' => 'kern.*',
'config' => {
'dynaFile' => 'remoteSyslog',
'file' => '/var/log/kern.log',
'cmd' => '/proc/cmdline',
}
},
'elasticsearch' => {
'type' => 'omelasticsearch',
'config' => {
'queue.type' => 'linkedlist',
'queue.spoolDirectory' => '/var/log/rsyslog/queue'
}
}
}
}
Hiera example:
rsyslog::config::actions:
all_logs:
type: omfile
facility: "*.*;auth,authpriv.none"
config:
dynaFile: "remoteSyslog"
specifics: "/var/log/test"
kern_logs:
type: omfile
facility: "kern.*"
config:
dynaFile: "remoteSyslog"
file: "/var/log/kern.log"
cmd: "/proc/cmdline"
elasticsearch:
type: omelasticsearch
config:
queue.type: "linkedlist"
queue.spoolDirectory: /var/log/rsyslog/queue
will produce
#Note: There is only 2 options passed so formats in a single line.
# all_logs
*.*;auth,authpriv.none action(type="omfile" dynaFile="remoteSyslog" specifics="/var/log/test" )
#Note: There is more than 2 options passed so formats into multi line with facility.
# kern_logs
kern.* action(type="omfile"
dynaFile="remoteSyslog"
file="/var/log/kern.log"
cmd="/proc/cmdline"
)
#Note: There is no facility option passed so formats it without facility.
action(type="omelasticsearch"
queue.type="linkedlist"
queue.spoolDirectory="/var/log/rsyslog/queue"
)
Configures input objects in rainerscript. Each element of the hash contains the type of input, followed by a hash of configuration options. Eg:
Puppet examples:
class { 'rsyslog:config':
'inputs' => {
'imudp' => {
'type' => 'imudp',
'config' => {
'port' => '514'
}
}
}
}
Hiera examples:
rsyslog::config::inputs:
imudp:
type: imudp
config:
port: '514'
will produce
# imdup
input(type="imudp"
port="514"
)
Configures lookup_tables objects in rainerscript AND generates the JSON lookup_table file. Each key of the hash contains the name of the lookup/lookup_table.
The elements of the hash contain a json
hash containing the values for the JSON file, a lookup_file element that is the path to where the JSON file will be stored,
and a reload_on_hup boolean.
The json hash contains 4 elements: version
, nolookup
, type
, and table
. They MUST be specified in this order as per the
lookup_tables documentation:
version
- Integer denoting the version/revision of the lookup_table file.nolookup
- String denoting what should be returned if a lookup doesn't find a match in the table.type
- Enumerable denoting the type of lookup table. This can bestring
,array
, orsparseArray
.table
- An Array of hashes containing the table index and value for each lookup.
Puppet example:
class { 'rsyslog::config':
'lookup_tables' => {
'ip_lookup' => {
'lookup_json' => {
'version' => 1,
'nolookup' => 'unk',
'type' => 'string',
'table' => [
{
'index' => '1.1.1.1',
'value' => 'AB'
},
{
'index' => '2.2.2.2',
'value' => 'CD'
}
]
},
'lookup_file' => '/etc/rsyslog.d/tables/ip_lookup.json',
'reload_on_hup' => true
}
}
}
Hiera Example:
rsyslog::config::lookup_tables:
ip_lookup:
lookup_json:
version: 1
nolookup: 'unk'
type: 'string'
table:
- index: '1.1.1.1'
value: 'AB'
- index: '2.2.2.2'
value: 'CD'
lookup_file: '/etc/rsyslog.d/tables/ip_lookup.json'
reload_on_hup: true
will produce
# /etc/rsyslog.d/tables/ip_lookup.json
{
"version": 1,
"nomatch": "unk",
"type": "string",
"table": [
{
"index": "1.1.1.1",
"value": "A"
},
{
"index": "2.2.2.2",
"value": "B"
}
]
}
and
lookup_table(name="ip_lookup" file="/etc/rsyslog.d/tables/ip_lookup.json" reloadOnHUP="on")
NOTE: This does not create the actual lookup()
call in the Rsyslog configuration file(s). Currently that is only supported via
the rsyslog::config::custom_config
hash as it requires setting rsyslog variables (I.E. - set $.iplook = lookup('ip_lookup', $hostname)
).
Configures parser objects in rainerscript. Each Element of the hash contains the type of parser, followed by a hash of configuration options. Eg:
Puppet Example:
class { 'rsyslog::config':
'parser' => {
'pmrfc3164_hostname_with_slashes' => {
'type' => 'pmrfc3164',
'config' => {
'permit.slashesinhostname' => 'on'
}
}
}
}
Hiera Example:
rsyslog::config::parser:
pmrfc3164_hostname_with_slashes:
type: pmrfc3164
config:
permit.slashesinhostname: 'on'
will produce
parser(name="pmrfc3164_hostname_with_slashes"
type="pmrfc3164"
permit.slashesinhostname="on"
)
Configures Rsyslog ruleset blocks in rainerscript. There are two elements in the rulesets hash:
parameters
- settings to pass to the ruleset determining things such as which rsyslog parser to use or the ruleset's queue size.rules
- the actual content that goes inside the ruleset. Currently the following are supported:action
- rsyslog actions defined inside of the ruleset.lookup
- Sets a variable to the results of an rsyslog lookup.set
- Set an rsyslog variable or property. Property explicitly requires that the set name be a string beginning with$!
, while a variable can be a plain string or a string starting with$.
.- NOTE: Setting the variable with a string that does NOT begin with
$.
is deprecated and will be removed in the next major release!
- NOTE: Setting the variable with a string that does NOT begin with
call
- call a specific action.exec
- execute the following system commandexpression_filter
- Filter based on one or more expressions.property_filter
- Filter based on one or more RsyslogD properties.
stop
- a Boolean to set if the ruleset ends with a stop or not.
NOTE: For any rule
key that can also be a standalone rsyslog resource (action
, expression_filter
, or property_filter
), the user MUST define a name key that will be passed as the resource name to the template. This will be simplified in a future release.
NOTE: While it is entirely possible to configure Rulesets using the Puppet DSL, it is recommended against as Rulesets can easily become difficult to read when compared to the YAML-based hieradata.
Puppet example:
class { 'rsyslog::config':
'rulesets' => {
'ruleset_eth0_514_tcp' => {
'parameters' => {
'parser' => 'pmrfc3164.hostname_with_slashes',
'queue.size' => '10000',
},
'rules' => [
{ 'set' => { '$!rcv_time' => 'exec_template("s_rcv_time")' }},
{ 'set' => { '$.utime_gen' => 'exec_template("s_unixtime_generated")' }},
{ 'set' => { 'uuid' => '$uuid' }},
{
'action' => {
'name' => 'utf8-fix',
'type' => 'mmutf8fix',
}
},
{
'action' => {
'name' => 'test-action',
'type' => 'omfile',
'facility' => '*.*;auth,authpriv.none',
'config' => {
'dynaFile' => 'remoteSyslog',
'specifics' => '/var/log/test'
}
}
},
{
'action' => {
'name' => 'test-action2',
'type' => 'omfile',
'config' => {
'dynaFile' => 'remoteSyslog',
'specifics' => '/var/log/test'
}
}
},
{
'lookup' => {
'var' => 'srv',
'lookup_table' => 'srv-map',
'expr' => '$fromhost-ip'
}
},
{ 'call' => 'action.parse.rawmsg' },
{ 'call' => 'action.parse.r_msg' },
],
'stop' => true,
}
}
}
Hiera example:
rsyslog::config::rulesets:
ruleset_eth0_514_tcp:
parameters:
parser: pmrfc3164.hostname_with_slashes
queue.size: '10000'
rules:
- set:
# Set a Property with a value from a template.
$!rcv_time: 'exec_template("s_rcv_time")'
- set:
# Set a Variable with a value from a template.
$.utime_gen: 'exec_template("s_unixtime_generated")'
- set:
# Set a Variable using the deprecated method with a value from $uuid
uuid: '$uuid'
- action:
name: utf8-fix
type: mmutf8fix
- action:
name: test-action
type: omfile
facility: "*.*;auth,authpriv.none"
config:
dynaFile: "remoteSyslog"
specifics: "/var/log/test"
- action:
name: test-action2
type: omfile
config:
dynaFile: "remoteSyslog"
specifics: "/var/log/test"
- lookup:
var: srv
lookup_table: srv-map
expr: '$fromhost-ip'
- call: 'action.parse.rawmsg'
- call: 'action.parse.r_msg'
- exec: '/bin/echo'
stop: true
Will produce:
ruleset (name="ruleset_eth0_514_tcp"
parser="pmrfc3164.hostname_with_slashes"
queue.size="10000"
) {
set $.rcv_time = exec_template("s_rcv_time");
set $.utime_gen = exec_template("s_unixtime_generated");
set $.uuid = $uuid;
# utf8-fix action
action(type="mmutf8fix"
name="utf8-fix"
)
# test-action action
*.*;auth,authpriv.none action(type="omfile"
name="test-action"
dynaFile="remoteSyslog"
specifics="/var/log/test"
)
# test-action2 action
action(type="omfile"
name="test-action2"
dynaFile="remoteSyslog"
specifics="/var/log/test"
)
set $.srv = lookup("srv-map", $fromhost-ip);
call action.parse.rawmsg
call action.parse.r_msg
^/bin/echo
stop
}
Rulesets can also contain filtering logic for calling other rulesets, setting other variables, or even dropping logs based on specific values. Filtering logic is required to utilize lookup_tables
and lookup
calls.
Rsyslog puppet supports two kinds of filters:
expression_filter
property_filter
More information about Rsyslog Filters can be found at: http://www.rsyslog.com/doc/v8-stable/configuration/filters.html
Expression filters use traditional if/else
and if/else if/else
logic to execute rules on specific return values. lookup_tables
are compatible ONLY with expression_filters
The Ruleset expression_filter
key has a few different keys than the rsyslog::config::expression_filters
parameter:
name
- Currently required to prevent errors. This is logical and only used by Puppet.filter
- Thefilter
key is synonymous with theconditionals
key found in thersyslog::config::expression_filters
parameter. See the Expression Filter Docs for more info.
Puppet Example:
class { 'rsyslog::config':
'rulesets' => {
'ruleset_eth0_514_udp' => {
'parameters' => {
'queue.type' => 'LinkedList'
},
'rules' => [
{
'expression_filter' => {
'filter' => {
'if' => {
'expression' => '$fromhost-ip == "192.168.255.1"',
'tasks' => [
{ 'call' => 'ruleset.action.rawlog.standard' },
{ 'stop' => true }
]
}
}
}
},
{ 'call' => 'ruleset.client.log.standard' },
{ 'call' => 'ruleset.unknown.standard' },
],
'stop' => true
}
}
}
Hiera Example:
rsyslog::config::rulesets:
ruleset_eth0_514_udp:
parameters:
queue.type: LinkedList
rules:
- expression_filter:
filter:
if:
expression: '$fromhost-ip == "192.168.255.1"'
tasks:
- call: "ruleset.action.rawlog.standard"
- stop: true
- call: "ruleset.client.log.standard"
- call: "ruleset.unknown.standard"
stop: true
will produce:
ruleset (name="ruleset_eth0_514_tcp"
queue.type="LinkedList"
) {
if $fromhost-ip == "192.168.255.1" then {
call ruleset.action.rawlog.standard
stop
}
call ruleset.client.log.standard
call ruleset.unknown.standard
stop
}
Puppet example with lookup tables:
NOTE: Good example for how to define multiple rsyslog resources in a single rsyslog::config
class
class { 'rsyslog::config':
'lookup_tables' => {
'srv-map' => {
'lookup_json' => {
'version' => 1,
'nolookup' => 'unk',
'type' => 'string',
'table' => [
{
'index' => '192.168.255.10',
'value' => 'windows'
},
{
'index' => '192.168.255.11',
'value' => 'windows'
},
{
'index' => '192.168.255.12',
'value' => 'linux'
}
],
},
'lookup_file' => '/etc/rsyslog.d/tables/srv-map.json',
'reload_on_hup' => true
}
},
'rulesets' => {
'ruleset_lookup_set_windows_by_ip' => {
'rules' => [
{
'lookup' => {
'var' => 'srv',
'lookup_table' => 'srv-map',
'expr' => '$fromhost-ip'
}
},
{
'expression_filter' => {
'filter' => {
'main' => {
'expression' => '$.srv == \"windows\"',
'tasks' => [
{ 'call' => 'ruleset.action.forward.windows' },
{ 'stop' => true }
]
},
'unknown_log' => {
'expression' => '$.srv == \"unk\"',
'tasks' => [
{ 'call' => 'ruleset.action.drop.unknown' },
{ 'stop' => 'true' }
]
},
'default' => {
'tasks' => [
{ 'stop' => 'true' }
]
}
}
}
}
]
}
}
}
Example with lookup:
rsyslog::config::lookup_tables:
srv-map:
lookup_json:
version: 1
nolookup: 'unk'
type: 'string'
table:
- index: '192.168.255.10'
value: 'windows'
- index: '192.168.255.11'
value: 'windows'
- index: '192.168.255.12'
value: 'linux'
lookup_file: '/etc/rsyslog.d/tables/srv-map.json'
reload_on_hup: true
rsyslog::config::rulesets:
ruleset_lookup_set_windows_by_ip:
rules:
- lookup:
var: srv
lookup_table: srv-map
expr: '$fromhost-ip'
- expression_filter:
filter:
main:
expression: '$.srv == "windows"'
tasks:
- call: "ruleset.action.forward.windows"
- stop: true
unknown_log:
expression: '$.srv == "unk"'
tasks:
- call: "ruleset.action.drop.unknown"
- stop: true
default:
tasks:
- stop: true
stop: true
Will produce:
#/etc/rsyslog.d/tables/srv-map.json
{
"version": 1,
"nomatch": "unk",
"type": "string",
"table": [
{
"index": "192.168.255.10",
"value": "windows"
},
{
"index": "192.168.255.11",
"value": "windows"
},
{
"index": "192.168.255.12",
"value": "linux"
}
]
}
#rsyslog.conf
lookup_table(name="srv-map" file="/etc/rsyslog.d/tables/srv-map.json" reloadOnHUP=on)
ruleset(name="ruleset_lookup_set_windows_by_ip"
) {
set $.srv = lookup("srv-map", $fromhost-ip);
if ($.srv == "windows") then {
call ruleset.action.forward.windows
stop
} else if ($.srv == "unk") then {
call ruleset.action.drop.unknown
stop
} else {
stop
}
}
property_filters
are unique to rsyslogd. They allow to filter on any property, like HOSTNAME, syslogtag and msg. property_filters
are faster than expression_filters
as they us built-in rsyslog properties to lookup and match data.
Puppet Example:
class { 'rsyslog::config':
'rulesets' => {
'ruleset_msg_check_for_error' => {
'rules' => [
{
'property_filter' => {
'property' => 'msg',
'operator' => 'contains',
'value' => 'error',
'tasks' => [
{ 'call' => 'ruleset.action.error' },
{ 'stop' => true }
]
}
}
]
}
}
}
Hiera Example:
rsyslog::config::rulesets:
ruleset_msg_check_for_error:
rules:
- property_filter:
property: 'msg'
operator: 'contains'
value: 'error'
tasks:
- call: 'ruleset.action.error'
- stop: true
Will Generate:
#rsyslog.conf
ruleset(name="ruleset_msg_check_for_error"
) {
:msg, contains, "informational" {
call ruleset.action.error
stop
}
}
Rsyslog has the ability to filter each log line based on log properties and/or variables.
There are four kinds of filters in Rsyslog:
- "traditional" severity/facility based Selectors - handled in the Actions parameter.
- BSD-style blocks - not supported in Rsyslog 7+ and as such are not supported in this module.
- Property-based Filters
- Expression-based Filters
This section covers Property and Expression based filters.
Property-based filters are unique to rsyslogd. They allow to filter on any property, like HOSTNAME, syslogtag and msg. Property-based filters are only supported with native properties in Rsyslog. See Rsyslog Properties for a list of supported properties.
The rsyslog::config::property_filters
parameter is a Hash of hashes where the hash-key is the logical name for the filter. This name is for Puppet resource naming purposes only and has no other function. The filter name has several additional child keys as well:
property
- the Rsyslogd property the filter will lookup.operator
- the Rsyslogd property filter-supported operator to compare the property value with the expected value. See Rsyslog Property Compare-Operations for a list of supported operators. These operators are validated with theRsyslog::PropertyOperator
data type.value
- the value that the property filter will match against.tasks
- A hash of actions to take in the event of a filter match.- All sub-keys for the
tasks
hash maps to another rsyslog configuration object.
- All sub-keys for the
Puppet Example:
class { 'rsyslog::config':
'property_filters' => {
'hostname_filter' => {
'property' => 'hostname',
'operator' => 'contains',
'value' => 'some_hostname',
'tasks' => [
{
'action' => {
'name' => 'omfile_defaults',
'type' => 'omfile',
'facility' => '*.*;auth,authpriv.none',
'config' => {
'dynaFile' => 'remoteSyslog',
'specifics' => '/var/log/test',
}
}
},
{ 'stop' => true }
]
},
'ip_filter' => {
'property' => 'fromhost-ip',
'operator' => 'startswith',
'value' => '192',
'tasks' => [
{ 'stop' => true }
]
}
}
}
Hiera Example:
rsyslog::config::property_filters:
hostname_filter:
property: hostname
operator: contains
value: some_hostname
tasks:
- action:
name: omfile_defaults
type: omfile
facility: "*.*;auth,authpriv.none"
config:
dynaFile: "remoteSyslog"
specifics: "/var/log/test"
- stop: true
ip_filter:
property: fromhost-ip
operator: startswith
value: '192'
tasks:
- stop: true
will produce
:hostname, contains, "some_hostname" {
*.*;auth,authpriv.none action(type="omfile" dynaFile="remoteSyslog" specifics="/var/log/test")
stop
}
:fromhost-ip, startswith, "192" {
stop
}
Expression-based filters allow filtering on arbitrary complex expressions, which can include boolean, arithmetic and string operations.
Expression-based filters are also what are used to match against lookup_table data.
The rsyslog::config::expression_filters
parameter is a Hash of hashes where the hash-key is the logical name for the filter. This name is for Puppet resource naming purposes only and has no other function. The filter name has a few additional child keys as well:
conditionals
- Hash describing the different conditional cases, which are hashes of hashes.cases
- Hash of hashes. This has two reserved keys and four reserved names:if
/main
- This is the primary condition for your expression.if
is provided for backwards compatibility. requiredelse
/default
- This defines the optional "default" or "fall through" condition.else
is provided for backwards compatibility.- [string] case - All other cases are defined by your own descriptive name. These names are non-functional and purely for organizational purposes. They will render as an
else if
in the rsyslog configuration.
expression
- The string "expression" that will be used to match values. With all the potential options for logic, this was the easiest way to provide everyone with what they may need.tasks
- A hash of actions to take in the event of a filter match.- All sub-keys for the
tasks
hash maps to another rsyslog configuration object.
- All sub-keys for the
Old Syntax (still works):
class { 'rsyslog::config':
'expression_filters' => {
'hostname_filter' => {
'conditionals' => {
'if' => {
'expression' => '$msg contains "error"',
'tasks' => [
{
'action' => {
'name' => 'omfile_error',
'type' => 'omfile',
'config' => { 'specifics' => '/var/log/errlog' }
}
}
]
}
}
}
}
}
New Syntax:
class { 'rsyslog::config':
'expression_filters' => {
'hostname_filter' => {
'conditionals' => {
'main' => {
'expression' => '$msg contains "error"',
'tasks' => [
{
'action' => {
'name' => 'omfile_error',
'type' => 'omfile',
'config' => { 'specifics' => '/var/log/errlog' }
}
}
]
}
}
}
}
}
Old syntax (still works):
rsyslog::config::expression_filters:
hostname_filter:
conditionals:
# Uses the "if" keyword
if:
expression: '$msg contains "error"'
tasks:
- action:
name: omfile_error
type: omfile
config:
specifics: /var/log/errlog
New syntax:
rsyslog::config::expression_filters:
hostname_filter:
conditionals:
# Uses the "main" keyword
main:
expression: '$msg contains "error"'
tasks:
- action:
name: omfile_error
type: omfile
config:
specifics: /var/log/errlog
both will produce:
if $msg contains "error" then {
action(type="omfile" specifics="/var/log/errlog")
}
NOTE: Due to the amount of potential options available to the user, the expression
key is a plain text string field and the expression logic must be written out. See next example for more details.
Old Syntax (still works):
class { 'rsyslog::config':
'expression_filters' => {
'complex_filter' => {
'conditionals' => {
'if' => {
'expression' => '$syslogfacility-text == "local0" and $msg startswith "DEVNAME" and ($msg contains "error1" or $msg contains "error0")',
'tasks' => [
{ 'stop' => true }
]
},
'else' => {
'tasks' => [
'action' => {
'name' => 'error_log',
'type' => 'omfile',
'config' => { 'specifics' => '/var/log/errlog' }
}
]
}
}
}
}
}
New Syntax:
class { 'rsyslog::config':
'expression_filters' => {
'complex_filter' => {
'conditionals' => {
'main' => {
'expression' => '$syslogfacility-text == "local0" and $msg startswith "DEVNAME" and ($msg contains "error1" or $msg contains "error0")',
'tasks' => [
{ 'stop' => true }
]
},
'default' => {
'tasks' => [
'action' => {
'name' => 'error_log',
'type' => 'omfile',
'config' => { 'specifics' => '/var/log/errlog' }
}
]
}
}
}
}
}
Old Syntax (still works):
rsyslog::config::expression_filters:
complex_filter:
conditionals:
# Uses the "if" keyword
if:
expression: '$syslogfacility-text == "local0" and $msg startswith "DEVNAME" and ($msg contains "error1" or $msg contains "error0")'
tasks:
- stop: true
# Uses the "else" keyword
else:
tasks:
- action:
name: error_log
type: omfile
config:
specifics: /var/log/errlog
New Syntax:
rsyslog::config::expression_filters:
complex_filter:
conditionals:
# Uses the "main" keyword
main:
expression: '$syslogfacility-text == "local0" and $msg startswith "DEVNAME" and ($msg contains "error1" or $msg contains "error0")'
tasks:
- stop: true
# Uses the "default" keyword
default:
tasks:
- action:
name: error_log
type: omfile
config:
specifics: /var/log/errlog
both will produce:
if $syslogfacility-text == "local0" and $msg startswith "DEVNAME" and ($msg contains "error1" or $msg contains "error0") then {
stop
}
else {
action(type="omfile" specifics="/var/log/errlog")
}
Example using more than two conditions:
class { 'rsyslog::config':
'expression_filters' => {
'conditionals' => {
'main' => {
'expression' => '$syslogfacility-text == "local0" and $msg startswith "DEVNAME" and ($msg contains "error1" or $msg contains "error0")',
'tasks' => [{ 'stop' => true }]
},
'errlog' => {
'expression' => '$msg contains "error"',
'tasks' => [
{
'action' => {
'name' => 'omfile_error',
'type' => 'omfile',
'config' => { 'specifics' => '/var/log/errlog' }
}
}
]
},
'default' => {
'tasks' => [
{
'action' => {
'name' => 'system_log',
'type' => 'omfile',
'config' => { 'specifics' => '/var/log/system' }
}
}
]
}
}
}
}
rsyslog::config::expression_filters:
complex_filter:
conditionals:
# Uses the "main" keyword
main:
expression: '$syslogfacility-text == "local0" and $msg startswith "DEVNAME" and ($msg contains "error1" or $msg contains "error0")'
tasks:
- stop: true
# Uses a descriptive keyname
errlog:
expression: '$msg contains "error"'
tasks:
- action:
name: omfile_error
type: omfile
config:
- specifics: /var/log/errlog
# Uses the "default" keyword
default:
tasks:
- action:
name: system_log
type: omfile
config:
specifics: /var/log/system
will produce:
if $syslogfacility-text == "local0" and $msg startswith "DEVNAME" and ($msg contains "error1" or $msg contains "error0") then {
stop
} else if $msg == "error" then {
action(type="omfile" specifics="/var/log/errlog")
} else {
action(type="omfile" specifics="/var/log/system")
}
Legacy config support is provided to facilitate backwards compatibility with sysklogd
format as this module mainly supports rainerscript
format.
A hash of hashes, each hash name is used as the comment/reference for the setting and the hash will have the following values:
key
: the key/logger rule settingvalue
: the value/target of the settingtype
: the type of format to use (legacy or sysklogd), if omitted sysklogd is used. If legacy type is usedkey
can be skipped and one long string can be provided as value.
class { 'rsyslog::config':
'legacy_config' => {
'auth_priv_rule' => {
'key' => 'auth,authpriv.*',
'value' => '/var/log/auth.log',
},
'auth_none_rule' => {
'key' => '*.*;auth,authpriv.none',
'value' => '/var/log/syslog',
},
'syslog_all_rule' => {
'key' => 'syslog.*',
'value' => '/var/log/rsyslog.log',
},
'mail_error_rule' => {
'key' => 'mail.err',
'value' => '/var/log/mail.err',
},
'news_critical_rule' => {
'key' => 'news.crit',
'value' => '/var/log/news/news.crit',
}
}
}
rsyslog::config::legacy_config:
auth_priv_rule:
key: "auth,authpriv.*"
value: "/var/log/auth.log"
auth_none_rule:
key: "*.*;auth,authpriv.none"
value: "/var/log/syslog"
syslog_all_rule:
key: "syslog.*"
value: "/var/log/rsyslog.log"
mail_error_rule:
key: "mail.err"
value: "/var/log/mail.err"
news_critical_rule:
key: "news.crit"
value: "/var/log/news/news.crit"
will produce
# auth_priv_rule
auth,authpriv.* /var/log/auth.log
# auth_none_rule
*.*;auth,authpriv.none /var/log/syslog
# syslog_all_rule
syslog.* /var/log/rsyslog.log
# mail_error_rule
mail.err /var/log/mail.err
# news_critical_rule
news.crit /var/log/news/news.crit
legacy type values can be passed as one long string skipping the key parameter like below and you can also override the priority in the hash to rearrange the contents eg:
emergency_rule:
key: "*.emerg"
value: ":omusrmsg:*"
testing_legacy_remotelog:
value: "*.* @@logmonster.cloudfront.net:1514"
type: "legacy"
priority: 12
testing_legacy_rule:
value: "*.* >dbhost,dbname,dbuser,dbpassword;dbtemplate"
type: "legacy"
will produce
# emergency_rule
*.emerg :omusrmsg:*
# testing_legacy_rule
*.* >dbhost,dbname,dbuser,dbpassword;dbtemplate
# testing_legacy_remotelog
*.* @@logmonster.cloudfront.net:1514
All rsyslog object types are positioned according to the default variables (see Ordering). The position can be overridden for any object by adding the optional priority
parameter.
eg:
rsyslog::config::actions:
elasticsearch:
type: omelasticsearch
config:
queue.type: "linkedlist"
queue.spoolDirectory: "/var/log/rsyslog/queue"
priority: 35
This module attempts to abstract rainerscript objects into data structures that can be handled easily within hiera, however there are clearly times when you need to add some more code structure around an object, such as conditionals. For simple code additions, the template
, action
, input
and global_config
object types support the optional parameter of format
which takes Puppet EPP formatted template as a value, using the variable $content
to signify the object itself. For example, to wrap an action in a simple conditional you could format it as
eg:
rsyslog::config::actions:
elasticsearch:
type: omelasticsearch
config:
queue.type: "linkedlist"
queue.spoolDirectory: "/var/log/rsyslog/queue"
format: |
if [ $fromhost == "foo.localdomain"] then {
<%= $content %>
}
For more complicated code structures that don't lend themselves well to a structured format, like multiple nested conditionals there is also a special configuration object type called custom_config. custom_config
takes two arguments, priority
to determine where in the file it should be configured, and content
a text string to insert. By default the priority is set by the custom_config_priority
parameter (see Ordering)
rsyslog::config::custom_config:
localhost_action:
priority: 45
content: |
if $fromhost == ["foo.localdomain","localhost"] then {
action(type="omfile" file="/var/log/syslog")
} else {
action(type="omelasticsearch"
queue.type="linkedlist"
queue.spoolDirectory="/var/log/rsyslog/queue"
)
}
stop:
content: |
if [ $fromhost == "foo" ] then stop
- Designed specifically for Rsyslog 8+ and the Rainerscript configuration format. Legacy configuration/Rsyslog < 8 support requires the use of the
custom_config
parameter. - The upstream repository for EL8 is currently broken and will not work.
- This module is licensed under Apache 2.0, see LICENSE for more details
- This module is maintained by Vox Pupuli. It was originally written by Craig Dunn (craig@craigdunn.org) @crayfishx.