Documentation update for syslog deployments

docs/rsyslog_deployment.rst

@@ -20,7 +20,7 @@ Introduced with the Nmon Performance Monitor 1.6.14, you can now get real time N
 
 * Splunk Universal or Heavy Forwarder instance installed in rsyslog servers (collector or additional relays) that will monitor and send data in a "key=value" format to Splunk
 
-* Optionally, a deployment tool manager like (Ansible, Chef…) is recommended, note that Playbooks for Ansible are provided with the nmon-logger package
+* Deploying the nmon-logger package to your end-servers, the package is now provided in **rpm** and **deb** packages
 
 *Please review requirements in the above section.
 
@@ -35,38 +35,37 @@ In about 5 minutes, have a running and automated deployment working !
 Key concepts of an rsyslog deployment
 """""""""""""""""""""""""""""""""""""
 
-**Why an rsyslog topology versus Universal Forwarders deployment ?:**
+**Why an syslog topology versus Universal Forwarders deployment ?:**
 
-At first, this provides a powerful and resilient alternative way to deploy the Nmon Performance monitor
-Syslog is a Unix / Linux standard, and available on many servers
-Because you may already have an rsyslog centralization available and you do not want to deploy any additional software on servers
-Because sometimes companies want to rely on standard tools, deploying Universal Forwarders is not an option
+* At first, this provides a powerful and resilient alternative way to deploy the Nmon Performance monitor
+* Syslog is a Unix / Linux standard, and available on many servers
+* Because you may already have an rsyslog centralization available and you do not want to deploy any additional software on servers
+* Because sometimes companies don't want to rely on proprietary software on end servers, deploying Universal Forwarders is not an option
 
 **Key concepts :**
 
-100% of Application functionality over a traditional Universal Forwarders deployment
-In a standard deployment of the TA-nmon and the Nmon Performance application, hosts generates Nmon Performance in csv structured data
-To be transported over syslog, csv data are being transformed in a key=value format
-While csv structured data is known in Splunk as "an indexing field structure", key=value format will be extracted on the fly (parsing at extraction time)
-csv structured data has an higher disk space cost but offers a very low level of volume of data to index (and so a low level of licence cost), and offers best performances at search time in SPL (Search Language Processing)
-key=value format generates an higher volume of data and requires more power at extraction time, but it does not generate indexed fields, this also represents less disk space
-Nmon Performance massively uses data model acceleration, key=value searching performance versus csv structured data will not be different from standard deployment
+* 100% of Application features over a traditional Universal Forwarders deployment
+* In a standard deployment of the TA-nmon and the Nmon Performance application, hosts generates Nmon Performance in csv structured data
+* To be transported over syslog, csv data are being transformed in a key=value format
+* While csv structured data is known in Splunk as "an indexing field structure", key=value format will be extracted on the fly (parsing at extraction time)
+* csv structured data has an higher disk space cost but offers a very low level of volume of data to index (and so a low level of licence cost), and offers best performances at search time in SPL (Search Language Processing)
+* key=value format generates an higher volume of data and requires more power at extraction time, but it does not generate indexed fields, this also represents less disk space
+* Nmon Performance massively uses data model acceleration, key=value searching performance versus csv structured data will not be different from standard deployment
 
-**Can i send data directory from rsyslog to Splunk ? Why is not a recommended scenario:**
+**Can i send data directory from syslog to Splunk ?**
 
-Yes it is possible, but this is not the deployment topology i would recommend for multiple reasons, as exposed above:
+*Yes it is possible, but this is not the deployment topology i would recommend for multiple reasons, as exposed above:*
 
-You cannot guarantee that rsyslog will send data in the order it should, this is especially true with multi-line events that could be mixed between hosts
-rsyslog can easily identify the host origin of the incoming data and generate per host files, which guarantees management of Nmon data (like large multi-line configuration data)
-For resilient reasons, once it is configured, you will few often restart rsyslog. (most often while rebooting the machine)
-This is less true with Splunk as you will want to upgrade it from time to time, or deploy new configuration or application to manage new data
-Finally rsyslog speaks to rsyslog, with the same native implementation and functionalities
+* You cannot guarantee that rsyslog will send data in the order it should, this is especially true with multi-line events that could be mixed between hosts
+* rsyslog can easily identify the host origin of the incoming data and generate per host files, which guarantees management of Nmon data (like large multi-line configuration data)
+* For resilient reasons, once it is configured, you will few often restart rsyslog. (most often while rebooting the machine)
+* This is less true with Splunk as you will want to upgrade it from time to time, or deploy new configuration or application to manage new data
+* Finally syslog speaks to syslog, with the same native implementation and features
 
 **What about the nmon-logger deployment management ?:**
 
-In standard Universal Forwarder management, you can easily rely on native Splunk deployment servers to push and maintain the TA-nmon package
-This is not true in a syslog deployment, and you will need to deploy manually, or much most probably using your deployment management tools (Pupper, Chef, Ansible, custom scripts…)
-The nmon-logger comes with Ansible Playbooks example you can easily adapt to your context, and use for auto deployments !
+* In standard Universal Forwarder management, you can easily rely on native Splunk deployment servers to push and maintain the TA-nmon package
+* In the syslog deployment, you will deploy nmon-logger packages (rpm, deb) or deploy nmon-logger manually
 
 **Topology: Examples of possible implementations:**
 
@@ -86,21 +85,21 @@ The nmon-logger comes with Ansible Playbooks example you can easily adapt to you
 PRE-REQUISITES:
 +++++++++++++++
 
-**Splunk + Nmon Performance app:**
+* Splunk + Nmon Performance app:
 
 First of all, have a Splunk working installation, and the Nmon Performance up and running ! (yeah, songs like an evidence :-)
 
 Some specific requirements must be respected to achieve a deployment that uses rsyslog as the transport layer:
 
-**RSYSLOG V8.x:**
+* RSYSLOG V8.x::
 
-For obvious reasons, Rsyslog v8.x is required to forward Nmon Performance data to centralized rsyslog servers, see:
+RSYSLOG V8.x is required to forward Nmon Performance data to centralized rsyslog servers, see:
 
 http://www.rsyslog.com/
 
-On end servers, the "imfile" plugin will be used to read and collect Nmon Performance data.
+On end servers, the “imfile” plugin will be used to read and collect Nmon Performance data.
 
-**PYTHON 2.7.x or PERL with the module Time::HiRes:**
+* Python 2.7.x **OR** Perl with the module Time::HiRes:
 
 The nmon-logger will by default search for a Python 2.7.x environment.
 If it is not available, scripts will use Perl, when using Perl note that the Time::HiRes module is required.
@@ -194,8 +193,6 @@ STEP 2 : Rsyslog configuration for end servers
 
 Each of your end servers must be configured to send its syslog data to the central rsyslog server.
 
-Having a deployment tool in place like Ansible is a very good idea :-)
-
 **First, let's activate the imfile module that will be used to read and send Nmon Performance data:**
 
 ::
@@ -243,11 +240,20 @@ Immediately after the restart, rsyslog starts to forward data to central rsyslog
 STEP 3 : Deploy the nmon-logger to your end servers
 +++++++++++++++++++++++++++++++++++++++++++++++++++
 
-On each end server, you must deploy the "nmon-looger" package:
+On each end server, you must deploy the "nmon-logger" package:
 
 https://github.com/guilhemmarchand/nmon-logger
 
-Ansible Playbooks are available in the Git repository, with Ansible the nmon-logger package is being totally deployed, up and running in a few seconds !!!
+Using your package manager
+**************************
+
+For compatible operating systems using the "deb" Debian package manager (Debian, Ubuntu...) and the "rpm" Redhat package manager (CentOS, RHEL...) you can easily deploy the pre-configured package matching your system:
+
+* https://github.com/guilhemmarchand/nmon-logger/tree/master/deb for deb packages
+* https://github.com/guilhemmarchand/nmon-logger/tree/master/rpm for rpm packages
+
+Manual deployment
+*****************
 
 **Deploying manually must be achieve the following way:**
 
@@ -396,6 +402,8 @@ Deploy the TA-nmon to your instance
     index = nmon
     sourcetype = nmon_processing:fromsyslog
     source = nmon_processing:syslog
+    # Wait additional time to avoid incorrect event breaking
+    multiline_event_extra_waittime = true
 
 **Restart Splunk**
 

docs/syslogng_deployment.rst

@@ -16,9 +16,13 @@ Introduced with the Nmon Performance Monitor 1.6.14, you can now get real time N
 **This will be achieved using:**
 
 * The nmon-logger package to be deployed on servers: https://github.com/guilhemmarchand/nmon-logger
+
 * syslog-ng locally available on servers and configured to send data to central syslog servers
+
 * Splunk Universal or Heavy Forwarder instance installed in syslog-ng servers (collector or additional relays) that will monitor and send data in a "key=value" format to Splunk
 
+* Deploying the nmon-logger package to your end-servers, the package is now provided in **rpm** and **deb** packages
+
 *Optionally, a deployment tool manager like (Ansible, Chef…) is recommended, note that Playbooks for Ansible are provided with the nmon-logger package*
 
 **Please review requirements in the above section.**
@@ -38,33 +42,32 @@ In about 5 minutes, have a running and automated deployment working !
 * At first, this provides a powerful and resilient alternative way to deploy the Nmon Performance monitor
 * Syslog is a Unix / Linux standard, and available on many servers
 * Because you may already have an syslog-ng centralization available and you do not want to deploy any additional software on servers
-* Because sometimes companies want to rely on standard tools, deploying Universal Forwarders is not an option
+* Because sometimes companies don't want to rely on proprietary software on end servers, deploying Universal Forwarders is not an option
 
 **Key concepts :**
 
-* 100% of Application functionalities over a traditional Universal Forwarders deployment
+* 100% of Application features over a traditional Universal Forwarders deployment
 * In a standard deployment of the TA-nmon and the Nmon Performance application, hosts generates Nmon Performance in csv structured data
 * To be transported over syslog, csv data are being transformed in a key=value format
 * While csv structured data is known in Splunk as "an indexing field structure", key=value format will be extracted on the fly (parsing at extraction time)
 * csv structured data has an higher disk space cost but offers a very low level of volume of data to index (and so a low level of licence cost), and offers best performances at search time in SPL (Search Language Processing)
 * key=value format generates an higher volume of data and requires more power at extraction time, but it does not generate indexed fields, this also represents less disk space
 * Nmon Performance massively uses data model acceleration, key=value searching performance versus csv structured data will not be different from standard deployment
 
-**Can i send data directory from syslog-ng to Splunk ? Why is not a recommended scenario:**
+**Can i send data directory from syslog to Splunk ?**
 
 *Yes it is possible, but this is not the deployment topology i would recommend for multiple reasons, as exposed above:*
 
 * You cannot guarantee that syslog-ng will send data in the order it should, this is especially true with multi-line events that could be mixed between hosts
 * syslog-ng can easily identify the host origin of the incoming data and generate per host files, which guarantees management of Nmon data (like large multi-line configuration data)
 * For resilient reasons, once it is configured, you will few often restart syslog-ng. (most often while rebooting the machine)
 * This is less true with Splunk as you will want to upgrade it from time to time, or deploy new configuration or application to manage new data
-* Finally syslog speaks to syslog, with the same native implementation and functionalities
+* Finally syslog speaks to syslog, with the same native implementation and features
 
 **What about the nmon-logger deployment management ?:**
 
 * In standard Universal Forwarder management, you can easily rely on native Splunk deployment servers to push and maintain the TA-nmon package
-* This is not true in a syslog deployment, and you will need to deploy manually, or much most probably using your deployment management tools (Pupper, Chef, Ansible, custom scripts…)
-* The nmon-logger comes with Ansible Playbooks example you can easily adapt to your context, and use for auto deployments !
+* In the syslog deployment, you will deploy nmon-logger packages (rpm, deb) or deploy nmon-logger manually
 
 **Topology: Examples of possible implementations:**
 
@@ -84,19 +87,19 @@ In about 5 minutes, have a running and automated deployment working !
 PRE-REQUISITES:
 +++++++++++++++
 
-**Splunk + Nmon Performance app:**
+* Splunk + Nmon Performance app:
 
 First of all, have a Splunk working installation, and the Nmon Performance up and running ! (yeah, songs like an evidence :-)
 
 Some specific requirements must be respected to achieve a deployment that uses syslog-ng as the transport layer:
 
-**SYSLOG-NG (V3.x minimal recommended):**
+* SYSLOG-NG (V3.x minimal recommended):
 
 Syslog-ng is required to forward Nmon Performance data to centralized syslog-ng servers, see:
 
 https://syslog-ng.org/
 
-**PYTHON 2.7.x or PERL with the module Time::HiRes:**
+* Python 2.7.x **OR** Perl with the module Time::HiRes:
 
 The nmon-logger will by default search for a Python 2.7.x environment.
 If it is not available, scripts will use Perl, when using Perl note that the Time::HiRes module is required.
@@ -224,8 +227,6 @@ STEP 2 : syslog-ng configuration for end servers:
 
 Each of your end servers must be configured to send its syslog data to the central syslog-ng server.
 
-Having a deployment tool in place like Ansible is a very good idea :-)
-
 **Create the central client configuration that forwards local log to central servers:**
 
 */etc/syslog-ng/conf.d/client.conf*
@@ -256,6 +257,17 @@ On each end server, you must deploy the "nmon-logger" package:
 
 https://github.com/guilhemmarchand/nmon-logger
 
+Using your package manager
+**************************
+
+For compatible operating systems using the "deb" Debian package manager (Debian, Ubuntu...) and the "rpm" Redhat package manager (CentOS, RHEL...) you can easily deploy the pre-configured package matching your system:
+
+* https://github.com/guilhemmarchand/nmon-logger/tree/master/deb for deb packages
+* https://github.com/guilhemmarchand/nmon-logger/tree/master/rpm for rpm packages
+
+Manual deployment
+*****************
+
 Ansible Playbooks are available in the Git repository, with Ansible the nmon-logger package is being totally deployed, up and running in a few seconds !!!
 
 **Deploying manually must be achieve the following way:**
@@ -404,6 +416,8 @@ STEP 4 : Splunk it !
     index = nmon
     sourcetype = nmon_processing:fromsyslog
     source = nmon_processing:syslog
+    # Wait additional time to avoid incorrect event breaking
+    multiline_event_extra_waittime = true
 
 **Restart Splunk**