Skip to content

/ install-script

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add TLS support for file monitoring #88

Merged
merged 13 commits into from Oct 10, 2017
+93 −40
Merged

Add TLS support for file monitoring #88

Changes from all commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
e6c079d
add TLS support to file monitoring script
Apr 12, 2017
fef869b
Update rsyslog-gnutls package downloading in case of TLS only
Apr 13, 2017
fad4d39
Stop statement is needed in the configuration
Apr 13, 2017
1619b2d
Update Linux script link
Apr 13, 2017
c1973df
Update configure-file-monitoring.sh
Apr 13, 2017
a713c3c
Modified some issues in 8+ configuration
Apr 17, 2017
40947d5
Update script version
Apr 17, 2017
01589fa
Update configure-linux.sh source link
Apr 19, 2017
8b74a32
Add line breaks in the script's configuration
May 17, 2017
69a9a52
Merge branch 'master' into tls-support-filemonitoring
May 29, 2017
1b3e9ba
modify-configuration-backup-file-path
Aug 29, 2017
a9ff39e
Remove unnecessary spaces
Sep 1, 2017
beeade4
Merge branch 'master' into tls-support-filemonitoring
Oct 10, 2017
File filter...
Filter file types
Jump to…
Jump to file
Failed to load files.

Always

Just for now

133 Modular Scripts/File Monitoring/configure-file-monitoring.sh
View file
Open in desktop
@@ -9,7 +9,7 @@ source configure-linux.sh "being-invoked"
#name of the current script
SCRIPT_NAME=configure-file-monitoring.sh
#version of the current script
SCRIPT_VERSION=1.13
SCRIPT_VERSION=1.14

This comment has been minimized.

Sign in to view
@Shwetajain148

Shwetajain148 Apr 17, 2017
Author Contributor

@mchaudhary @mostlyjason Please let me know if 1.14 is not suitable.

#file to monitor (contains complete path and file name) provided by user
LOGGLY_FILE_TO_MONITOR=
@@ -47,6 +47,9 @@ FILE_TO_MONITOR=
IS_DIRECTORY=

IS_WILDCARD=

FILE_TLS_SENDING="true"

########## Variable Declarations - End ##########

# executing the script for loggly to install and configure syslog
@@ -72,16 +75,16 @@ installLogglyConfForFile()

#construct variables using filename and filealias
constructFileVariables

#check if the alias is already taken
checkIfFileAliasExist

#check for the log file size
checkLogFileSize $LOGGLY_FILE_TO_MONITOR

#checks if the file has proper read permission
checkFileReadPermission

#configure loggly for Linux
installLogglyConf

@@ -151,7 +154,7 @@ constructFileVariables()
FILE_SYSLOG_CONFFILE="$RSYSLOG_ETCDIR_CONF/21-filemonitoring-$FILE_ALIAS.conf"

#conf file backup name
FILE_SYSLOG_CONFFILE_BACKUP="$FILE_ALIAS.loggly.bk"
FILE_SYSLOG_CONFFILE_BACKUP="$RSYSLOG_ETCDIR_CONF/$FILE_ALIAS.loggly.bk"

#application tag
APP_TAG="\"file-alias\":\"$LOGGLY_FILE_TO_MONITOR_ALIAS\""
@@ -400,49 +403,94 @@ write21ConfFileContents()
logMsgToConfigSysLog "INFO" "INFO: Creating file $FILE_SYSLOG_CONFFILE"
sudo touch $FILE_SYSLOG_CONFFILE
sudo chmod o+w $FILE_SYSLOG_CONFFILE

imfileStr="
\$ModLoad imfile
\$InputFilePollInterval 10
\$WorkDirectory $RSYSLOG_DIR
"
if [[ "$LINUX_DIST" == *"Ubuntu"* ]]; then
imfileStr+="\$PrivDropToGroup adm
"
fi

rsyslog_version="$(rsyslogd -v)"
r_ver=${rsyslog_version:9:1}
if [ $r_ver -le 6 ]
if [ $r_ver -le 7 ]
then
imfileStr+="
# File access file:
\$InputFileName $FILE_TO_MONITOR
\$InputFileTag $LOGGLY_FILE_TO_MONITOR_ALIAS:
\$InputFileStateFile stat-$STATE_FILE_ALIAS
\$InputFileSeverity info
\$InputFilePersistStateInterval 20000
\$InputRunFileMonitor
#Add a tag for file events
\$template $CONF_FILE_FORMAT_NAME,\"<%pri%>%protocol-version% %timestamp:::date-rfc3339% %HOSTNAME% %app-name% %procid% %msgid% [$LOGGLY_AUTH_TOKEN@41058 $TAG] %msg%\n\"
if \$programname == '$LOGGLY_FILE_TO_MONITOR_ALIAS' then @@logs-01.loggly.com:514;$CONF_FILE_FORMAT_NAME
if \$programname == '$LOGGLY_FILE_TO_MONITOR_ALIAS' then ~
imfileStr="
\$ModLoad imfile
\$InputFilePollInterval 10
\$WorkDirectory $RSYSLOG_DIR
\$ActionSendStreamDriver gtls
\$ActionSendStreamDriverMode 1
\$ActionSendStreamDriverAuthMode x509/name
\$ActionSendStreamDriverPermittedPeer *.loggly.com
#RsyslogGnuTLS
\$DefaultNetstreamDriverCAFile /etc/rsyslog.d/keys/ca.d/logs-01.loggly.com_sha12.crt
# File access file:
\$InputFileName $FILE_TO_MONITOR
\$InputFileTag $LOGGLY_FILE_TO_MONITOR_ALIAS
\$InputFileStateFile stat-$STATE_FILE_ALIAS
\$InputFileSeverity info
\$InputFilePersistStateInterval 20000
\$InputRunFileMonitor
#Add a tag for file events
template (name=\"$CONF_FILE_FORMAT_NAME\" type=\"string\" string=\"<%pri%>%protocol-version% %timestamp:::date-rfc3339% %HOSTNAME% %app-name% %procid% %msgid% [$LOGGLY_AUTH_TOKEN@41058 $TAG] %msg%\n\")

This comment has been minimized.

Sign in to view
@mostlyjason

mostlyjason Aug 30, 2017
Contributor

How is $TAG set?

This comment has been minimized.

Sign in to view
@Shwetajain148

Shwetajain148 Sep 1, 2017
Author Contributor

@mostlyjason, From the very beginning, the default tag i.e. file is set into the LOGGLY_FILE_TAG variable if the user has not provided one externally using the parameter -tag in the command. Please refer the below code line-

Ref: https://github.com/loggly/install-script/blob/master/Modular%20Scripts/File%20Monitoring/configure-file-monitoring.sh#L37

Now, the user can pass multiple tags externally which get stored into the same LOGGLY_FILE_TAG variable separated by a comma and iterate inside the addTagsInConfiguration function and finally set into the $TAG variable. Please refer the below code line-

Ref: https://github.com/loggly/install-script/blob/master/Modular%20Scripts/File%20Monitoring/configure-file-monitoring.sh#L347

Once $TAG is set with either one or multiple tags, the same variable is used in the file-monitoring configuration. Please refer the below code line-

Ref: https://github.com/loggly/install-script/blob/master/Modular%20Scripts/File%20Monitoring/configure-file-monitoring.sh#L427

This is how $TAG set.

This comment has been minimized.

Sign in to view
@mostlyjason

mostlyjason Sep 1, 2017
Contributor

Okay we should document how to set tags in https://www.loggly.com/docs/configure-syslog-script/. I don't see any descriptions for that parameter

This comment has been minimized.

Sign in to view
@Shwetajain148

Shwetajain148 Sep 4, 2017
Author Contributor

@mostlyjason, I have added a Note to tell the users that how they can set the tag for all three file-monitoring, Nginx and Apache script on the document https://www.loggly.com/docs/configure-syslog-script/. I added this information after the Script Usage section, please review.
if \$programname == '$LOGGLY_FILE_TO_MONITOR_ALIAS' then action(type=\"omfwd\" protocol=\"tcp\" target=\"logs-01.loggly.com\" port=\"6514\" template=\"$CONF_FILE_FORMAT_NAME\")
if \$programname == '$LOGGLY_FILE_TO_MONITOR_ALIAS' then stop
"
imfileStrNonTls="
\$ModLoad imfile
\$InputFilePollInterval 10
\$WorkDirectory $RSYSLOG_DIR
# File access file:
\$InputFileName $FILE_TO_MONITOR
\$InputFileTag $LOGGLY_FILE_TO_MONITOR_ALIAS
\$InputFileStateFile stat-$STATE_FILE_ALIAS
\$InputFileSeverity info
\$InputFilePersistStateInterval 20000
\$InputRunFileMonitor
#Add a tag for file events
template (name=\"$CONF_FILE_FORMAT_NAME\" type=\"string\" string=\"<%pri%>%protocol-version% %timestamp:::date-rfc3339% %HOSTNAME% %app-name% %procid% %msgid% [$LOGGLY_AUTH_TOKEN@41058 $TAG] %msg%\n\")
if \$programname == '$LOGGLY_FILE_TO_MONITOR_ALIAS' then action(type=\"omfwd\" protocol=\"tcp\" target=\"logs-01.loggly.com\" port=\"514\" template=\"$CONF_FILE_FORMAT_NAME\")
if \$programname == '$LOGGLY_FILE_TO_MONITOR_ALIAS' then stop
"
else
imfileStr+="
# File access file:
\$InputFileName $FILE_TO_MONITOR
\$InputFileTag $LOGGLY_FILE_TO_MONITOR_ALIAS
\$InputFileStateFile stat-$STATE_FILE_ALIAS
\$InputFileSeverity info
\$InputFilePersistStateInterval 20000
\$InputRunFileMonitor
#Add a tag for file events
template (name=\"$CONF_FILE_FORMAT_NAME\" type=\"string\" string=\"<%pri%>%protocol-version% %timestamp:::date-rfc3339% %HOSTNAME% %app-name% %procid% %msgid% [$LOGGLY_AUTH_TOKEN@41058 $TAG] %msg%\n\")
if \$programname == '$LOGGLY_FILE_TO_MONITOR_ALIAS' then action(type=\"omfwd\" protocol=\"tcp\" target=\"logs-01.loggly.com\" port=\"514\" template=\"$CONF_FILE_FORMAT_NAME\")
if \$programname == '$LOGGLY_FILE_TO_MONITOR_ALIAS' then ~
imfileStr="
module(load=\"imfile\")
#RsyslogGnuTLS
\$DefaultNetstreamDriverCAFile /etc/rsyslog.d/keys/ca.d/logs-01.loggly.com_sha12.crt
# Input for FILE1
input(type=\"imfile\" tag=\"$LOGGLY_FILE_TO_MONITOR_ALIAS\" ruleset=\"filelog\" file=\"$FILE_TO_MONITOR\") #wildcard is allowed at file level only
# Add a tag for file events
template(name=\"$CONF_FILE_FORMAT_NAME\" type=\"string\" string=\"<%pri%>%protocol-version% %timestamp:::date-rfc3339% %HOSTNAME% %app-name% %procid% %msgid% [$LOGGLY_AUTH_TOKEN@41058 $TAG] %msg%\n\")
ruleset(name=\"filelog\"){
action(type=\"omfwd\" protocol=\"tcp\" target=\"logs-01.loggly.com\" port=\"6514\" template=\"$CONF_FILE_FORMAT_NAME\" StreamDriver=\"gtls\" StreamDriverMode=\"1\" StreamDriverAuthMode=\"x509/name\" StreamDriverPermittedPeers=\"*.loggly.com\")
}
"
imfileStrNonTls="
module(load=\"imfile\")
# Input for FILE1
input(type=\"imfile\" tag=\"$LOGGLY_FILE_TO_MONITOR_ALIAS\" ruleset=\"filelog\" file=\"$FILE_TO_MONITOR\") #wildcard is allowed at file level only
# Add a tag for file events
template(name=\"$CONF_FILE_FORMAT_NAME\" type=\"string\" string=\"<%pri%>%protocol-version% %timestamp:::date-rfc3339% %HOSTNAME% %app-name% %procid% %msgid% [$LOGGLY_AUTH_TOKEN@41058 $TAG] %msg%\n\")
ruleset(name=\"filelog\"){
action(type=\"omfwd\" protocol=\"tcp\" target=\"logs-01.loggly.com\" port=\"514\" template=\"$CONF_FILE_FORMAT_NAME\") stop
}
"
fi

if [ $FILE_TLS_SENDING == "false" ];
then
imfileStr=$imfileStrNonTls
fi

#write to 21-<file-alias>.conf file
sudo cat << EOIPFW >> $FILE_SYSLOG_CONFFILE
$imfileStr
@@ -625,6 +673,11 @@ if [ "$1" != "being-invoked" ]; then
CONF_FILE_FORMAT_NAME=$CONF_FILE_FORMAT_NAME$1
echo "File alias: $LOGGLY_FILE_TO_MONITOR_ALIAS"
;;
--insecure )
LOGGLY_TLS_SENDING="false"
FILE_TLS_SENDING="false"
LOGGLY_SYSLOG_PORT=514
;;
-tag| --filetag ) shift
LOGGLY_FILE_TAG=$1
echo "File tag: $LOGGLY_FILE_TAG"
ProTip! Use n and p to navigate between commits in a pull request.
You can’t perform that action at this time.