Conversation
notes: rsyslog will have to handle multiple output streams, 1) loggly and 2) an output channel for logrotation example here: http://www.rsyslog.com/doc/v7-stable/tutorials/log_rotation_fix_size.html?highlight=size%20rotation
…ommit ; git push)
… rotated log it sees in /var/log.
|
can we move |
|
there's also a significant flaw w/ this. basically, you're using the same script that has a wildcard in it to rotate the logs. so if any log that rsyslog is being used for reaches the max size, it'll rotate all of them |
…yslog to support multiple apps per instance (as one does).
There was a problem hiding this comment.
probably move this down below git_node_service (since this runs in order), or maybe just closer. if you add tags here, then you don't need the tags field all over your loggly/tasks/main.yml
There was a problem hiding this comment.
moved it below git_node_service but i'm not sure what i want to do with the tags per se so will leave as is (IINBDFI).
There was a problem hiding this comment.
saw your other comment, and this change. starting a new discussion thread. why does the [0-9] come into play? who is creating those files?
There was a problem hiding this comment.
according to rsyslog the default rotate pattern should be "." just like traditional syslog file rotation. but it occurred to me that if i'm always compressing .0, then i'll keep overwriting the prior rotation (which is not terrible but not necessarily what we want).
so...... i'm going to make a change to only run the bzip2 for files > 24 hours? 48 hours? let's do 48 hours for now.
There was a problem hiding this comment.
oh, btw..
bash-3.2$ find . -name ''a.[0-9]+;' bash-3.2$ find . -name ''a.{0-999}'' bash-3.2$ ls a.0 a.1 a.12 a.2 bash-3.2$ shopt -s extglob bash-3.2$ find . -name ''a.+([0-9])'' bash-3.2$ ls a.+([0-9]) a.0 a.1 a.12 a.2
There was a problem hiding this comment.
We don't really care about calling bzip9 again on an already compressed file because bzip2 will just ignore it and move on to the next file:
bash-3.2$ bzip2 -9 /tmp/pewp/a.[0-9]* bzip2: Input file /tmp/pewp/a.0.bz2 already has .bz2 suffix.
…mpress them > 24hrs and remove them > 72 hours.
…their main playbook (or down the road we just automatically include it via some ansible sorcery). shipped.
|
alright boys, let's ship this sucker. |
|
@und1sk0 walked me through these changes. They LGTM. Important note we must send logs to loggly over SSL before we launch. |
There was a problem hiding this comment.
Did you confirm that 1mb is acceptable?
There was a problem hiding this comment.
this was a copy from our existing loggly config, I set it to 1MB for out log sizes
|
I want to see this working before merge, just for sanity |
|
At long last... |
5 participants
Create an output channel for app logs that rotate based on file-size for sources tagged with syslog facility local7.
Remove input channel tailing raw JSON output from Bunyan.
Manage old logs with a rotate script that compresses logs as they are rotated and removes anything > 72hrs.
ETA - looks like the Ansible is... totally wrong ;)
Update - the rotation script has to truncate the logfile, and rsyslog should not drop privs in /var/log.
test
code review
if no checkboxes with your name are present, and you are reviewing this PR please add your name
if no checkboxes are checked, then just check your checkbox
if there is one checkbox already checked, and you are ok with change, check your box and merge