Logstash Rolling Encrypted Logs
logstash logging encryption
Logstash does not currently support rolling file output. The pipe output can be used with a Python script using logging support which does support rolling files. As an added bonus, each log message is encrypted.
The first requirement is a Python script which received logging messages on stdin and logs them to the Python logging facility.
Encryption is an optional addition to the process. The following script reads messages from stdin, encrypts them, and writes to stdout. The Python Logging script above is agnostic about what it is logging out, so encrypting the content makes no difference there.
pip3 install pycrypto bitstring
Each line is independently encrypted with an IV written to the start of the line. This allows rolling logs and commands like head and tail to be used without worrying about breaking the encryption
To decrypt the log output later use the following script:
Logstash is configured to pipe output through the two Python scripts.