Permalink
Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.
Sign up
| ------------------------------------------------------------------------------ | |
| Version 8.36.0 [v8-stable] 2018-06-26 | |
| - build system change: | |
| Liblogging-stdlog was introduced to provide a broader ability to send rsyslog | |
| internal logs to different sources. However, most distros did not pick up | |
| that capability and so instead we do a regular syslog() call. We assume that | |
| the actual functionality is never used in practice, so we plan to retire it. | |
| That makes building rsyslog from source easier. | |
| The plan is to disable use of liblogging-stdlog by default during | |
| configure. So users (and distros!) can still opt-in to have it enabled if | |
| they desire. | |
| A couple of releases later, we want to completely remove the functionality, | |
| except if there has desire been shown in the meantime which justifies to keep | |
| liblogging-stdlog. | |
| This version disabled liblogging-stdlog by default. We now also | |
| emit a warning message ("liblogging-stdlog will go away") so that users | |
| know what is going on and my react. | |
| closes https://github.com/rsyslog/rsyslog/issues/2705 | |
| see also https://github.com/rsyslog/rsyslog/issues/2706 | |
| - add openssl driver alongside GnuTLS one for TLS communication | |
| The openssl driver is curently experimental. It will become the new preferred | |
| driver as it permits us to provide much better end-user error message than | |
| we could provide with GnuTLS. It is also less picky with certificate files | |
| and provides specific error messages if there are certificate problems. | |
| closes: https://github.com/rsyslog/rsyslog/issues/1390 | |
| closes: https://github.com/rsyslog/rsyslog/issues/1840 | |
| closes: https://github.com/rsyslog/rsyslog/issues/1352 | |
| closes: https://github.com/rsyslog/rsyslog/issues/1702 | |
| closes: https://github.com/rsyslog/rsyslog/issues/2547 | |
| - GnuTLS TLS driver: support intermediate certificates | |
| this is necessary for certificate chains | |
| Thanks to Arne Nordmark for providing the patch. | |
| closes https://github.com/rsyslog/rsyslog/issues/2762 | |
| - omelasticsearch: write op types; bulk rejection retries | |
| * Add support for a 'create' write operation type in addition to | |
| the default 'index'. Using create allows specifying a unique id | |
| for each record, and allows duplicate document detection. | |
| * Add support for checking each record returned in a bulk index | |
| request response. Allow specifying a ruleset to send each failed | |
| record to. Add a local variable `omes` which contains the | |
| information in the error response, so that users can control how | |
| to handle responses e.g. retry, or send to an error file. | |
| * Add support for response stats - count successes, duplicates, and | |
| different types of failures. | |
| * Add testing for bulk index rejections. | |
| Thanks to Richard Megginson for the patch. | |
| - lookup tables: reload message now with "info" severity (was "error") | |
| thanks to Adam Chalkley for the patch | |
| - imjournal: add statistics counter | |
| following statistics counter are now supported by imjournal | |
| - submitted = total number of messages submitted for processing | |
| closes https://github.com/rsyslog/rsyslog/issues/2549 | |
| - config: permit 4-digit file creation modes | |
| permit 4-digit file creation modes (actually 5 with the leading zero) so | |
| that the setgid bit can also be set (and anything else on that position. | |
| closes https://github.com/rsyslog/rsyslog/issues/1092 | |
| - omprog: simplify 'plugin-with-feedback.py' example | |
| Make the skeleton easier to understand by removing transaction support. | |
| Also, transaction failures did not work as explained in the skeleton, | |
| because of issue #2420. In the future, a 'plugin-with-transactions.py' | |
| example can be added, ideally once the issue is solved. | |
| Thanks to Joan Sala for contributing this. | |
| - core: misadressing when writing disk queue files | |
| when writing disk queue files during shutdown, access to freed | |
| memory can occur under these circumstances: | |
| - action A is processing data, but could not complete it | |
| most importantly, the current in-process batch needs not to | |
| be totally completed. Most probable cause for this scenario | |
| is a suspended action in retry mode. | |
| - action A is called from a ruleset RA which | |
| - does not have a queue assigned | |
| - where RA is called from a ruleset RO which is bound | |
| to the input from which the message originated | |
| - RO must be defined before RA inside the expanded config | |
| - Disk queues (or the disk part of a DA queue) must be utilized by A | |
| When re-injecting the unprocessed messages from A into the disk queue, the | |
| name of ruleset RO is accessed (for persisting to disk). However, RO is | |
| already destructed at this point in time. | |
| The patch changes the shutdown processing of rulesets, so that all | |
| shutdown processing is done before any ruleset data is destructed. This | |
| ensures that all data items which potentially need to be accessed | |
| remain valid as long as some part may potentially try to access them. | |
| This follows a the approach used in | |
| https://github.com/rsyslog/rsyslog/pull/1857 | |
| where obviously that part of the problem was not noticed. | |
| see also https://github.com/rsyslog/rsyslog/issues/1122 | |
| closes https://github.com/rsyslog/rsyslog/issues/2742 | |
| - core: fix message loss on target unavailibility during shutdown | |
| Triggering condition: | |
| - action queue in disk mode (or DA) | |
| - batch is being processed by failed action in retry mode | |
| - rsyslog is shut down without resuming action | |
| In these cases messages may be lost by not properly writing them | |
| back to the disk queue. | |
| closes https://github.com/rsyslog/rsyslog/issues/2760 | |
| - imrelp bugfix: error message "librelp too old" is always emitted ... | |
| ... even if librelp is current. The codition check was actually missing. | |
| This commit adds it. | |
| closes https://github.com/rsyslog/rsyslog/issues/2712 | |
| - imrelp: segfault on startup when cert without priv key is configured | |
| closes https://github.com/rsyslog/rsyslog/issues/2747 | |
| - omrelp bugfix: segfault on first message sent when authmode was wrong | |
| A segfault could occur if the authmode was configured to an invalid value. | |
| This is now caught during config processing and an error is reported. | |
| closes https://github.com/rsyslog/rsyslog/issues/2743 | |
| - imfile bugfix: double-free on module shutdown | |
| detected by code review, not seen in practice | |
| - imfile/core bugfix: potential misadressing in string copy routine | |
| This can be exposed via imfile, as follows: | |
| - use a regex to process multiline messages | |
| - configure timeouts | |
| - make sure imfile reads a partial message | |
| - wait so that at least one timeout occurs | |
| - add the message termination sequence | |
| This leads to a misadressing, which may have no obvious effects potentially | |
| up to a segfault. | |
| closes https://github.com/rsyslog/rsyslog/issues/2661 | |
| - imfile bugfix: if freshStartTail is set some initial file lines missing | |
| When the option is set and a new file is created after rsyslog startup, | |
| freshStartTail is also applied to it. That is data written quickly to it | |
| (before rsyslog can process it) will potentially be discarded. If so, | |
| and how much, depends on the timing between rsyslog and the logging process. | |
| This problem is most likely to be seen in polling mode, where a relatively | |
| long time may be required for rsyslog to find the new file. | |
| This is changed so that now freshStartTail only applies to files that | |
| are already-existing during rsyslog's initial processing of the file | |
| monitors. HOWEVER, depending on the number and location (network?) of | |
| existing files, this initial startup processing may take some time as | |
| well. If another process creates a new file at exactly the time of | |
| startup processing and writes data to it, rsyslog might detect this | |
| file and it's data as prexisting and may skip it. This race is inevitable. | |
| So when freshStartTail is used, some risk of data loss exists. The same | |
| holds true if between the last shutdown of rsyslog and its restart log | |
| file content has been added. This is no rsyslog bug if it occurs. | |
| As such, the rsyslog team advises against activating the freshStartTail | |
| option. | |
| closes https://github.com/rsyslog/rsyslog/issues/2464 | |
| - core: fix undefined behaviour (unsigned computation may lead to value < 0) | |
| This was detected by LLVM UBSAN. On some platforms re-setting the rawmsg | |
| inside the message object could lead to invalid computation due to the | |
| fact the the computation was carried out as unsigned and only then | |
| converted to integer. | |
| No known problem in practice. | |
| ------------------------------------------------------------------------------ | |
| Version 8.35.0 [v8-stable] 2018-05-15 | |
| - imptcp: add ability to configure socket backlog | |
| this can be useful under heavy load. | |
| For a detailled discussion see https://github.com/rsyslog/rsyslog/pull/2561 | |
| Thanks to Maxime Graff for implementing this. | |
| - omfile: do not permit filename that only consists of whitespace | |
| - fmhash: new hash function module | |
| implements hash32() and hash64() functions | |
| Thanks to Harshvardhan Shrivastava for implementing these | |
| - some better error messages | |
| - imklog: add ratelimiting capability | |
| On Linux kernel logs are ratelimited only for messages using | |
| printk_ratelimit(). Some logs do not use this facility, so | |
| we ratelimit kernel ourselves. | |
| Thanks to Berend De Schouwer for the patch. | |
| - omkafka: added impstats counters for librdkafka returned statistics | |
| Adds: | |
| * statscallback counters | |
| * librdkafka failure and error counters | |
| * acked message counter | |
| Thanks to Abdul Waheed for implementing this. | |
| - imudp | |
| * use rsyslog message rate-limiter instead of home-grown one | |
| imudp introduced it's own (feature-limited) rate-limiting capability for | |
| message on disallowed senders before we had central rate-limiters | |
| inside rsyslog. Also, that code evolved from running on a single | |
| thread to running on multiple threads, which introduced data races | |
| and so made unreliable. | |
| Now we removed the old rate-limiting capability and depend on the | |
| system rate limiter for internal rsyslog messages. | |
| closes https://github.com/rsyslog/rsyslog/issues/2467 | |
| * add stats counter "disallowed" | |
| counts the number of messages discarded due to being received from | |
| disallowed senders | |
| see also https://github.com/rsyslog/rsyslog/issues/2467 | |
| - imrelp: add parameter "oversizeMode" | |
| Permits to instruct librelp how to handle oversize messages. The new default | |
| is to truncate messages. Previously, the connection was aborted, what often | |
| lead to stuck messages at the sender side. Now, there are three options passed | |
| down to librelp: | |
| * abort - same behavior as previously, connection is aborted on error | |
| * truncate - do not abort but instead truncate oversize message to | |
| configured max size | |
| * accept - accept all oversize messages (note: this can cause security issues, | |
| see doc for details) | |
| see also https://github.com/rsyslog/rsyslog/pull/1525#issuecomment-384179393 | |
| see also https://github.com/rsyslog/rsyslog/issues/2190 | |
| closes https://github.com/rsyslog/rsyslog/issues/2633 | |
| closes https://github.com/rsyslog/rsyslog/issues/1741 | |
| closes https://github.com/rsyslog/rsyslog/issues/1782 | |
| closes https://github.com/rsyslog/rsyslog/issues/2496 | |
| - core: consistent handling of oversize input messages | |
| In the community we frequently discuss handling of oversize messages. | |
| David Lang rightfully suggested to create a central capability inside | |
| rsyslog core to handle them. | |
| We need to make a distinction between input and output messages. Also, | |
| input messages frequently need to have some size restrictions done at | |
| a lower layer (e.g. protocol layer) for security reasons. Nevertheless, | |
| we should have a central capability | |
| * for cases where it need not be handled at a lower level | |
| * as a safeguard when a module invalidly emits it (imfile is an example, | |
| see https://github.com/rsyslog/rsyslog/pull/2632 for a try to fix it | |
| on the module level - we will replace that with the new capability | |
| described here). | |
| The central capability works on message submission, and so cannot be | |
| circumvented. It has these capabilities: | |
| * overisze message handling modes: | |
| - truncate message | |
| - split message | |
| this is of questionable use, but also often requested. In that mode, | |
| the oversize message content is split into multiple messages. Usually, | |
| this ends up with message segments where all but the first is lost | |
| anyhow as the regular filter rules do not match the other fragments. | |
| As it is requested, we still implemented it. | |
| - accept message as is, even if oversize | |
| This may be required for some cases. Most importantly, it makes | |
| quite some sense when writing messages to file, where oversize | |
| does not matter (accept from a DoS PoV). | |
| * report message to a special "oversize message log file" (not via the | |
| regular engine, as that would obviously cause another oversize message) | |
| This commit, as the title says, handles oversize INPUT messages. | |
| see also https://github.com/rsyslog/rsyslog/issues/2190 | |
| closes https://github.com/rsyslog/rsyslog/issues/2681 | |
| closes https://github.com/rsyslog/rsyslog/issues/498 | |
| Note: this commit adds global parameters: | |
| * "oversizemsg.errorfile", | |
| is used to specify the location of the oversize message log file. | |
| * "oversizemsg.report", | |
| is used to control if an error shall be reported when an oversize | |
| message is seen. The default it "on". | |
| * add global parameter "oversizemsg.input.mode" | |
| is used to specify the mode with which oversized messages will | |
| be handled. | |
| - omfwd: add support for bind-to-address for UDP | |
| To allow the same source address to be used regardless of the egress | |
| interface taken, an option is added for an address to bind the datagram | |
| socket to. Similarly to imudp, it is necessary to add an ipfreebind | |
| option which is set by default, so as to avoid an excess of errors at | |
| startup before the network interface has come up. This enhancement | |
| allows a usecase on networking devices, by which a source interface | |
| that is typically a loopback is specified, on which an address to bind | |
| to is configured. This is so that the same source address is used for | |
| all packets from rsyslog. | |
| Thanks to Mike Manning for the patch. | |
| - template systemd service file proposes higher permitted file handle limit | |
| Especially on busy systems the default are too low. Please keep in mind | |
| that on a very busy system even the now-proposed setting may be too low. | |
| Thanks to github user jvymazal for the patch. | |
| - imuxsock: replace select() call by poll() | |
| While extremely unlikely, imuxsock could abort if a file descriptor | |
| > 1024 was received during the startup phase (never occured in | |
| practice, but theoretically could if imfile monitored a large number | |
| of files and were loaded before imuxsock - and maybe other | |
| strange cases). | |
| see also https://github.com/rsyslog/rsyslog/issues/2615 | |
| - nsdsel_ptcp: replace select() by poll() | |
| This removes us of problems with fds > 1024. The performance will | |
| probably also increase in most cases. | |
| Note this is not a replacement for the epoll drivers, but a general | |
| stability improvement when epoll() is not available for some reason. | |
| see also https://github.com/rsyslog/rsyslog/issues/2615 | |
| closes https://github.com/rsyslog/rsyslog/issues/1728 | |
| closes https://github.com/rsyslog/rsyslog/issues/1459 | |
| - omprog: refactor tests, fix child closing issues | |
| Refactor omprog tests. Fix sync issues in these tests by | |
| using the feedback mode (confirmMessages=on) to synchronize | |
| the test with the external program. Closes #2403 (I hope) | |
| Fix omprog not properly closing child process when | |
| signalOnClose=on. Needed for the new tests. Closes #2599 | |
| Fix omprog not waiting for the child process to terminate | |
| when signalOnClose=off. Needed for the new tests. Closes #2600 | |
| Close all fds before executing the child even when valgrind | |
| is enabled (--enable-valgrind). Needed for the new tests. | |
| Fix memory leak when the xxxTransactionMark parameters were | |
| used. | |
| Thanks to Joan Sala for the patch. | |
| - core: config optimizer did not handle call_indirect | |
| This also caused the emission of an "internal error" error message | |
| closes https://github.com/rsyslog/rsyslog/issues/2665 | |
| - debug support: add capability to print testbench-specific timeout reports | |
| done by setting RSYSLOG_DEBUG_TIMEOUTS_TO_STDERR to "on" | |
| this is by default activated inside the testbench | |
| - mmgrok: fix potential segfault | |
| The modules used strtok(), which is not thread-safe. So it will potentially | |
| segfault when multiple instances are spawned (what e.g. happens on busy | |
| systems). | |
| This patch replaces strtok() with its thread-safe counterpart | |
| strtok_r(). | |
| see also https://github.com/rsyslog/rsyslog/issues/1359 | |
| - imrelp bugfix: maxDataSize could be set lower than maxMessageSize | |
| maxDataSize specifies the length which will still be accepted | |
| It previously could be set to any value, including values lower than the | |
| configured rsyslog max message size, which makes no sense. Now this is | |
| checked an error message is emitted if the size is set too low. | |
| - build system bugfix: build broken if liblogging-stdlog installed in custom path | |
| Thanks to Dirk Hörner for the patch. | |
| - core bugfix: segfault on queue shutdown | |
| if a ruleset queue is in direct mode, a segfault can occur during | |
| rsyslog shutdown. The root cause is that a direct queue does not | |
| have an associated worker thread pool, but the ruleset destructor | |
| does not anticipate that and tries to destruct the worker thread | |
| pool. It needs to do this itself, as otherwise we get a race | |
| between rulesets on shutdown. | |
| This was a regression from | |
| https://github.com/rsyslog/rsyslog/commit/3fbd901b3e6300010 | |
| closes https://github.com/rsyslog/rsyslog/issues/2480 | |
| - imfile bugfix: statefiles contain invalid JSON | |
| When imfile rewrites state files, it does not truncate previous | |
| content. If the new content is smaller than the existing one, the | |
| existing part will not be overwritten, resulting in invalid json. | |
| That in turn can lead to some other failures. | |
| closes https://github.com/rsyslog/rsyslog/issues/2662 | |
| - omfile bugfix: segfault if empty filename was given | |
| closes https://github.com/rsyslog/rsyslog/issues/2417 | |
| - fix build issues when atomic operations are not present | |
| for details, see https://github.com/rsyslog/rsyslog/pull/2604 | |
| - lmsig_ksils12 bugfix: build and static analyzer issues | |
| The module had a couple of problems building as well as some potential | |
| errors detected by clang static analyzer. These have been fixed. | |
| Thanks to Allan Park for the patch. | |
| closes https://github.com/rsyslog/rsyslog/issues/2517 | |
| - impstats bugfix: segfault if bound to non-existing ruleset | |
| segfault happens during shutdown; up until unload of impstats, | |
| rsyslog works correctly, except that no pstats are emitted. This | |
| can be considered to be expected, because the error message | |
| indicates the default ruleset is used instead. This is what | |
| now actually happens. | |
| closes https://github.com/rsyslog/rsyslog/issues/2645 | |
| - mmjsonparse bugfix: invalid container name was not detected | |
| in debug builds, this will trigger an assertion. In production | |
| builds, an rsyslog internal error is logged, but rsyslog | |
| continues to run. | |
| closes https://github.com/rsyslog/rsyslog/issues/2584 | |
| - mmkubernetes bugfixes: fix lnrules, add defaults, add test | |
| - Fix lnrules for CONTAINER_NAME | |
| - Add pkg check for lognorm >= 2.0.3 so we can set the macro | |
| to enable ln_loadSamplesFromString | |
| - Add some reasonable default values for parameters, such as | |
| kubernetesurl https://kubernetes.default.svc.cluster.local:443 | |
| - Clean up sample.conf configuration file | |
| Thanks to Richard Megginson for the patch set. | |
| - build system bugfix: --enable-atomic-operations did not work | |
| closes https://github.com/rsyslog/rsyslog/pull/2604 | |
| - bugfix: rsyslog aborts on startup when specific config errors are made | |
| The following errors must be made in rsyslog.conf: | |
| * no action present | |
| * a call statement is used on an undefined ruleset | |
| In this case, rsyslog emits an error message on the missing actions and | |
| then segfaults. Depending on memory layout, it may also continue to run | |
| but do nothing except accepting messages as no action is configured. | |
| This patch make rsyslog properly terminate after the error message. It | |
| is a change in behavior, but there really is no reason why a defunct | |
| instance should be kept running. | |
| closes https://github.com/rsyslog/rsyslog/issues/2399 | |
| - build system: remove no longer needed --enable-libcurl configure switch | |
| The --enable-libcurl switch was added to be able to disable libcurl | |
| functionality inside the rsyslog core, see 46f4f43. As libcurl is no | |
| longer used inside the core (due to introducing function modules), | |
| --enable-libcurl needs to be removed. | |
| closes https://github.com/rsyslog/rsyslog/issues/2628 | |
| - QA/CI | |
| * fixed races in some tests; root cause was that default enq timeout was too | |
| low - we may also see in the future that other tests also need adjustment | |
| (note that this is not a code problem but rather slow CI environments, | |
| so increasing the timeout to get to a stable test state is the absoulutely | |
| correct thing to do) | |
| * enabled some additional useful compiler warnings | |
| * new test for diskqueue hitting configured disk space limit | |
| * new tests for omfile | |
| * added tests for mmkubernetes | |
| * added tests for some script functions that were missing them | |
| * made far more test compatible with FreeBSD, so that we now have fuller | |
| coverage there | |
| ------------------------------------------------------------------------------ | |
| Version 8.34.0 [v8-stable] 2018-04-03 | |
| - mmkubernetes: new module | |
| Thanks to Richard Megginson and Peter Portante for contributing the module. | |
| - rsyslog script: introduce loadable function modules | |
| rsyslog scripting can now also be extended via loadable modules - they | |
| provides functions (just like loadable input, output, ... modules) | |
| see also http://jan.gerhards.net/2018/03/loadable-rainerscript-functions.html | |
| - imfile: large refactoring of complete module | |
| This commit greatly refactors imfile internal workings. It changes the | |
| handling of inotify, FEN, and polling modes. Mostly unchanged is the | |
| processing of the way a file is read and state files are kept. | |
| This is about a 50% rewrite of the module. | |
| Polling, inotify, and FEN modes now use greatly unified code. Some | |
| differences still exists and may be changed with further commits. The | |
| internal handling of wildcards and file detection has been completely | |
| re-written from scratch. For example, previously when multi-level | |
| wildcards were used these were not reliably detected. The code also | |
| now provides much of the same functionality in all modes, most importantly | |
| wildcards are now also supported in polling mode. | |
| The refactoring sets ground for further enhancements and smaller | |
| refactorings. This commit provides the same feature set that imfile | |
| had previously and all existing CI tests pass, as do some newly | |
| created tests. | |
| Some specific changes: | |
| - bugfix: module parameter "sortfiles" ignored | |
| This parameter only works in Solaris FEN mode, but is otherwise | |
| ignored. Most importantly it is ignored under Linux. | |
| fixes https://github.com/rsyslog/rsyslog/issues/2528 | |
| - bugfix: imfile did not pick up all files when not present | |
| at startup | |
| fixes https://github.com/rsyslog/rsyslog/issues/2241 | |
| fixes https://github.com/rsyslog/rsyslog/issues/2230 | |
| fixes https://github.com/rsyslog/rsyslog/issues/2354 | |
| fixes https://github.com/rsyslog/rsyslog/issues/1716 | |
| - bugfix: directories only support "*" wildcard, no others | |
| fixes https://github.com/rsyslog/rsyslog/issues/2303 | |
| - bugfix: parameter "sortfiles" did only work in FEN mode | |
| fixes https://github.com/rsyslog/rsyslog/issues/2528 | |
| - provides the ability to dynamically add and remove files via | |
| multi-level wildcards | |
| see also https://github.com/rsyslog/rsyslog/issues/1280 | |
| - the state file name currently has been changed to inode number | |
| This will further be worked on in upcoming PRs | |
| see also https://github.com/rsyslog/rsyslog/issues/2231 | |
| - some enhancements were also done to CI tests, most importantly | |
| they were made more compatibile with BSD | |
| Note that most of the mentioned bug fixes cannot be applied to older | |
| versions, as they fix design issues which are solved by the refactoring. | |
| Thus there are not separate commits for them. | |
| There are probably also a number of different issues fixed, which have | |
| not yet been full confirmed. Especially anyone having troubles with imfile | |
| and wildcards will benefit from the refactoring. | |
| closes https://github.com/rsyslog/rsyslog/issues/2359 | |
| - omelasticsearch: add support for CA cert, client cert auth | |
| This allows omelasticsearch to perform client cert based authentication | |
| to Elasticsearch. | |
| adds parameters: tls.cacert, tls.mycert, tls.myprivkey | |
| Thanks to Richard Megginson for the patch. | |
| - omfile-hardening: new contributed module for "omfile hardened operations" | |
| This extends omfile with features considered useful for hardening. Comes | |
| at the expense of performance loss and changed semantics. | |
| Thanks to Mikko Kortalainen for contributing this work. | |
| - stream/bugfix: memory leak on stream open if filename as already generated | |
| this can happen if imfile reads a state file. On each open, memory for the | |
| file name can be lost. | |
| We detected this while working on imfile refactoring, so there is no related | |
| bug report. No specific test has been crafted, as the refactored imfile | |
| tests catch it (as soon as they are merged). | |
| - bugfix/omkafka: did not build on platforms without atomic operations | |
| Thanks to github user bruce87en for the patch | |
| - bugfix/core/ratelimiting: SystemLogRateLimitBurst was limited to 65535 | |
| rsyslog uses unsigned short for configuration setting SystemLogRateLimitBurst. | |
| Being just 16 bits, unsigned short cannot hold values bigger than 65535. in a | |
| practical setting rsyslog misbehaved with SystemLogRateLimitBurst being bigger | |
| than 65535. | |
| Thanks to github user KaleviKolttonen for the patch. | |
| - bugfix imfile: memory leak in readMode 0 | |
| closes https://github.com/rsyslog/rsyslog/issues/2421 | |
| - bugfix omfile: some error messages had parameters in wrong order | |
| which made the message look strange, but still readable | |
| Thanks to Hans Rakers for the patch. | |
| - bugfix omprog: file handle leak | |
| There was a fd leak in the feedback feature added in v8.31.0 (github PR #1753). | |
| The leak occurred when omprog was used with the confirmMessages=on setting | |
| and no output setting. One fd was leaked every time the external program was | |
| started. | |
| Thanks to Joan Sala for the patch. | |
| - bugfix imuxsock: data alignment problems | |
| gcc did rightly complain that the cred and ts pointers would cause | |
| alignment problems, so they were converted to structs and the necessary | |
| data was memcpy()'ed to them. | |
| the aux[] buffer was also potentially misaligned, so making a union | |
| out of it and struct cmsghdr insured it was properly aligned. | |
| The problems were especially visible on alpha and ia64 machines. | |
| Thanks to Jason Duerstock for the fix. | |
| - bugfix testbench: some test were accidently not executed | |
| Thanks to Kasumi Hanazuki for the patch. | |
| - bugfix tcp subsystem: keepalive settings mixed up | |
| TCPKeepAliveIntvl and TPCKeepAliveTime were switched. This is now correct and | |
| thus causes a CHANGE OF BeHAVIOR of these settings. We applied this change only | |
| after careful consideration of the effect. The contributor Alexandre Pierret | |
| explained the situation as such: | |
| "From my side, I work with thousands of servers centralizing logs to rsyslog | |
| servers in tcp. All of them are running RHEL 6 and 7. The default rsyslog settings | |
| in RHEL is TCPKeepAlive off. Since there are thousands of connections on the | |
| rsyslog servers, I had to turn the TCPKeepAlive on to setup an agressive policy | |
| regarding ghost connections (following firewall tcp-timeout issue). Basically, | |
| I set up: intvl=10 - probe=5 - time=2 If intvl and time are switched, it won't | |
| break anything. It will just send 5 more empty tcp packets for 10 seconds (5 | |
| probe x 2 seconds), which is painless (any professional grade 100€ router can | |
| send more than 1.000.000 PPS). For 3000 servers, it means 3000 pps for 5 | |
| seconds (3000 servers x 2 back-and-forth / 2 seconds). Let's take another worst | |
| example: intvl=3600 - probe=5 - time=1. It means one keepalive every hour and | |
| a 5 retry every 1s after a network issue. If the time and intvl values are | |
| switched, it will generate 1 keepalive every seconds, It's a LOT more, but | |
| after 5 probe or packet, it will stop. To summarize, I think it won't | |
| break anything: | |
| * Keepalive is off by default is many linux distribution | |
| * When we enable it, it's to set up an agressive policy. And setting up a | |
| 10-5-1, 60-5-2, 2-5-60 or 1-20-1 policy is almost the same. | |
| Bonus: For people setting up their rsyslog from the documentation, it will | |
| now work as expected." | |
| This was convincing, and we actually think that the fast majority of users set | |
| up keepalive based on the doc and did never verify it actually worked as | |
| excpected. So we think that in all those cases, rsyslog will finally work as | |
| intended. So we consider it justified to "change the behavior" here. | |
| full discussion in PR: https://github.com/rsyslog/rsyslog/pull/2367 | |
| Thanks to Alexandre Pierret for analyzing the situation and providing the | |
| patch. | |
| - fix some cosmetic issues found by lgtm.com static code analyzer | |
| e.g. header file guard not correctly set - if you really are interested in | |
| details, check git log | |
| - CI | |
| * add build test without atomic operations - now catches missing mutex macros | |
| * add lgtm.com static analyzer (automatically called via GitHub PR) | |
| * improved stability of global-umask.sh test, which unnecessarily used | |
| wildcards for test output file checking. | |
| Thanks to Kasumi Hanazuki for the patch. | |
| * added some test for omprog with transactional interface | |
| Thanks to Joan Sala for the new tests. | |
| * fixed some omjournal tests which did not properly check result | |
| ------------------------------------------------------------------------------ | |
| Version 8.33.1 [v8-stable] 2018-03-06 | |
| - 8.33.0 tarball release was actually pre-8.33.0 | |
| ... and so did not contain all features. This alone made a re-release | |
| necessary, which is what now happens with 8.33.1. | |
| Note: the git 8.33.0 label was correctly applied, "just" the tarball | |
| was wrong. | |
| - devcontainer: use some more sensible defaults | |
| and add ability to specify generic docker run options | |
| this makes integration into CI (and other scripting) easier | |
| - fix problems with make dist | |
| make dist did not package everything that was needed for CI, thus | |
| resulting in make check failures if build from tarball. | |
| Thanks to Thomas D. (whissi), and Michael Biebl for alerting us on the | |
| problem, providing advise and some of the patches. We also added addt'l | |
| patches ourselves. The problem occured as the CI check for tarball | |
| completeness was more or less disabled a couple of weeks ago, which | |
| unfortunately went unnoticed. We have also applied some more safeguards | |
| to detect such problems in the future. | |
| ------------------------------------------------------------------------------ | |
| Version 8.33.0 [v8-stable] 2018-02-20 | |
| - auto-detect if running inside a container (as pid 1) | |
| adjust defaults in this case to be more container-friendly | |
| - config: add include() script object | |
| This permits to include files (like legacy $IncludeConfig) via a | |
| script object. Needless to say, the script object offers more | |
| features: | |
| - include files can now be | |
| - required, with rsyslog aborting when not present | |
| - required, with rsyslog emitting an error message but otherwise | |
| continuing when not present | |
| - optional, which means non-present include files will be | |
| skipped without notice | |
| This is controlled by the "mode" parameter. | |
| - text can be included form e.g. an environment variable | |
| --> ex: include(text=`echo $ENVVAR`) | |
| This finally really obsoletes $IncludeConfig. | |
| closes https://github.com/rsyslog/rsyslog/issues/2151 | |
| - template: add option to generate json "container" | |
| This enables easy JSON generation via template. | |
| This commit also corrects an issue with the constant "jsonf" | |
| format. That was recently added, and the implementation problem | |
| only became visible when used inside a larger json object. No | |
| officially released code is affected, thought - so it rellay | |
| is just a side-note. | |
| closes https://github.com/rsyslog/rsyslog/issues/2347 | |
| - core/template: add format jsonf to constant template entries | |
| closes https://github.com/rsyslog/rsyslog/issues/2348 | |
| - config: add ability to disable config parameter ("config.enabled") | |
| For auto-generated configs, it is useful to have the ability to disable some | |
| config constructs even though they may be specified inside the config. This | |
| can now be done via the ```config.enabled``` parameter, applicable to all | |
| script objects. If set to ```on``` or not specified, the construct will be | |
| used, if set to any other value, it will be ignored. This can be used | |
| together with the backtick functionality to configure enable and disable | |
| from either a file or environment variable. | |
| closes https://github.com/rsyslog/rsyslog/issues/2431 | |
| - script: permit to use environment variables during configuration | |
| new constant string type "backticks", inspired by sh | |
| (sample: `echo $VARNAME`). | |
| - new global config parameter "shutdown.enable.ctlc" | |
| permits to shutdown rsyslog via ctl-c; useful e.g. in containers | |
| - config optimizer: detect totally empty "if" statements and optimize | |
| them out | |
| - template: constant entry can now also be formatted as json field | |
| This enhancements permits to craft clean templates that generate JSON, | |
| e.g. for ElasticSearch consumption (or any other REST API) | |
| - omstdout: support for new-style configuration parameters added | |
| - core: set TZ on startup if not already set | |
| In theory, TZ should be set by the OS. Unfortuantely, this seems | |
| to be not the case any longer on many Linux distros. We now check | |
| it and set it appropriate if not already given. | |
| Thanks to github user JPvRiel for providing an excellent explanation | |
| of the reasoning for this and how to work around it. | |
| closes https://github.com/rsyslog/rsyslog/issues/2054 | |
| - imjournal bugfix: file handle leak during journal rotation | |
| Thanks to Peter Portante for the patch | |
| see also: https://github.com/rsyslog/rsyslog/pull/2437 | |
| - lmsig_ksils12 bugfix: dirOwner and dirGroup config was not respected | |
| - script bugfix: replace() function worked incorrectly in some cases | |
| If the end of the message was similar to the replacement string, parts | |
| of the string could (not always) be missing. | |
| Thanks to Yaroslav Bo for the patch. | |
| - build system bugfix: --disable-libcurl did not work | |
| Thanks to Dan Molik, Thomas D. (whissi), and Michael Biebl for the patches. | |
| - fixed build issues on Alpine Linux | |
| - core bugfix: misadressing in external command parser | |
| This parser is used whenever a module (e.g. omprog) needs to process | |
| command lines. If command parameters were given, memory misadressing | |
| occured. This could lead to a segfault. | |
| This is a regression in 8.32.0. | |
| closes https://github.com/rsyslog/rsyslog/issues/2408 | |
| - core bugfix: small memory leak in external command parser | |
| This parser is used whenever a module (e.g. omprog) needs to process | |
| command lines. On each action definitions for actions that use the | |
| parser a small amount of memory was leakd. It is an uncritical leak | |
| as it only occurs during config parsing. So it leaks a couple of | |
| KiB during startup but does not grow during actual message processing. | |
| This is a regression in 8.32.0. | |
| - core bugfix: string not properly terminated when RFC5424 MSGID is used | |
| This could lead to misadressing when the jsonmesg property was used. | |
| closes https://github.com/rsyslog/rsyslog/issues/2396 | |
| - bugfix: strndup() compatibility layer func copies too much | |
| The function did not obey the upper limit, effectively becoming | |
| a strdup(). This was only noticed when the compatibility layer | |
| was required, most importantly on Solaris 10. | |
| - CI system | |
| - we now use well-defined containers for parts of the CI runs | |
| - now also build test under Alpine Linux | |
| - test added for omprog feedback feature | |
| ------------------------------------------------------------------------------ | |
| Version 8.32.0 [v8-stable] 2018-01-09 | |
| - NEW BUILD REQUIREMENTs: | |
| * libfastjson 0.99.8 is now required; older versions lead to bugs in rsyslog | |
| * libczmq >= 3.0.2 is now required for omczmq | |
| This was actually required for quite some while, but not properly checked | |
| during configure run. If the lib was older, build failed. Now configure | |
| detects this and emits the appropiate error message. | |
| * libcurl is now needed for rsyslog core | |
| due to new script function http_request(). This can be turned off by the | |
| new configure option --disable-libcurl. If so, http_request() is not | |
| available. | |
| - rsyslogd: add capability to specify that no pid file shall be written | |
| Command line option -iNONE provides this capability. This utilizes the | |
| pre-existing -i option, but uses the special name "NONE" to turn of the | |
| pid file check feature. Turning off is useful for systems where this no | |
| longer is needed (e.g. systemd based). | |
| closes https://github.com/rsyslog/rsyslog/issues/2143 | |
| - ompgsql: considerable enhancements | |
| The PostgreSQL output module was woefully out-of-date the following | |
| list is changes made to update the module to current Rsyslog standards. | |
| * allow for v6 configuration syntax | |
| * configurable ports | |
| * support transactional interface | |
| * push db connection into workers (libpq is threadsafe) | |
| * enable module testing on travis | |
| * ensure configuration syntax backwards compatibility | |
| * formatting around postgres core templating | |
| * use new test conventions | |
| * add new configuration syntax test | |
| * add valgrind tests for new and old syntax | |
| * add threading tests | |
| * add action queue long running tests | |
| * add action queue valgrind test | |
| Thanks to Dan Molik for contributing this great enhancement! | |
| - build system: removed --enable-rtinst configure option | |
| This was a historic debugging option which has been superseeded by | |
| newer tools like valgrind, helgrind, ASAN, TSAN, etc... | |
| - pmrfc3164: support for headerless messages | |
| pmrfc3164 now detects if the first non-SP, non-HT character is either | |
| '{' or '[' and if so assume that no header (TAG, HOSTNAME, DATE) is | |
| given. If so, it uses defaults for these values. The full message is | |
| used as the MSG part in that case. Note that an initial PRI may still | |
| be specified. | |
| This follows the trend to send JSON messages via syslog without any | |
| header. We use default header values in this case. | |
| This right now is a bit experimental; we may roll it back if | |
| problems show up in practice. | |
| closes https://github.com/rsyslog/rsyslog/issues/2030 | |
| - omhiredis: add option to use RPUSH instead of LPUSH | |
| see also https://github.com/rsyslog/rsyslog/issues/1625 | |
| - mmexternal improvements | |
| * better error reporting if to-be-executed script cannot be executed | |
| * some general cleanup | |
| * better redirection of stdin/out/err for the executed script | |
| * bugfix: argv[0] of the script was missing if command line parameters | |
| were not specified | |
| - omprog: refactored, code shared with mmexternal moved to common object | |
| - logctl tool: refactor to support newer rsyslog standards | |
| * Made the logctl usertool ISO C90 compliant. | |
| * Made the logctl usertool use a homogeneous coding style. | |
| Thanks to Hugo Soszynski for contributing this work (as well as | |
| suggesting some workarounds related to libmongoc/libbson). | |
| - imfile: added support for Solaris File Event notification (FEN) | |
| also improves performance under Solaris, with FEN imfile provides | |
| features equivalent to inotify mode | |
| - core/action: new parameter "action.errorfile" | |
| permits to write failed messages to an "error file" so that they | |
| can be reviewed and potentially be reprocessed | |
| - imfile: added new module parameter "sortFiles" | |
| This permits to process newly created files in sorted order. | |
| - imuxsock: improved status reporting: socket name received from systemd | |
| Providing an indication of what we got from systemd facilitates problem | |
| analysis. | |
| - build system: added new testbench configure switches | |
| now --enable-testbench1 and --enable-testbench2 exists which permit | |
| to enable/disable parts of the testbench. By default, both are on | |
| when --enable-testbench is given. For full testbench coverage, both | |
| options must be given. These options are necessary because under | |
| Travis we hit the max runtime for tests and needed to split tests | |
| over multiple incarnations. | |
| - mmpstrucdata: new parameter "sd_name.lowercase" | |
| to permit preserving case for structured data identifiers | |
| Special thanks to github user alanrobson for the intial commit that | |
| preseves case (on which we based other work). | |
| - omfile: add module-global option "dynafile.donotsuspend" | |
| this permits to enable SUSPENDing dynafile actions. Traditionally, | |
| SUSPEND mode was never entered for dynafiles as it would have blocked | |
| overall processing flow. Default is not to suspend (and thus block). | |
| closes https://github.com/rsyslog/rsyslog/issues/2236 | |
| - testbench: add a capability to turn off libfaketime tests via configure | |
| Unfortunately, libfaketime becomes more and more problematic in newer | |
| versions and causes aborts on some platforms. This provides the ability | |
| to turn it off via --disable-libfaketime. | |
| In the longer term, we should consider writing our own replacement. | |
| - testbench: name valgrind tests consistently | |
| all valgrind tests now end in -vg.sh | |
| - RainerScript: add function parse_json() | |
| - RainerScript: add function substring() | |
| - RainerScript: add function http_request() | |
| - RainerScript: add function previous_is_suspended() | |
| This function returns a boolean indicating if the previous action is | |
| suspended (0 - no, 1 - yes). This is useful if arbitrary processing | |
| (other than calling actions) should be made depending on that state. | |
| A prime example for this is probably calling a ruleset. | |
| closes https://github.com/rsyslog/rsyslog/issues/1939 | |
| - Patches from BSD projects have been imported | |
| ... as far as they still apply. Some patches that patched BSD-specific | |
| code were broadened to be more generic. | |
| - script bugfix: invalid function names were silently ignored | |
| no error message was produced | |
| thanks to Matt Ezell for providing the patch. | |
| - rainerscript: add int2hex() function | |
| - rainerscript: add is_time() function | |
| Thanks to Stephen Workman for implementing this. | |
| - RainerScript: add function script_error() and error-reporting support | |
| This permits script functions that could fail to report errors back, so | |
| that the user can handle them if desired. We use an errno-style of | |
| interface. That means script_error() needs to be called after functions | |
| that supports it. It will return either 0 (all OK) or something else | |
| (an error condition). | |
| The commit also modifies the parse_time() function to use the new | |
| interface. First of all, this is useful for users, and secondly we | |
| needed a capability to craft a testbench test. | |
| closes https://github.com/rsyslog/rsyslog/issues/1978 | |
| - testbench: fixed build problem of testbench tools under Alpine Linux | |
| - added --enable-libsystemd configure option to enforce use of libsystemd | |
| so we can fail the build on platforms where this is required | |
| - core/glbl: remove long-unused option $optimizeforuniprocessor | |
| This was still available, but had no effect (for ~10 years or so). The | |
| plumbing is now removed. If someone tries to use the option, an | |
| error message is generated. | |
| closes https://github.com/rsyslog/rsyslog/issues/2280 | |
| - core/queue: emit better status messages at rsyslog shutdown | |
| this helps to diagnose issue - unfortunately we need more work to ensure | |
| that the messages always make it to the user. This is a start and | |
| hopefully useful at least for the testbench, possibly more. | |
| - fixed a couple of build issues with gcc-7 (in less frequently used modules) | |
| - fixed a couple of build issues on the arm platform (actually raspbian) | |
| - impstats: fix invalid counter definitions for getrusage() reporting | |
| some of the counters were defined as int (32 bit) vs. intctr_t (64 bit). | |
| On some platforms "long" seems to be 64bit, and getrusage() provides | |
| what we store as int via long. So this caused truncation and/or overflow. | |
| This had undefined effects. Most often, everything worked fine | |
| for values smaller than 2^31 but sometimes we got negative values. | |
| closes https://github.com/rsyslog/rsyslog/issues/1517 | |
| - imudp bugfix: potential segfault in ratelimiting | |
| The rate-limiter inside imudp was not set to be thread safe, but was | |
| used across multiple threads. This worked in default configuration, | |
| but failed when RepeatedMsgReduction was set to "on". | |
| Note that it in general is a bug to use a rate-limiter in | |
| non-threadsafe mode across multiple threads. This also causes invalid | |
| rate limiting counts in the default case. | |
| closes https://github.com/rsyslog/rsyslog/issues/441 | |
| fixes https://github.com/rsyslog/rsyslog/issues/2132 | |
| - imptcp bugfix: access to free'ed memory | |
| When notifyconnectionclose was on, a string buffer was accessed immediately | |
| after it was freed (as part of the connection close operation). | |
| Detected by LLVM ASAN. | |
| - mmanon bugfix: fix wrong ipv6 embedded recognition | |
| mmanon recognized IPv6 with embedded IPv4 that have too few (16 bit) fields. | |
| example: 13:abd:45:0.0.0.0 | |
| closes https://github.com/rsyslog/rsyslog/issues/2357 | |
| - imfile bugfix: not detecting files in directory when wildcards are used. | |
| When directories and files are created at the same time, | |
| imfile may missed subdirs or file if the machine is on high load. | |
| The handling has been enhanced to scan newly created directories ALWAYS for | |
| matching files. | |
| fixes https://github.com/rsyslog/rsyslog/issues/2271 | |
| However there still exist problems with multilevel directory configurations. | |
| Details are discussed in https://github.com/rsyslog/rsyslog/issues/2354 | |
| Fixes for the remaining issues are expected for 8.33.0. | |
| - script bugfix: improper string-to-number conversion for negative numbers | |
| - core/action bugfix: 100% CPU utilization on suspension of output module | |
| Triggering condition: | |
| * output module using the legacy transaction interface | |
| (e.g. omelasticsearch, omlibdbi) | |
| * output module needs to suspend itself | |
| In these cases, rsyslog enters a busy loop trying to resolve the | |
| suspend condition. The bug is rooted in rsyslog core action code. | |
| This patch fixes it by inserting a 1-second sleep during calls | |
| to the resume handler. | |
| Note: we cannot sleep exactly as long as tryResume needs. This | |
| would require larger refactoring, which probably is not worth for | |
| the legacy interface. The current solution is almost as good, as | |
| the one second sleep has very little overhead on a real system. | |
| Thus we have choosen that approach. | |
| This patch now also ensures that failed messages are properly | |
| handled and do not cause eternal hang. | |
| closes https://github.com/rsyslog/rsyslog/issues/2113 | |
| - core/variables bugfix: bare $! cannot be used in set statement | |
| fixes https://github.com/rsyslog/rsyslog/issues/326 | |
| - core bugfix: auto commit of actions improperly handled | |
| The internal state variable bHadAutoCommit was handled in thread-unsafe way | |
| and most probably caused (hard to see) issues when action instances were | |
| run on multiple worker threads. It looks like the state variable was | |
| forgotten to move over to worker state when action workers were introduced. | |
| closes https://github.com/rsyslog/rsyslog/issues/2046 | |
| - core bugfix: filename length limitation of 199 bytes | |
| file names (including path names) longer than 199 bytes could not be | |
| handled at many places. This has now been uplifted to 4KiB after careful | |
| review for the largest size supported among all relevant platforms. | |
| - core bugfix: undefined behavior due to integer overflow | |
| when searching strings, we may have an (unsigned) interger overflow | |
| which can lead to misadressing. | |
| Detected by clang ubsan. | |
| - core bugfix: race on LocalHostIP property during startup | |
| The way the default was set could lead to a race if e.g. two internal | |
| messages were emitted at startup when the property was not yet set. This | |
| has been seen to happen in practice. It had no bad effect except a very | |
| small stationary memory leak, but made using thread analyzers unreliable | |
| (as it was rightfully detected as a problem). | |
| closes https://github.com/rsyslog/rsyslog/issues/2012 | |
| - bugfix: potential segfault on startup | |
| timezone info table was "sorted" even though it may be NULL. There is | |
| no practical case known where this lead to an actual abort, but in | |
| theory it would be possible. If so, it would happen immediately on | |
| startup. | |
| Detected by clang ubsan. | |
| - omhiredis bugfix: rsyslog segfault on startup if no template is specified | |
| - omprog bugfix: argv[0] not set when using binary without arguments | |
| When using the omprog plugin with a binary without arguments, argv[0] (the | |
| name of the binary) is not set, causing binaries that depend on this value | |
| being set to crash or misbehave. | |
| This patch also mildly refactors omprog argv[] creations, albeit some more | |
| refactoring would be useful. | |
| closes https://github.com/rsyslog/rsyslog/issues/1858 | |
| - core: refactoring of rsyslog's cstr "class" | |
| Function cstrGetSzStrNoNULL shall modified the string buffer on each call, | |
| albeit it is considered a "read-only" function. It currently adds a '\0' | |
| at the end. This is bad, especially when multiple threads access the same | |
| string. As always the same data is written, it was not causing real issues | |
| (except unnecessary cache writes), but it polluted the thread debugger and | |
| as such prevent more elaborate automatted tests. | |
| - parent directory creation function refactored | |
| This should not cause any change of behavior, but is worth noting in case | |
| we see a regression not caught by the CI system. | |
| - mmsnmptrapd bugfix: potential misadressing | |
| This only occured in situations when the system was totally out of memory. | |
| - imkafka: fix potential small ressource leak | |
| If rdkafka handle cannot fully populated, cleanup is added. Previously, we | |
| could leak a handle if e.g. no brokers were avaiable. Note that this was | |
| a cosmetic leak, as proper processing is not possible in any case and the | |
| leak is once per listener, so not growing. But we now also ensure that | |
| proper error reporting and handling is done in any case. Previously, we | |
| may have some misleading error messages if the defunct kafka handle was | |
| used. | |
| closes https://github.com/rsyslog/rsyslog/issues/2084 | |
| - imkafka bugfix: do not emit error message on regular state | |
| This was misunderstood as an error state and could spam the system | |
| log considerably. Regression from 8.31.0. | |
| - omkafka: expose operational status to user where useful | |
| omkafka emits many useful operational status messages only to the debug | |
| log. After careful review, we have exposed many of these as user error | |
| and warning message (ex: librdkafka queue full, so user knows why we | |
| suspend the plugin temporarily). This may have made the module too | |
| chatty. If so, one can always filter out messages via configuration. And | |
| if we really went overboard, we can undo those parts with the next | |
| release. So it's better to give a bit more than less, as this definitely | |
| eases troubleshooting for users. | |
| closes https://github.com/rsyslog/rsyslog/pull/2334 | |
| - omkafka bugfix: potential message duplication | |
| If a message that already failed in librdkafka was resubmitted and that | |
| resubmit also failed, it got duplicated. | |
| - omkafka: fix multithreading | |
| omkafka has several issue if multiple worker instances are used. This commit | |
| actually make the module use a single worker thread at max. Reasoning: | |
| Librdkafka creates background threads itself. So omkafka basically needs to move | |
| memory buffers over to librdkafka, which then does the heavy hauling. As such, we | |
| think that it is best to run max one wrkr instance of omkafka -- otherwise we just | |
| get additional locking (contention) overhead without any real gain. As such, | |
| we use a global mutex for doAction which ensures only one worker can be active | |
| at any given time. That mutex is also used to guard utility functions (like | |
| tryResume) which may also be accessed by multiple workers in parallel. | |
| Note: shall this method be changed, the kafka connection/suspension handling needs | |
| to be refactored. The current code assumes that all workers share state information | |
| including librdkafka handles. | |
| closes https://github.com/rsyslog/rsyslog/issues/2313 | |
| - omkafka bugfix: potential misadressing | |
| The failed message list is improperly cleaned. This is a regression | |
| from recent commit 4eae19e089b5a83da679fe29398c6b2c10003793, which | |
| was introduced in 8.31.0. | |
| This problem is more likely to happen under heavy load or bad | |
| connectivity, when the local librdkafka queue overruns or message | |
| delivery times out. | |
| closes https://github.com/rsyslog/rsyslog/issues/2184 | |
| closes https://github.com/rsyslog/rsyslog/issues/2067 | |
| - omkafka bugfix: build fails with older versions of librdkafka | |
| closes https://github.com/rsyslog/rsyslog/issues/2168 | |
| - omgssapi bugfix: fix compiler warnings with gcc-7 | |
| closes https://github.com/rsyslog/rsyslog/issues/2097 | |
| - dnscache bugfix: entries were cached based on IP AND port number | |
| That hash key which is used to find out already cached dns entry gets | |
| incorrectly computed from the whole sockaddr_storage | |
| (sockaddr_in/sockaddr_in6) structure including a sin_port (which doesn't | |
| have a static value) instead of only an address, thus creating redundant | |
| dns cache entries/consuming more space. This lead to potentially high memory | |
| usage and ineffectiveness of the case. It could be considered a memory leak. | |
| Thanks to Martin Osvald for the patch. | |
| see also: https://github.com/rsyslog/rsyslog/pull/2160 | |
| - omkafka bugfix: fixed memory leak | |
| a memory leak occurred when librdkafka communicated error/status information | |
| to omkafka. this seems to happen relatively frequently, so this leak | |
| could be pretty problematic. | |
| - mmdblookup bugfix: replace thread-unsafe strtok() by thread-safe counterpart | |
| Many thanks to Will Storey (github user @horgh) for mentioning this and | |
| his phantastic help in debugging this rsyslog problem! | |
| - pmnormalize bugfix: remove unsave "strcat" implementation | |
| - rainerscript bugfix: ltrim() and rtrim function misadressing | |
| This could lead to a segfault and was triggerred by certain input data | |
| schemes. For example, a ltrim(" a") could lead to the fault. | |
| - imklog bugfix: local host IP was hardcoded to 127.0.0.1 | |
| This is now taken from the global localHostIP setting, which is used | |
| consistent accross all modules. | |
| Also, the removed (2012?) directive $klogLocalIPIF has been added | |
| again but directly marked as removed. That way, an informative error | |
| message is generated if someone tries to use it. | |
| closes https://github.com/rsyslog/rsyslog/issues/2276 | |
| - cleanup: remove obsolete pre-KSI GuardTime signature interface | |
| this is no longer functional starting Jan 2018 as it reached end of life | |
| closes https://github.com/rsyslog/rsyslog/issues/2128 | |
| - cleanup: obsolete defintion SOL_TCP replaced by newer IPPROTO_TCP | |
| this should not have any effect at all except better portability, but is | |
| worth mentioning in the ChangeLog nevertheless. | |
| - lookup tables: fixed undefined behavior detected by UBSan | |
| - CI testing | |
| - ARM (via Raspberry PI) added to CI system | |
| - Debian 9 added to CI system | |
| - omgssapi and usertools components are now also tested in Travis | |
| - test coverage on BSD has been enhanced | |
| ------------------------------------------------------------------------------ | |
| Version 8.31.0 [v8-stable] 2017-11-28 | |
| - NEW BUILD DEPENDENCY: ommongodb now requires libmongo-c | |
| instead of deprecated libmongo-client. | |
| - remove systemd embedded code, use libsystemd instead | |
| Since the early days rsyslog used the orginal systemd embedded | |
| interface code. This version now uses libsystemd where available. | |
| If we do not find libsystemd, we assume the system does not use | |
| systemd, which is a safe assumption nowadays. This ensures we use the | |
| fresh interface code and also removes bloat from our project. | |
| closes https://github.com/rsyslog/rsyslog/issues/1933 | |
| - mmanon: add support for IPv6 adresses with embedded IPv4 address | |
| While this format is uncommon, it may still be present in logs and as | |
| such should be supported. It is configurable via individual settings, | |
| though. Especially the number of bits to anonymize may be desired to | |
| be different than in pure IPv6. | |
| - ommongodb: big refactoring, more or less a feature-enhanced rewrite | |
| New features are : | |
| * Handle TLS connexion to mongodb | |
| * Handle MongoDB Replicasets | |
| * Added the 'ssl_ca' and 'ssl_cert' directives to configure tls connexion | |
| * Added 'uristr' directive to configure the connection uri in the form | |
| of 'mongodb://...' | |
| Now uses the official mongo-c-driver library instead of the deprecated | |
| mongo-client library | |
| Special thanks to Hugo Soszynski and Jérémie Jourdin for there hard work | |
| to make this a reality! | |
| See also: https://github.com/rsyslog/rsyslog/pull/1917 | |
| - rainerscript: add parse_time() function | |
| Thanks to Stephen Workman for implementing this. | |
| - omelasticsearch: add LF to every elastic response for error file | |
| error file content was written without LF after each message, making | |
| it hard to work with and monitor. | |
| Thanks to Yaroslav Bo for the patch. | |
| - omelasticsearch: add pipeline support | |
| supports static and dynamic ElasticSearch pipeline parameter. | |
| closes https://github.com/rsyslog/rsyslog/issues/1411 | |
| Thanks to github users scibi and WaeCo for the implementation. | |
| - lmsig_ksi_ls12: support asynchronous mode of libksi | |
| - omprog: added error handling and transaction support for external plugins | |
| This permits much better integration of external output modules. | |
| Special thanks to Joan Sala for providing this work! | |
| - imzmq3/omzmq3: marked as deprecated, modules will be remove in v8.41 | |
| see also: https://github.com/rsyslog/rsyslog/issues/2100 | |
| - imzmq3/omzmq3: fixed build issues with gcc-7 | |
| - core: emit error message on abnormal input thread termination | |
| This indicates a serious issue of which the user should be notified. | |
| Was not done so far and caused grief when troubleshooting issues. | |
| - core: refactored locking for json variable access | |
| refactored the method so that it consistent in all functions and easier | |
| to follow. Most importantly, now an as simple as possible locking paradigm | |
| of lock ... unlock within the function. Hopefully easier to understand | |
| both for humans and static code analyzers. | |
| - core: refactored creation of UDP sockets | |
| was very old, hard to follow code; streamlined that a bit | |
| - core/dnscache: refactor locking | |
| keep a simple lock ... unlock scheme within the function. That is | |
| easier to follow for both humans as well as static analyzers. | |
| Also removes Coverity scan CID 185419 | |
| - rainerscript: use crypto-grade random number generator for random() function | |
| We now use /dev/urandom if available. If not, we fall back to the weak PRNG. | |
| - imkafka: improve error reporting and cleanup refactoring | |
| - imkafka bugfix: segfault if "broker" parameter is not specified | |
| Now emits warning message instead and uses hardcoded default | |
| (which usually matches where the kafka broker can be found). | |
| fixes https://github.com/rsyslog/rsyslog/issues/2085 | |
| - omkafka: improve error reporting | |
| - omkafka: slight speedup do to refactoring of LIST class | |
| double-linked list was used for temporarily undeliverable message tracking | |
| where singly-linked list was sufficient. Changed that. | |
| - TCP syslog: support SNI when connecting as a client | |
| This is done if a hostname is configured and turned off if an IP is used. | |
| Thanks to Art O Cathain for the patch. | |
| see also https://github.com/rsyslog/rsyslog/pull/1393 | |
| - msg variable bugfix: potential segfault on variable access | |
| A segfault is likely to happen if during processing a variable with | |
| more than one path component is accessed (e.g. $!path!var) and NO | |
| variables oft hat kind (local, message, global) are defined at all. | |
| closes https://github.com/rsyslog/rsyslog/issues/1920 | |
| - ratelimiting bugfix: data race in Linux-like ratelimiter | |
| access to the Linux-like ratelimiter was not properly synchronized, and | |
| the system rate-limiter was not using it in any case. | |
| This could lead to the rate-limit settings not being properly | |
| respected, but no "hard error". | |
| - core/template bugfix: potential NULL pointer access at config load | |
| could happen if the config was loaded and the parameters could not properly | |
| be processed. If so, this occured only during the startup phase. | |
| Detected by Coverity scan, CID 185318 | |
| - core/json var subsystem bugfix: segfault in MsgSetPropsViaJSON | |
| Invalid libfastjson API use lead to double-free. This was most importantly | |
| triggerred by mmexternal (but could potentially be triggered by other | |
| uses as well) | |
| closes https://github.com/rsyslog/rsyslog/issues/1822 | |
| - core/wrkr threads bugfix: race condition | |
| During e.g. shutdown it could happen that a worker thread was started | |
| and immediately be requested to terminate. In such situations there was | |
| a race the prevented proper initialization. This could lead to follow-on | |
| problems. | |
| We believe (but could not proof) that this also could lead to a hang of | |
| the termination process. Thus we claim to close an issue tracker down | |
| here below, but are not 100% certain it really closes it (we hope for | |
| user feedback on this). In any case, the thread debuger showed invalid | |
| operations and this has been fixed, so it is useful in any case. | |
| closes https://github.com/rsyslog/rsyslog/issues/1959 | |
| - core/wtp: potential hang during shutdown | |
| when the wtp worker is cancelled in the final stage of shutting down | |
| while the mutex is locked and there is one worker left, the system | |
| will hang. The reason is that the cancelled thread could not free the | |
| mutex that the other needs in order to shut down orderly. | |
| Deteced with clang thread sanitizer. | |
| - omfwd bugfix: generate error message on connection failure | |
| - imtcp bugfix: "streamdriver.mode" parameter could not be set to 0 | |
| - imjournal bugfix: module was defunctional | |
| The open function was broken by commit 92ac801, resulting in | |
| no data being ever read from the journal. | |
| patch bases on the idea of Radovan Sroka given here: | |
| https://github.com/rsyslog/rsyslog/issues/1895#issuecomment-339017357 | |
| but follows the current imjournal-paradigm of having the journal | |
| handle inside a global variable. | |
| see also https://github.com/rsyslog/rsyslog/issues/1895 | |
| closes https://github.com/rsyslog/rsyslog/issues/1897 | |
| - imjournal: refactor error handling, fix error messages | |
| imjournal did not try to recover from errors and used the error state | |
| returned by journal functions incorrectly, resulting in misleading | |
| error messages. Fixed this and also increased the number of error | |
| messages so that it now is easier to diagnose problems with this module. | |
| Also a little bit of internal brush-up. | |
| -mmdblookup bugfix: fix potential segfault due to threading issues | |
| libmaxminddb seems to have issues when running under multiple threads. As | |
| a first measure, we prevent mmdblookup now from running on more than one | |
| thread concurrently. | |
| see also: https://github.com/rsyslog/rsyslog/issues/1885#issuecomment-344882616 | |
| - omelasticsearch bugfix: operational error messages not passed to user | |
| This lead to sometimes hard to diagnose problem. Note that for obvious | |
| raesons the amont of messages from omelasticsearch has increased; this | |
| is not a sign of a problem in itself -- but it may reveal problems that | |
| existed before and went unnoticed. Also note that too-verbose messages | |
| can be filtered out via regular rsyslog methods (e.g. message discarding | |
| based on content). | |
| - omkafka bugfixes | |
| * statistics counter maxoutqsize could not be reset | |
| Thanks to github user flisky for the patch. | |
| * potential hang condition | |
| omkafka did not release a mutex under some error conditions (most | |
| notably out of memory on specific alloc calls). This lead to a hang | |
| during actively processing messages or during shutdown (at latest). | |
| This could happen only if dynamic topics were configured. | |
| Detected by Coverity Scan, CID 185781 (originally 185721, detected | |
| as a different issue by Coverity after code refactoring done in regard | |
| to 185721 -- then the problem became more obvious). | |
| * file handle leak, which could occur when local buffering | |
| of messages was needed | |
| * invalid load of failedmsg file on startup if disabled | |
| error "rsyslogd: omkafka: could not load failed messages from " | |
| "file (null) error -2027 - failed messages will not be resend." | |
| occurs because, depite `keepFailedMessages="off"` as a default, | |
| omkafka still tries to check for and load a `(none)` file which | |
| triggers an IO error of sorts according to the 2027 link above. | |
| Obviously, it probably shouldn't try load the file if | |
| `keepFailedMessages` is not enabled. | |
| Thanks to github user JPvRiel for a very good error report and | |
| analysis. | |
| closes https://github.com/rsyslog/rsyslog/issues/1765 | |
| * various config parameters did not work | |
| These were not even recognized when used and lead to a config startup | |
| error message: | |
| ~ closeTimeout | |
| ~ reopenOnHup | |
| ~ resubmitOnFailure | |
| ~ keepFailedMessages | |
| ~ failedMsgFile | |
| closes https://github.com/rsyslog/rsyslog/issues/2052 | |
| * considerable memory leak | |
| Whenever a message could (temporarily) not be delivered to kafka, | |
| a non-trivial amount of memory was leaked. This could sum up to | |
| quite a big memory leak. | |
| fixes https://github.com/rsyslog/rsyslog/issues/1991 | |
| * some small memory leaks fixed | |
| most of them cosmetic or a few bytes statically (not growing as | |
| omkafka was used) -- thus we do not mention each one explicitely. | |
| For more details, see git commit log or this pull request: | |
| https://github.com/rsyslog/rsyslog/pull/2051 | |
| - kafka bugfix: problem on invalid kafka configuration values | |
| omkafka ended up in an endless loop and high cpu. | |
| imkafka tried to subscribe to a not connected kafka server. | |
| closes https://github.com/rsyslog/rsyslog/issues/1806 | |
| - [io]mgssapi: fix build problems (regression from 8.30.0) | |
| - [io]czmq: fix build problems on some platforms (namely gcc 7, clang 5) | |
| - tcpsrv bugfix: potential hang during shutdown | |
| - queue bugfix: potential hang during shutdown | |
| - queue bugfix: NULL pointer dereference during config processing | |
| If the queue parameters were incorrectly given, a NULL pointer derefernce | |
| could happen during config parsing. Once post that stage, no problem could | |
| occur. | |
| Detected by Coverity scan, CID 185339 | |
| - imczmq bugfix: segfault | |
| happened in a call to | |
| 371: zcert_destroy(&serverCert) called from rcvData(). | |
| Thanks to ~achiketa Prachanda for the patch. | |
| - imfile: some small performance enhancements | |
| Thanks to Peter Portante for the patch | |
| - omfile: hande file open error via SUSPEND mode | |
| For a while, an open file error lead to suspension as the error was | |
| not detected by the rule engine. This has changed with fixes | |
| in 8.30.0. I asked users what they prefer (and expect) and | |
| everyone meant it should still be handled via suspension. See | |
| github tracker below for more details. | |
| closes https://github.com/rsyslog/rsyslog/issues/1832 | |
| - omfile bugfix: race during directory creation can lead to loop | |
| There was a race where two threads were competing for directory creation | |
| which could lead to none succeeding and a flood of error message like this | |
| "error during config processing: omfile: creating parent directories for | |
| file". This has been solved. | |
| Thanks to Niko Kortström for the patch. | |
| - imudp: improve error reporting | |
| When udp listener cannot be created, an error message containing | |
| the ip-address and port is put out. | |
| closes https://github.com/rsyslog/rsyslog/issues/1899 | |
| - omrelp bugfix: incorrect error handling | |
| if librelp with TLS but without Authentication was included, librelp | |
| did not emit the correct error message due to invalid error code | |
| check. It also did not err-out but instead suspended itself. | |
| Detected by Coverity scan, CID 185362 | |
| - [io]mrelp bugfix: segfault on startup if configured cert not readable | |
| When the certificate file specified in the omrelp/imrelp configuration | |
| can't be accessed, e.g. because it doesn't exist or you don't have | |
| permission to do so, a Segmentation Fault will appear when you start | |
| Rsyslog. This commit fixes that problem. | |
| closes https://github.com/rsyslog/rsyslog/issues/1869 | |
| - mmanon fix: make build under gcc 7 | |
| Thanks to William Dauchy for the patch | |
| - mmpstrucdata bugfix: formatting error of ']' char | |
| This was invalidly formatted as '"'. Thanks to github user | |
| wu3396 for the error report including the patch idea. | |
| closes https://github.com/rsyslog/rsyslog/issues/1826 | |
| - mmexternalb bugfix: memory leak | |
| - core/stats bugfix: memory leak if sender stats or tracking are enabled | |
| - core bugfix: potential segfault during startup | |
| A NULL pointer could be accessed if there was a problem with the | |
| timezone parameters. Affects only startup, once started, no problem | |
| existed. | |
| Detected by Coverty scan; CID 185414 | |
| - core bugfix: potential race in variable handling | |
| Root of variable tree is accessed prior to locking access to it. | |
| This introduces a race that may result in various kinds of | |
| misadressing. | |
| Found while reviewing code, no bug report exists. | |
| - core bugfix: potential segfault when shutting down rsyslog | |
| when rulesets are nested a segfault can occur when shutting down | |
| rsyslog. the reason is that rule sets are destructed in load order, | |
| which means a "later" ruleset may still be active when an "earlier" | |
| one was already destructed. In these cases, a "call" can invalidly | |
| call into the earlier ruleset, which is destructed and so leads to | |
| invalid memory access. If a segfault actually happens depends on the | |
| OS, but it is highly probable. | |
| The cure is to split the queue shutdown sequence. In a first step, | |
| all worker threads are terminated and the queue set to enqOnly. | |
| While some are terminated, it is still possible that the others | |
| enqueue messages into the queue (which are then just placed into the | |
| queue, not processed). After this happens, a call can no longer | |
| be issued (as there are no more workers). So then we can destruct | |
| the rulesets in any order. | |
| closes https://github.com/rsyslog/rsyslog/issues/1122 | |
| - core/action bugfix: potential misadressing when processing hard errors | |
| For batches that did fail in an output module, the rsyslog core | |
| tries to find offending messages that generate hard (non-recoverable) | |
| errors. During this process, the action templates for each message | |
| are copied into a temporary buffer. That copy was invalidly sized, | |
| actually copying only the first template string. As such, outputs | |
| that requested more template strings AND had erros in batch submission | |
| received uninitialized pointers. This could lead to all sorts of | |
| problems. | |
| see also https://github.com/rsyslog/rsyslog/issues/1885 | |
| closes https://github.com/rsyslog/rsyslog/issues/1906 | |
| - template object bugfix: NULL pointer access on invalid parameters | |
| could happen only during startup | |
| Detected by Coverity scan, CID 185376 | |
| - omjournal bugfix: NULL pointer access on invalid parameters | |
| could happen only during startup | |
| - omelasticsearch bugfix: configured credentials not used during health check | |
| Authentication credentials were not applied during health check, | |
| permission to use unsigned CERTS was not applied to regular data post. | |
| closes https://github.com/rsyslog/rsyslog/issues/1949 | |
| - omelasticsearch bugfix: abort on unavailable ES server | |
| Depending on the state of unavailability (libcurl return code), | |
| omelasticsearch tries to process a NULL return message, what | |
| leads to a segfault. | |
| This fixes the problem and introduces better error handling and | |
| better error messages. | |
| see also https://github.com/rsyslog/rsyslog/issues/1885 | |
| - omelasticsearch: fix memory leak and potential misadressing | |
| Commit 96b5fce introduced regressions, leading to potential misadressing | |
| and a very probable memory leak. This commit fixes the issues and | |
| hardens the module to better detect some error cases in the | |
| future. | |
| It also adds valgrind-based testbench tests which ensure that no | |
| pointer errors exist. If they had been in place, the regressions | |
| would never have been undetected. | |
| Note that commit 96b5fce was introduced in 8.23.0 on Nov, 15th 2016. | |
| Many thanks to Yaroslav Bo for alerting me on the root problem and | |
| providing a very good analysis and guidance. | |
| see also https://github.com/rsyslog/rsyslog/issues/1906 | |
| see also https://github.com/rsyslog/rsyslog/issues/1964 | |
| closes https://github.com/rsyslog/rsyslog/issues/1962 | |
| - omelasticsearch bugfix: output from libcurl to stdout | |
| omelasticsearch made libcurl output messages to stdout. This | |
| commit fixes that. It also automatically enables libcurl verbose | |
| mode during debug runs - it needs to be seen if this is smart or | |
| not (previously, code needed to be commented in). | |
| closes https://github.com/rsyslog/rsyslog/issues/1909 | |
| - iczmq bugfix: potential memory leak | |
| - imptcp bugfix: potential misadressing | |
| When during a connection request the remote peer could not be | |
| identified, imptcp could misadress memory if it is configured | |
| to emit messages on connection open. | |
| Detected by clang 5.0 static analyzer. | |
| - imptcp: potential buffer overflow | |
| if the local hostname or IP is larger than NI_MAXHOST-1, an internal | |
| buffer is overflowed. This is irrespective of any input data. | |
| Detected by Coverity scan, CID 185390 | |
| - core/nsd_gtls: fix potential unitialized data access | |
| could occur during certificate check | |
| found by clang 5.0 static analyzer | |
| - stats bugfix: potential program hang | |
| due to missing unlock. This could only occur if pstats was set to | |
| CEE-format logging (very uncommon) AND if the system runs out of | |
| memory (in which case other things go pretty bad as well). | |
| found by Coverty scan | |
| - omfwd bugfix: memory leak if network namespaces are used | |
| very seldom used feature, occurs only during error case | |
| found by Coverty scan. | |
| - core: potential misadressing when accessing JSON properties | |
| When a JSON property is accessed in template processing, memory may | |
| have been misadressed or a double-free may occur while obtaining the | |
| propety. | |
| This was detected by a Coverty scan. | |
| - gcry crypto provider bugfixes: potential misadressing and memory leak | |
| If the config parameters were invalid, a misadressing could occur. If so, | |
| this happens only during startup. | |
| Also, a memory leak existed when the crypto provider errored out. This could | |
| build up if it were used in frequently-changing dynafiles. This was | |
| detected by Coverity scan, CID 185360. | |
| - core/file stream object bugfix: memory leak | |
| If a multiline read errored out, a line buffer could be leaked. | |
| Detected by Coverity scan, CID 185328 | |
| - imdiag bugfix: double mutex unlock when working with stats | |
| Note: while this could potentially lead to a program hang, it affected | |
| only testbench execution as imdiag is a testbench-only tool. | |
| Detected by Coverity scan, CID 185348 and 185350 | |
| - fixed several minor and cosmetic issues found by Coverty scan | |
| includding false positives. For details see "$ git log". All noteworthy | |
| issues are seperately mentioned in this ChangeLog. The ones not mentioned | |
| are for example problems that can only occur during out of memory | |
| conditions, under which it is extremely likely tha the rsyslog process | |
| will be killed in any case | |
| - testbench: | |
| * added compile-check for [io]mgssapi, mmcount | |
| * harden tests against hanging previous instances | |
| * re-enable RELP tests on Solaris | |
| * added basic test for imjournal | |
| * added threading tests via valgrind's helgrind tool | |
| * added valgrind test for kafka modules | |
| * added capability to run elasticsearch tests with | |
| a) different ElasticSearch versions | |
| b) independently from OS-installed version | |
| This also sets base to enable more elaboreate ES tests | |
| * further relaxed timing of dynstats tests, as they tend to create | |
| false positives on slow machines | |
| - CI: improved test coverage on FreeBSD | |
| - Travis: clang static analyzer 5.0 now run over all source files | |
| - build: make compile warning-free under gcc 7 | |
| ------------------------------------------------------------------------------ | |
| Version 8.30.0 [v8-stable] 2017-10-17 | |
| - NEW BUILD REQUIREMENTS | |
| * libfastjson 0.99.7 is now mandatory | |
| the new version is required to support case-insensitive variable | |
| comparisons, which are now the default | |
| * when building imjournal, libsystemd-journal >= 234 is now recommended | |
| This is to support the imjournal enhancement. Note that it is possible | |
| to build with lower version, but this will degrade imjournal functionality. | |
| - CHANGE OF BEHAVIOUR: all variables are now case-insensitive by default | |
| Formerly, JSON based variables ($!, $., $/) were case-sensitive. | |
| Turn old default back on: global(variables.casesensitve="on") | |
| See ChangeLog entry below for more details. | |
| - core: handle (JSON) variables in case-insensitive way | |
| The variable system inside rsyslog is JSON based (for easy consumption | |
| of JSON input, the prime source of structured data). In JSON, keys | |
| ("variable names") are case-sensitive. This causes constant problems | |
| inside rsyslog configurations. A major nit is that case-insensitivity | |
| option inside templates (even if turned on) does not work with JSON | |
| keys because they of inner workings*1. | |
| It is much more natural to treat keys in a case-INsensitive way (e.g. | |
| "$!Key" and "$!key" are the same). We do not expect any real problems | |
| out of this, as key names only differing in case is highly unlikely. | |
| However, as it is possible we provide a way to enable case-sensitivity | |
| via the global(variables.casesensitve="on") global config object. | |
| Note that the default is to do case-insensitive matches. The reason | |
| is that this is the right thing to do in almost all cases, and we do | |
| not envision any problems at all in existing deployments. | |
| *1 Note: case-insensitivity in rsyslog is achieved by converting all | |
| names to lower case. So that the higher speed of strcmp() can be used. | |
| The template option does actually that, convert the template keys to | |
| lower case. Unfortunately, this does not work with JSON, as JSON keys | |
| are NOT converted to lower case. | |
| closes https://github.com/rsyslog/rsyslog/issues/1805 | |
| - imjournal: made switching to persistent journal in runtime possible | |
| with this patch imjournal can continue logging after switch to | |
| persistent journal without need to restart rsyslog service | |
| Thanks to github user jvymazal for the patch | |
| - mmanon: complete refactor and enhancements | |
| - add pseudonymization mode | |
| - add address randomization mode | |
| - add support for IPv6 (this also supports various replacement modes) | |
| closes https://github.com/rsyslog/rsyslog/issues/1614 | |
| also fixes bugs | |
| - in IPv4 address recognition | |
| closes https://github.com/rsyslog/rsyslog/issues/1720 | |
| - in IPv4 simple mode to-be-anonymized bits can get wrong | |
| closes https://github.com/rsyslog/rsyslog/issues/1717 | |
| - imfile: add "fileoffset" metadata | |
| This permits to send the offset from which the message was read together | |
| with the message text. | |
| Thanks to github user derekjsonar for the initial implementation which | |
| we extended to use the message start offset. | |
| - RainerScript: add ltrim and rtrim functions | |
| closes https://github.com/rsyslog/rsyslog/issues/1711 | |
| - core: report module name when suspending action | |
| Thanks to Orion Poplawski for the patch. | |
| - core: add ability to limit number of error messages going to stderr | |
| This permits to put a hard limit on the number of messages that can | |
| go to stderr. If for nothing else, this capability is helpful for the | |
| testbench. It permits to reduce spamming the test log while still | |
| providing the ability to see initial error messages. Might also be | |
| useful for some practical deployments. | |
| global parameter: global(errorMessagesToStderr.maxNumber) | |
| - tcpsrv subsystem: improvate clarity of some error messages | |
| operating system error message are added to some messages, providing | |
| better advise of what actually is the error cause | |
| - imptcp: include module name in error msg | |
| - imtcp: include module name in error msg | |
| - tls improvement: better error message if certificate file cannot be read | |
| - omfwd: slightly improved error messages during config parsing | |
| They now contain config file/line number information where this was missing. | |
| - ommysql improvements | |
| * Return specifc code for unrecoverable errors. This makes retry processing | |
| more performant and robust. | |
| * error messages improved | |
| * Update to utilize native v8 transactional interface. Previously, it used | |
| the v7 interface with v8 emulation. | |
| * treat server and client-generated messages differently | |
| Server-generated error messages are considered non-recoverable, while | |
| client generated once point into connection problems (which simply can | |
| be retried). This is part of the improvements towards better | |
| message-induced errors. Previous commits did only handle SQL parsing | |
| errors, now we aim to address all of the message-induced error. We assume | |
| that all server-generated errors are such - mysql API unfortunately does | |
| not provide a clear indication of what type if error this is and it is | |
| out of question to check for hundereds of error codes. | |
| closes https://github.com/rsyslog/rsyslog/issues/1830 | |
| - ommysql bugfix: do not duplicate entries on failed transaction | |
| If a multi-message batch contains data errors, messages may be | |
| duplicated as connection close is implicit commit (not rollback). | |
| This patch introduces a specific rollback request in those cases. | |
| closes https://github.com/rsyslog/rsyslog/issues/1829 | |
| - imtcp bugfix: parameter priorityString was ignored | |
| defaults were always used | |
| - template/bugfix: invalid template option conflict detection | |
| This prevented "option.casesenstive" to be used with the SQL and JSON | |
| options. | |
| - core/actions: fix handling of data-induced errors | |
| Rsyslog core should try to detect data-induced (unrecoverable) errors | |
| during action processing. An example of such is invalid SQL statements. | |
| If the action returns a data error, rsyslog shall retry individual | |
| messages from a batch in an effort to log those without errors. The others | |
| shall be dropped. | |
| This logic was temporarily disabled after the switch to the new v8 | |
| transaction interface. Now this bug is fixed and the testbench has been | |
| ammended to detect problems in the future. | |
| closes https://github.com/rsyslog/rsyslog/issues/974 | |
| - core/action bugfix: no "action suspended" message during retry processing | |
| The action engine does not emit "action suspended" messages but "resumed" | |
| ones in retry processing. This is wrong, as they are a strong indication | |
| that something does not work correctly. Nevertheless, "resumed" messages | |
| were emitted, which was very puzzling for the user. | |
| This patch fixes it so that "suspend" messages are given during retry | |
| processing. These do not contain a retry timestamp, providing evidence | |
| that a retry is currently being tried. | |
| coses https://github.com/rsyslog/rsyslog/issues/1069 | |
| - core/ratelimit bugfix: race can lead to segfault | |
| There was a race in iminternalAddMsg(), where the mutex is | |
| released and after that the passed message object is accessed. | |
| If the mainloop iterates in between, the msg may have already | |
| been deleted by this time, resulting in a segfault. | |
| Most importantly, there is no need to release the mutex lock | |
| early, as suggested by current code. Inside mainloop the mutex | |
| is acquired when it is time to do so, so at worst we'll have a | |
| very slight wait there (which really does not matter at all). | |
| This only happens if a large number of internal messages are emitted. | |
| closes https://github.com/rsyslog/rsyslog/issues/1828 | |
| - core bugfix: rsyslog aborts if errmsg is generated in early startup | |
| Note that the segfault can occur only during early startup. Once | |
| rsyslog has started, everything works reliably. This condition can | |
| especially be triggerred by specifying invalid TLS default certificates. | |
| closes https://github.com/rsyslog/rsyslog/issues/1783 | |
| closes https://github.com/rsyslog/rsyslog/issues/1786 | |
| - core bugfix: informational messages was logged with error severity | |
| When the main loop reaped a child process (a normal action), this was | |
| reported as an error. This caused user confusion. Now it is reported as | |
| an informational message. | |
| - core bugfix: --enable-debugless build was broken | |
| This was a regression from the v8.29.0 debug enhancements | |
| Thanks to Stephen Workman for the patch. | |
| - queue bugfix: file write error message was incorrect | |
| when a queue was restarted from disk file, it almost always | |
| emitted a message claiming | |
| "file opened for non-append write, but already contains xxx bytes" | |
| This message was wrong and did not indicate a real error condition. | |
| The predicate check was incorrect. | |
| closes https://github.com/rsyslog/rsyslog/issues/170 (kind of) | |
| - omrelp bugfix: segfault when rebindinterval parameter is used | |
| - imudp bugfix: UDP oversize message not properly handled | |
| When a message larger than supported by the UDP stack is to be sent, | |
| EMSGSIZE is returned, but not specifically handled. That in turn | |
| will lead to action suspension. However, this does not make sense | |
| because messages over the UDP max message size simply cannot be sent. | |
| closes https://github.com/rsyslog/rsyslog/issues/1654 | |
| - core bugfix: memory corruption during configuration parsing | |
| when omfwd is used with the $streamdriverpermittedpeers legacy | |
| parameter, a memory corruption can occur. This depends on the | |
| length of the provided strings and probably the malloc subsystem. | |
| Once config parsing succeeds, no problem can happen. | |
| Thanks to Brent Douglas for initially reporting this issue and | |
| providing great analysis. | |
| Thanks to github user bwdoll for analyzing this bug and providing | |
| a suggested fix (which is almost what this commit includes). | |
| closes https://github.com/rsyslog/rsyslog/issues/1408 | |
| closes https://github.com/rsyslog/rsyslog/issues/1474 | |
| - core bugfix: race on worker thread termination during shutdown | |
| The testbench got some occasionally failing tests. Review of | |
| them brought up the idea that there is a race during worker | |
| threat termination. Further investigation showed that this | |
| might be a long-standing issue, but so far did not really | |
| surface as the timing was almost always correct. However, | |
| with the new functionality to emit a message on worker | |
| shutdown (v8.29), the timing got more complex and now this | |
| seemed to occasionally surface. | |
| closes https://github.com/rsyslog/rsyslog/issues/1754 | |
| - omelasticsearch: avoid ES5 warnings while sending json in bulkmode | |
| do this by adding proper content type header to ES request | |
| Thanks to William Dauchy for the patch | |
| - omelasticsearch bugfix: incompatibility with newer ElasticSearch version | |
| ElasticSearch changed its API in newer versions. When "bulkmode" is enabled | |
| in omelasticsearch, rsyslog seems to consider all responses from Elasticsearch | |
| as errors, even the successful ones. As a consequence, every single request | |
| ends up being logged into the error file. | |
| closes https://github.com/rsyslog/rsyslog/issues/1731 | |
| Thanks to Vincent Quéméner for the patch. | |
| - imptcp bugfix: invalid mutex addressing on some platforms | |
| code did not compile on platforms without atomic instructions | |
| Thanks to github user petiepooo for the patch | |
| - imptcp bugfix: do not accept missing port in legacy listener definition | |
| If legacy listener definition was used, a missing port was accepted during | |
| the config read phase but errored out upon listener activation. This now | |
| errors out directly when processing the config directive. | |
| ------------------------------------------------------------------------------ | |
| Version 8.29.0 [v8-stable] 2017-08-08 | |
| - imptcp: add experimental parameter "multiline" | |
| This enables a heuristic to support multiline messages on raw tcp syslog | |
| connections. | |
| - imptcp: framing-related error messages now also indicate remote peer | |
| This is the case e.g. for oversize messages. | |
| - imtcp: framing-related error messages now also indicate remote peer | |
| This is the case e.g. for oversize messages. | |
| - imptcp: add session statistics conunter | |
| - session.opened | |
| - session.openfailed | |
| - session.closed | |
| - imtcp: add ability to specify GnuTLS priority string | |
| This permits to set cipher details on a very low level. | |
| - impstats: add new ressoure counter "openfiles" | |
| - pmnormalize: new parser module | |
| Parser module that uses liblognorm to parse incoming messages. | |
| - core/queue: provide informational messages on thread startup and shutdown | |
| This provides better insight into operational state of rsyslog and is useful | |
| in monitoring system health. Note that this causes the emission of messages | |
| not previously seen. They are of syslog.informational priority. | |
| - omfwd/udp: improve error reporting, depricate maxerrormessages parameter | |
| Generally improved udp-related error messages (e.g. they now contain the | |
| socket number, which makes it easier to related them to errors reported by | |
| net.c subsystem). | |
| We also depricated (removed) the "maxerrormessages" configuration parameters. | |
| It provided some very rough rate-limiting capabilities and was introduced | |
| before we had native rate-limiters. The default was that only the first 5 | |
| error messages were actually reported. For long-running instances, that | |
| meant that in many cases no errors were ever reported. We now use the default | |
| internal message rate limter, which works far better and ensures that also | |
| long-running instances will be able to emit error messages after prolonged | |
| runtime. In contrast, this also means that users will see more error | |
| messages from rsyslog, but that should actually improve the end user | |
| experience. | |
| - core: add parameters debug.file and debug.whitelist | |
| allows to generate debug log output only of specific files | |
| Background information available at: | |
| https://www.linkedin.com/pulse/improving-rsyslog-debug-output-jan-gerhards | |
| - core/net.c: improve UDP socket creation error messages | |
| - omfwd/udp: add "udp.sendbuf" parameter | |
| - core: make rsyslog internal message rate-limiter configurable | |
| New parameters "internalmsg.ratelimit.interval" and "internalmsg.ratelimit.burst" | |
| have been added. | |
| - omelasticsearch bugfixes and changed ES5 API support: | |
| * avoid 404 during health check | |
| Omleasticsearch responds differently to HTTP HEAD and GET requests and | |
| returns correct state only on GET requests. This patch works around | |
| that ES bug and always does a GET request even when technically a HEAD | |
| request would be sufficient. | |
| * avoid ES5 warnings while sending json | |
| ES5 is generating warnings when sending json without the proper header: | |
| $ curl -i -H "Content-Type: text/json" -XGET 'http://elasticsearch5:9200/' \ | |
| -d '{}\n' | |
| HTTP/1.1 200 OK | |
| Warning: 299 Elasticsearch-5.4.3-eed30a8 "Content type detection for rest | |
| requests is deprecated. Specify the content type using the [Content-Type] | |
| header." "Wed, 26 Jul 2017 14:33:28 GMT" | |
| no issue on previous version. | |
| Now, the header is set as application/json. It works for all versions | |
| (tested on ES2 and ES5) we also handle the bulkmode where it should be | |
| set to application/x-ndjson | |
| closes https://github.com/rsyslog/rsyslog/issues/1546 | |
| * bugfix for memomry leak while writing error file | |
| Thanks to William Dauchy for providing the patches | |
| - imfile bugfix: wildcard detection issue on path wildcards | |
| Wildcards mode was not properly detected when wildcards | |
| were only used in a directory name on startup. | |
| This caused imfile not to create a propper dynamic filewatch. | |
| closes: https://github.com/rsyslog/rsyslog/issues/1672 | |
| - omfwd bugfix: always give reason for suspension | |
| In many cases, no error message were emitted when omfwd | |
| went into action suspension, which was confusing for end | |
| users. This patch enables explicit error messages in all | |
| those cases. | |
| closes https://github.com/rsyslog/rsyslog/issues/782 | |
| - omfwd bugfix: configured compression level was not properly used | |
| Thanks to Harshvardhan Shrivastava for the patch. | |
| - imptcp bugfix: potential socket leak on session shutdown | |
| imptcp first tries to remove a to-be-shut-down socket from the | |
| epoll set, and errors out if that does not work. In that case, the | |
| underlying socket will be leaked. | |
| This patch refactors the code; most importantly, it is not necessary | |
| to remove the socket from the epoll set, as this happens automatically | |
| on close. As such, we simply remove that part of the code, which | |
| also removes the root cause of the socket leak. | |
| - omfwd/omudpspoof bugfix: switch udp client sockets to nonblocking mode | |
| On very busy systems, we see "udp send error 11" inside the logs, and the requesting | |
| action is being suspended (and later resumed). During the suspension period (in | |
| default configuration), messages are lost. Error 11 translates to EAGAIN and the | |
| cause of this problem is that the system is running out of UDP buffer space. This | |
| can happen on very busy systems (with busy networks). | |
| It is not an error per se. Doing a short wait will resolve the issue. The real root | |
| cause of the issue is that omfwd uses a nonblocking socket for sending. If it were | |
| blocking, the OS would block until the situation is resolved. The need for a | |
| non-blocking sockets is a purely historical one. In the days of single-threaded | |
| processing (pre v3), everything needed to be done by multiplexing, and blocking was | |
| not permitted. Since then, the engine has dramatically changed. Actions now run on | |
| their own thread(s). As such, there is no longer a hard need to use non-blocking i/o | |
| for sending data. Many other output plugins also do blocking wait (e.g. omelasticsearch). | |
| As such, the real root cause of the trouble is unnecessarily using non-blocking mode, | |
| and consequently the right solution is to change that. | |
| Note that using blocking i/o might change some timeing inside rsyslog, especially | |
| during shutdown. So theoretical there is regression potential in that area. However, | |
| the core is designed to handle that situation (e.g. there is special shutdown code to | |
| handle the blocking case), so this does not stand against the "proper" solution. | |
| This patch applies the change on the rsyslog core level, within net.c. The only | |
| users of the changed functionality are omfwd and omudpspoof. Imudp is unaffected as | |
| it requests server sockets. | |
| Note that according to the sendto() man page, there is a second cause for the EAGAIN | |
| error, this is when the system temporarily runs out of emphermeral ports. It is not | |
| 100% clear if this can also happen in the blocking case. However, if so, we can argue | |
| this is a case where we really want the default retry logic. So for the time being, | |
| it is appropriate to not handle EAGAIN in a special case any longer. | |
| closes https://github.com/rsyslog/rsyslog/issues/1665 | |
| - imklog: fix permitnonkernelfacility not working | |
| - impstats bugfix: impstats does not handle HUP | |
| If the parameter "log.file" is specified, impstats writes its own | |
| log file. However, HUP is not handled for this file, which makes | |
| the functionality unusable with log rotation. It is also counter- | |
| intuitive for users. | |
| This patch enables correct HUP processing. As a sideline, it also | |
| introduces a generic HUP processing framework for non-action type | |
| of loadable modules. | |
| closes https://github.com/rsyslog/rsyslog/issues/1662 | |
| closes https://github.com/rsyslog/rsyslog/issues/1663 | |
| - core bugfix: segfault after configuration errors | |
| - core/queue bugfixes: | |
| * Fix behaviour of PersistStateInterval | |
| If PersistStateInterval=1, then each log line read should cause the state file | |
| to be updated, but this was not happening because nRecords was being post-increment. | |
| Thanks to Anthony Howe for the patch. | |
| * potential problem during deserialization | |
| if queue object deserialization fails, invalid memory blocks might be | |
| free'ed. | |
| For more information see https://github.com/rsyslog/rsyslog/pull/1647 | |
| Thanks to Derek Smith for the patch. | |
| - core bugfix: messsage garbled after message modification | |
| The MsgDup() function will return a garbled message object under these | |
| conditions: The message was originally created with message length equal or | |
| larger to CONF_RAWMSG_BUFSIZE. This makes rsyslog store the message in | |
| dynamically allocated buffer space. Then, a component reduces the message | |
| size to a size lower than CONGF_RAWMSG_BUFSIZE. A frequent sample is the | |
| parser removing a known-bad LF at the end of the messages. Then, MsgDup is | |
| executed. It checks the message size and finds that it is below | |
| CONF_RAWMSG_BUFSIZE, which make it copy the msg object internal buffer | |
| instead of the dynamically allocated one. That buffer was not written to in | |
| the first place, so unitialized data is copied. Note that no segfault can | |
| happen, as the copied location was properly allocated, just not used in | |
| this processing flow. In the end result, the new message object contains | |
| garbage data. Whenever the new object is used (e.g. in a async ruleset or | |
| action) that garbage will be used. Whenever the old object is accessed, | |
| correct data will be used. Both types of access can happen inside the | |
| same processing flow, which makes the problem appear to be random. | |
| closes https://github.com/rsyslog/rsyslog/issues/1658 | |
| - lmsig_ksi: removed pre-KSI_LS12 components | |
| As of GuardTime, the provider, these no longer work due to backend | |
| changes. The lmsig_ksi_ls12 module shall be used instead. This is | |
| available since 8.27.0. | |
| - testbench bugfix: hang in tests if omstdout is not present | |
| Many tests depend on omstdout. Given the fact that omstdout | |
| is really only useful for the testbench (at least that's the intent), | |
| we now require --enable-omstdout if --enable-testbench is given. | |
| The alternative would have been to disable all those tests that | |
| need it, which would have lead to considerable less testbench | |
| coverage. | |
| closes https://github.com/rsyslog/rsyslog/issues/1649 | |
| ------------------------------------------------------------------------------ | |
| Version 8.28.0 [v8-stable] 2017-06-27 | |
| - NEW BUILD REQUIREMENT: librelp 1.2.14 (to build relp components) | |
| This was necessary because imrelp requires an API introduced in 1.2.14. | |
| - omfwd: add parameter "tcp_frameDelimiter" | |
| - omkafka: large refactor of kafka subsystem | |
| This offers improvements and greatly increases reliablity. | |
| Closes https://github.com/rsyslog/rsyslog/issues/1559 | |
| Closes https://github.com/rsyslog/rsyslog/issues/1584 | |
| Closes https://github.com/rsyslog/rsyslog/issues/1515 | |
| Closes https://github.com/rsyslog/rsyslog/issues/1052 | |
| May fix https://github.com/rsyslog/rsyslog/issues/1230 | |
| - imfile: improved handling of atomically renamed file (w/ wildcards) | |
| if a file is atomically renamed, the state file is also being renamed, | |
| so processing continues as if the original name were kept. | |
| see also: https://github.com/rsyslog/rsyslog/issues/1417 | |
| - imfile: add capability to truncate oversize messages or split into multiple | |
| also in this case an error message is emitted. Both of these actions are | |
| configurable. This also solves memory issues when an endregex does not | |
| match for prolonged time. In that case, large parts of the file were | |
| previously buffered, which could cause big problems in case e.g. the | |
| endregex was simply wrong and never matched. For the later, see also | |
| https://github.com/rsyslog/rsyslog/issues/1552 | |
| - mmdblookup | |
| * upgraded from "contrib" to "fully supported" state | |
| * refactored and simplified code | |
| * added ability to specify custom names for extracted fields | |
| * added ability to spehttp://www.landesrecht-bw.de/jportal/portal/t/6el/page/bsbawueprod.psml/action/portlets.jw.MainAction?p1=0&eventSubmit_doNavigate=searchInSubtreeTOC&showdoccase=1&doc.hl=0&doc.id=jlr-UmwVwGBWrahmen&doc.part=R&toc.poskey=#focuspointcify container name for extracted fields | |
| * bugfix: fixed multiple memory leaks | |
| - imptcp: add new parameter "flowControl" | |
| - imrelp: add "maxDataSize" config parameter | |
| Thanks to Nathan Brown for the patch. | |
| - multiple modules: gtls: improve error if certificate file can't be opened | |
| - omsnare: allow different tab escapes | |
| Thanks to Shane P. Lawrence for the patch. | |
| - omelasticsearch: converted to use libfastjson instead of json-c | |
| json-c was used for historical purposes, and it's source included | |
| within the rsyslog source tree. We now use libfastjson inside all | |
| components. | |
| - imjournal: _PID fallback | |
| * added fallback for _PID proprety when SYSLOG_PID is not available | |
| * introduced new option "usepid" which sets which property should | |
| rsyslog use, it has 3 states system|syslog|both, default is both | |
| * deprecated "usepidfromsystem" option, still can be used | |
| and override the "usepid" | |
| * it is possible to revert previous default with usepid="syslog" | |
| Thanks to Radovan Sroka for the patch | |
| - multiple modules: add better error messages when regcomp is failing | |
| - omhiredis: fix build warnings | |
| Thanks to Brian Knox for the fix. | |
| - imfile bugfix: files mv-ed in into directory were not handled | |
| Thanks to Zachary M. Zhao for the patch. | |
| see also https://github.com/rsyslog/rsyslog/issues/1588 | |
| - omprog bugfix: execve() incorrectly called | |
| this caused failures on some but not all platforms | |
| Thanks to 張君天(Chun-Tien Chang) and Matthew Seaman for the patch. | |
| - imfile bugfix: multiline timeout did not work if state file exists | |
| The timeout feature for multiline reads does not correctly work for | |
| files for which a state file existed. This is usually the case for files | |
| that had been processed by a previous run and that still exist on the | |
| new start. For all other files, especially those monitored by a | |
| wildcard and newly created after the rsyslog start, timeout worked as | |
| expected. | |
| closes https://github.com/rsyslog/rsyslog/issues/1445 | |
| - lmsig_ksi-ls12 bugfix: build problems on some platforms | |
| - core bugfix: invalid object type assertion | |
| This lead to aborts due to failing assertion. Note that this could only | |
| happen during debugging runs which includes runtime instrumentation, | |
| something that never should be done in a stable production build. | |
| So this did not affect ordinary users, only developers in with | |
| deep debugging settings. | |
| - regression fix: local hostname was not always detected properly... | |
| ... on early start (w/o network). Introduced in 8.27.0. | |
| Thanks to github user jvymazal for the patch and whissi for | |
| reporting and helping with the analysis. | |
| - bugfix: format security issues in zmq3 modules | |
| see also: https://github.com/rsyslog/rsyslog/pull/1565 | |
| Thanks to Thomas D. (whissi) for the patch. | |
| - bugfix build system: add libksi only to those binaries that need it | |
| Thanks to Allan Park for the patch. | |
| - bugfix KSI ls12 components: invalid tree height calculation | |
| Thanks to Allan Park for the patch. | |
| - testbench/CI enhancements | |
| * re-enable and add kafka tests | |
| Kafka tests were disabled in 8.27.0 (a regression from imkafka). | |
| * better testbench coverage for mmdblookup | |
| * lmsig_ksi-ls12 is now being built at least on Centos7 | |
| ------------------------------------------------------------------------------ | |
| Version 8.27.0 [v8-stable] 2017-05-16 | |
| - imkafka: add module | |
| - imptcp enhancements: | |
| * optionally emit an error message if incoming messages are truncated | |
| * optionally emit connection tracking message (on connection create and | |
| close) | |
| * add "maxFrameSize" parameter to specify the maximum size permitted | |
| in octet-counted mode | |
| * add parameter "discardTruncatedMsg" to permit truncation of | |
| oversize messages | |
| * improve octect-counted mode detection: if the octet count is larger | |
| then the set frame size (or overly large in general), it is now | |
| assumed that octet-stuffing mode is used. This probably solves a | |
| number of issues seen in real deployments. | |
| - imtcp enhancements: | |
| * add parameter "discardTruncatedMsg" to permit truncation of | |
| oversize messages | |
| * add "maxFrameSize" parameter to specify the maximum size permitted | |
| in octet-counted mode | |
| - imfile bugfix: "file not found error" repeatedly being reported | |
| for configured non-existing file. In polling mode, this message | |
| appeared once in each polling cycle, causing a potentially very large | |
| amout of error messages. Note that they were usually emitted too | |
| infrequently to trigger the error message rate limiter, albeit often | |
| enough to be a major annoance. | |
| - imfile: in inotify mode, add error message if configured file cannot | |
| be found | |
| - imfile: add parameter "fileNotFoundError" to optinally disable | |
| "file not found" error messages | |
| - core: replaced gethostbyname() with getaddrinfo() call | |
| Gethostbyname() is generally considered obsolete, is not reentrant and | |
| cannot really work with IPv6. Changed the only place in rsyslog where | |
| this call remained. | |
| Thanks to github user jvymazal for the patch | |
| - omkafka: add "origin" field to stats output | |
| See also https://github.com/rsyslog/rsyslog/issues/1508 | |
| Thanks to Yury Bushmelev for providing the patch. | |
| - imuxsock: rate-limiting also uses process name | |
| both for the actual limit procesing as well as warning messages emitted | |
| see also https://github.com/rsyslog/rsyslog/pull/1520 | |
| Thanks to github user jvymazal for the patch | |
| - Added new module: KSI log signing ver. 1.2 (lmsig_ksi_ls12) | |
| - rsylsog base functionality now builds on osx (Mac) | |
| Thanks to github user hdatma for his help in getting this done. | |
| - build now works on solaris again | |
| - imfile: fix cross-platform build issue | |
| see also https://github.com/rsyslog/rsyslog/issues/1494 | |
| Thanks to Felix Janda for bug report and solution suggestion. | |
| - bugfix core: segfault when no parser could parse message | |
| This could happen if the default parser chain was changed and the | |
| RFC3164 parser was not included. Never seen in practice, just by | |
| experimenting in lab. | |
| - bugfix core: rate-limit internal messages when going to external log system | |
| Rate-limiting was only applied to messages processed internally. | |
| While the external logging system probably also applies rate-limiting, | |
| it would be preferrable that rsyslog applies the same policies on | |
| internal messages, no matter where they go. This is now the case. | |
| - bugfix core: when obtaining local hostname, a NULL pointer could be | |
| accessed. This depends on many factors, among them that no local host | |
| name is configured in rsyslog.conf AND the local system configuration | |
| also is set to an empty hostname. | |
| Thanks to github user jvymazal for the patch. | |
| - bugfix core: on shutdown, stderr was written to, even if already closed | |
| This lead to messages going to whatever was associated with file | |
| descriptor two. | |
| Thanks to Allan Park for the patch. | |
| - bugfix core: perform MainqObj destruction only when not NULL already | |
| This affects the config object; in theory may lead to misadressing during | |
| config parsing. | |
| Thanks to github user jvymazal for the patch | |
| - bugfix core: memory leak when internal messages not processed internally | |
| In this case, the message object is not destructed, resulting in | |
| a memory leak. Usually, this is no problem due to the low number | |
| of internal message, but it can become an issue if a large number | |
| of messages is emitted. | |
| closes https://github.com/rsyslog/rsyslog/issues/1548 | |
| closes https://github.com/rsyslog/rsyslog/issues/1531 | |
| - bugfix imptcp: potential overflow in octet count computation | |
| when a very large octet count was specified, the counter could overflow | |
| ------------------------------------------------------------------------------ | |
| Version 8.26.0 [v8-stable] 2017-04-04 | |
| - NEW BUILD REQUIREMENT: liblognorm 2.0.3 is required for mmnormalize | |
| If mmnormalize is not built, the build requirements are unchanged. | |
| The new version is necessary because it contains an enhanced API for a | |
| new mmnormalize feature. | |
| - enable internal error messages at all times | |
| This is an important change to the design of the rsyslog core internal | |
| error message system. Previous code could deadlock if internal messages were | |
| issued inside queue processing code, which effectively limited error-reporting | |
| capabilities. This change makes it possible to call error messages from any | |
| part of the code at any time. | |
| As a side-effect, this also fixes an issue where rsyslog could deadlock if | |
| imuxsock submited messages to the system log socket when that socket blocked. | |
| This was a rare race, albeit consistently reproducible and also seen in | |
| practice. The work-around for this issue was to set | |
| global(processInternalMessages="on") | |
| in rsyslog.conf. With the new code, this race can never happen again. The new | |
| code also sets stage for emitting better error messages, especially in places | |
| where we previously refrained from doing so and messages went only to the | |
| debug log. For some file output and queue subsytem related messages, this | |
| is already done, but there is still further work required. | |
| Note well: this is a redesign of an important core component. While intensely | |
| tested, this may still have some larger regeression potential than usual code | |
| changes. | |
| - core: added logging name of source of rate-limited messages | |
| This adds the name to the rate-limiting message itself, making it easier | |
| to identify the actual source of "spam" messages. | |
| Thanks to github user jvymazal for the patch. | |
| - omfwd: omfwd: add support for network namespaces | |
| Thanks to Bastian Stender for the patch. | |
| - imrelp: honor input name if provided when submitting to impstats | |
| Thanks to Jon Henry for the patch. | |
| - imptcp: add ability to set owner/group/access perms for uds | |
| Thanks to Nathan Brown for implementing this feature. | |
| - mmnormalize: add ability to load a rulebase from action() parameter | |
| This is especially useful for small rulebases, as it avoids having | |
| a separate rulebase file. | |
| closes https://github.com/rsyslog/rsyslog/issues/625 | |
| - pmrfc3164 improvements | |
| - permit to ignore leading space in MSG | |
| - permit to use at-sign in host names | |
| - permit to require tag to end in colon | |
| Thanks to github user bdolez for the contribution | |
| - add new global parameter "umask" | |
| This is equivalent to "$umask" and permits to convert that construct | |
| to new-style config format. | |
| closes https://github.com/rsyslog/rsyslog/issues/1382 | |
| - core: make use of -T command line option more secure | |
| When the -T option is used, a chdir is now done right after chroot. It must | |
| be noted, though, that -T is a testing option and has no intent to provide | |
| real security. So this change does not mean it actually is sufficiently | |
| secure. | |
| Thanks to github user jvymazal for the patch. | |
| - omfile: add error if both file and dynafile are set | |
| - bugfix: build problem on MacOS (not a supported platform) | |
| Thanks to FX Coudert for the fix. | |
| - regression fix: in 8.25, str2num() returned error on empty string | |
| past behaviour was to return 0 in this case; past behavior was reinstanciated | |
| Thanks to github user jvymazal for the patch. | |
| - bugfix omsnmp: improper handling of new-style configuration parameters | |
| Thanks to Radovan Sroka for the patch. | |
| - bugfix: rsyslog identifies itself as "liblogging-stdlog" in internal messages | |
| This occured when liblogging-stdlog was used, and was used by default (without | |
| explicit configuration). This is a regression of the new default, which does | |
| not correctly call stdlog_open() in the default case. | |
| closes https://github.com/rsyslog/rsyslog/issues/1442 | |
| - bugfix imfile: wrong files were monitored when using multiple imfile inputs | |
| The bug was introduced by the changes for the multilevel wildcard feature | |
| in 8.25.0. We have to handle FileCreate events different if the directory | |
| watch is configured or added dynamically. | |
| closes https://github.com/rsyslog/rsyslog/issues/1452 | |
| - bugfix: setting net.aclResolveHostname/net.acladdhostnameonfail segfaults | |
| When compiling using hardned gcc (gentoo), specifying net.aclResolveHostname | |
| or net.acladdhostnameonfail results in rsyslogd segfaulting on startup. | |
| Thanks to Radovan Sroka for the patch. | |
| - bugfix: immark emitted error messages with text "imuxsock" | |
| Thanks to Chris Pilkington for the patch. | |
| - bugfix tcpflood: build failed if RELP was disabled | |
| - fix gcc6 compiler warnings | |
| This also fixes a small bug with incorrectly working deprecated -u | |
| command line option. | |
| Thanks to Nikola Pajkovsky for the patch. | |
| - the output module array passing interface has been removed | |
| It wasn't functional since the v8 update, and the only user was omoracle, | |
| which is a contributed module that is no longer maintained. So we | |
| removed that interface to streamline the code. Should it ever be needed | |
| again, we could check the 8.25 code base. Note, though, that that code | |
| still needs to be adjusted to the v8 engine. | |
| - testbench: | |
| * tcpflood now automatically enters silent mode during Travis CI testing | |
| This reduces testbench output, which is limited under Travis. | |
| * the libqpid-proton package is no longer available for Ubuntu trusty. As | |
| such, we disabled its use in Travis on this platform. Right now, this | |
| means omaqp1 module is no longer tested on trusty. | |
| ------------------------------------------------------------------------------ | |
| Version 8.25.0 [v8-stable] 2017-02-21 | |
| - imfile: add support for wildcards in directory names | |
| This now permits to monitor newly created directories without altering | |
| the configuration. | |
| - add new global option "parser.PermitSlashInProgramname" | |
| - mmdblookup: fix build issues, code cleanup | |
| Thanks to Dan Molik for the patch. | |
| - improved debug output for queue corruption cases | |
| - an error message is now displayed when a directory owner cannot be set | |
| This mostly happens with omfile and dynafils. The new messages | |
| facilitates troubleshooting. | |
| - rainerscript: | |
| * add new function ipv42num | |
| * add new function num2ipv4 | |
| - bugfix: ratelimiter does not work correctly is time is set back | |
| Thanks to github user hese10 for the patch. | |
| see also https://github.com/rsyslog/rsyslog/issues/1354 | |
| - core: fix potential message loss in old-style transactional interface | |
| This was experienced for example by omrelp. Could loose one message per | |
| broken connection, iff that message did not make it to the unacked list. | |
| - bugfix queue subsystem: queue corrupted if certain msg props are used | |
| The core issues was in the msg object deserializer, which had the wrong | |
| deserialization sequence. That in turn lead to queue corruption issues. | |
| Corruption of disk queue (or disk part of DA queue) always happens if | |
| the "json" property (message variables) is present and "structured-data" | |
| property is also present. This causes rsyslog to serialize to the | |
| queue in wrong property sequence, which will lead to error -2308 on | |
| deserialization. | |
| Seems to be a long-standing bug. Depending on version used, some or | |
| all messages in disk queue may be lost. | |
| closes https://github.com/rsyslog/rsyslog/issues/1404 | |
| - bugfix imjournal: fixed situation when time goes backwards | |
| This is resolving the situation when system is after reboot and | |
| boot_id doesn't match so cursor pointing into "future". | |
| Usually sd_journal_next jump to head of journal due to journal | |
| aproximation, but when system time goes backwards and cursor is | |
| still invalid, rsyslog stops logging. | |
| We use sd_journal_get_cursor to validate our cursor. | |
| When cursor is invalid we are trying to jump to the head of journal | |
| This problem with time should not affect persistent journal, | |
| but if cursor has been intentionally compromised it could stop | |
| logging even with persistent journal. | |
| - bugfix: bFlushOnTxEnd == 0 not honored when writing w/o async writer | |
| If bFlushOnTXEnd is set, we need to flush on transaction end - in | |
| any case. It is not relevant if this is using background writes | |
| (which then become pretty slow) or not. And, similarly, no flush | |
| happens when it is not set. | |
| see also https://github.com/rsyslog/rsyslog/issues/1297 | |
| - bugfix core: str2num mishandling empty strings | |
| If str2num() receives an empty string, misadressing happens. | |
| This theoretically can lead to a segfault if a RainerScript function | |
| is used inside the configuration which calls code that could trigger | |
| this bug. | |
| closes https://github.com/rsyslog/rsyslog/issues/1412 | |
| - bugfix rainerscript: set/unset statement do not check variable name validity | |
| Only JSON-based variables can be use with set and unset. Unfortunately, | |
| this restriction is not checked. If an invalid variable is given | |
| (e.g. $invalid), this is not detected upon config processing on | |
| startup. During execution phase, this can lead to a segfault, a | |
| memory leak or other types of problems. | |
| Thanks to github user mostolog for reporting and helping to analyze | |
| this issue. | |
| see also https://github.com/rsyslog/rsyslog/issues/1376 | |
| closes https://github.com/rsyslog/rsyslog/issues/1377 | |
| - bugfix mmrm1stspace: last character of rawmsg property was doubled | |
| - bugfix: rsyslog loops on freebsd when trying to write /dev/console | |
| Rsyslog 8.23.0 loops on FreeBSD when trying to access a (now revoked) | |
| /dev/console file descriptor, as per Alexandre's original bug report [1]. | |
| The original patch fixes the problem when tryTTYRecover() sees errno 6 ENXIO. | |
| Running FreeBSD 10-stable here and getting errno 5 EIO, same as Xavier gets | |
| in his 2016 bug report [2]. | |
| New patch [3] includes errno 5 to tryTTYRecover() in runtime/stream.c and | |
| fixes the problem for me, on multiple machines. | |
| [1] https://github.com/rsyslog/rsyslog/issues/371 | |
| [2] https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=211033 | |
| [3] https://bz-attachments.freebsd.org/attachment.cgi?id=178452 | |
| closes https://github.com/rsyslog/rsyslog/issues/1351 | |
| Thanks to Damien Fleuriot for the patch. | |
| - bugfix imtcp: fix very small (cosmetic) memory leak | |
| For each listener, the name of an assigned ruleset is not freed. This | |
| is cosmetic, because it is a very small static leak AND it needs to | |
| be kept until end of run anyways (and at end of run the OS frees it). | |
| However, the leak breaks memleak checks in the testbench. | |
| - fix build issues on some platforms (detected on newer Fedora) | |
| ------------------------------------------------------------------------------ | |
| Version 8.24.0 [v8-stable] 2017-01-10 | |
| - rsyslog now builds on AIX | |
| see also: https://github.com/rsyslog/rsyslog/pull/1247 | |
| Thanks to github user purnimam1 and the team at IBM | |
| Note: the rsyslog project has no AIX platform to ensure that future versions | |
| will remain buildable on AIX. If you would like to contribute ressources, | |
| please contact the rsyslog team. | |
| - mmdblookup: new maxminddb lookup message modify plugin | |
| Thanks to 饶琛琳 (github user chenryn) for the contribution | |
| - mmrm1stspace: new module; removes first space in MSG if present | |
| - KSI signature provider: file permissions can now be specified | |
| This happens via parameters equal to those used by omfile itself. | |
| Note that KSI files can have different permissions/owner than the log | |
| files themself. | |
| Thanks to Allan Park for the patch. | |
| - omzmq: new features | |
| Thanks to Brian Knox for the patch. | |
| - change: when the hostname is empty, we now use "localhost-empty-hostname" | |
| In 8.23.0, "localhost" was used in this case, but that could be misleading. | |
| The new name makes the error condition (gethostname() should always return | |
| a non-empty name) more obvious. | |
| - omelasticsearch: remove "asyncrepl" config parameter | |
| The _bulk index endpoint on ElasticSearch 5.0 no longer | |
| ignores the ?replication=async query parameter. It was deprecated | |
| since 1.5 and silently ignored in 2.x but passing it to a 5.x | |
| instance breaks omeleastisearch with a 400 response. | |
| closes https://github.com/rsyslog/rsyslog/issues/1250 | |
| - omfwd: Add support for bind-to-device (see below on same for imudp) | |
| - imudp: Add support for bind-to-device | |
| Add support for bind-to-device option to omfwd and imudp modules. | |
| Configured using device="name". Only new syntax format is supported. | |
| e.g., | |
| input(type="imudp" port=["10514"] device="eth0" name="udp") | |
| Action(type="omfwd" Target="192.168.1.23" Port="10514" Device="eth0") | |
| see also https://github.com/rsyslog/rsyslog/pull/1261 | |
| Thanks to David Ahern for the patch. | |
| - imudp: limit rcvbufsize parameter to max 1GiB | |
| - rainerscript: implement new "call_indirect" statement | |
| - bugfix imjournal: make state file handling more robust | |
| There is a bug in rsyslog which is caused by not very atomic writes of | |
| imjournal statefile. It's hardly reproducible but there is a way. | |
| fscanf error appears only when rsyslog reads an empty statefile which | |
| causes that imjournal is stopped so no logging from journal is | |
| performed. When the statefile contains random bytes error appears | |
| again but from journal and imjournal is stopped too. | |
| In this patch Rsyslog writes imjournal statefile more atomically and | |
| secure. Reading the statefile is more robust and doesn't affect | |
| imjournal module so when corrupted statefile is read imjournal | |
| ignores statefile, continues with logging and it doesn't stop. Logger | |
| can be used as a test if it's logging or not. | |
| Patch introduces a new option with both old and new config format | |
| "IgnoreNonValidStateFile" which is "on" by default and it can turn | |
| off ignorance of non valid statefile. | |
| Thanks to github user tomassykora for the patch. | |
| - bugfix core: lookup table reload was not properly integrated | |
| The opcode was not handled, what lead to misleading messages | |
| in debug log. Since we run debug builds more strictly, it also | |
| causes an assertion to trigger, thus aborting the test | |
| - bugfix core: potential dealock on shutdown | |
| could happen when rsyslog was started and quickly shut down OR when | |
| co-incidently a new thread was spawend "with bad timing" around the time | |
| of shutdown. | |
| See also https://github.com/rsyslog/rsyslog/pull/1274 | |
| Thanks to github user tomassykora for the final patch and Rado Sroka for | |
| the analysis and an initial patch. | |
| - bugfix ommongodb: did not work in v8 due to invalid indirection | |
| Thanks to Benoit Dolez for the patch. | |
| - bugfix ommongodb: fix tryResume handling | |
| To make tryResume working, the connection to mongodb need to be closed. | |
| Thus close it on "insert error". | |
| Thanks to Benoit Dolez for the patch. | |
| - bugfix omfwd: retry processing was not done correctly, could stall | |
| see also https://github.com/rsyslog/rsyslog/pull/1261 | |
| Thanks to David Ahern for the patch. | |
| - bugfix imuxsock: segfault non shutdown when $OmitLocalLogging is on | |
| Imuxsock tries to close socket on index 0 which ends with segfault. | |
| Thanks to Tomas Sykora for the patch. | |
| - testbench: | |
| * empty-hostname test did not work correctly | |
| * improve debugging by better output | |
| ------------------------------------------------------------------------------ | |
| Version 8.23.0 [v8-stable] 2016-11-15 | |
| - NEW BUILD REQUIREMENT: libfastjson 0.99.3 | |
| This was introduced in 8.20.0 as a suggestion and has now become a hard | |
| requirement. See 8.20.0 ChangeLog entry for more details. | |
| - KSI signatures: removed SHA2-224 hash algorithm | |
| This is considered insecure and no longer supported by the underlying | |
| KSI library. If still used within a configuration, a descriptive error | |
| message is emitted during config processing. | |
| Thanks to Henri Lakk for the initial patch. | |
| - imfile: new timeout feature for multi-line reads | |
| When using startmsg.regex, messages are held until the next one is written. | |
| We now provide a "readTimeout" parameter family (see doc) to timeout such | |
| reads and ensure messages are not held for a very long time. | |
| see also https://github.com/rsyslog/rsyslog/issues/1133 | |
| - omfile: improve robustness against network file system failures | |
| in case of failure, a close and re-open is tried, which often solves the | |
| issue (and wasn't handle before this patch). | |
| see also https://github.com/rsyslog/rsyslog/pull/1161 | |
| Thanks to github user hese10 for the patch. | |
| - pmaixforwardedfrom: support for AIX syslogd -s option | |
| if syslog in AIX started with "-s" option, AIX syslog adds only "From " | |
| instead of "Message forwarded from ". With this patch, both are now | |
| detected. | |
| Thanks to github user patelritesh for the patch. | |
| - omelasticsearch: add ability to specify max http request size | |
| This permits to keep batches below ES-configured limits. | |
| Thanks to github user barakshechter for the patch. | |
| - omelasticsearch: high availability addressing of ElasticSearch cluster | |
| allow to specify an array of servers, which is tried until a working | |
| one is found (and given up only if none works). | |
| Thanks to github user barakshechter for the patch. | |
| - omelasticsearch: make compatible with ElasticSearch 2.x and 5.x | |
| fixes omelasticsearch logs response from ElasticSearch 5.0 _bulk | |
| endpoint as error | |
| See also https://github.com/rsyslog/rsyslog/pull/1253 | |
| Thanks to Christian Rodemeyer for the patch. | |
| - omhiredis: add dynakey attribute. | |
| If set to on, this causes omhiredis to treat the key attribute as the | |
| name of a template so that you can build a dynamic redis queue name | |
| or list key. | |
| see also: https://github.com/rsyslog/rsyslog/pull/1218 | |
| Thanks to github user bobthemighty for the patch | |
| - omtcl: new contributed module | |
| see also https://github.com/rsyslog/rsyslog/pull/1041 | |
| Please note: contributed modules are not directly supported by the | |
| project. You might want to contact the author for questions. | |
| Thanks to Francisco Castro for contributing it. | |
| - RainerScript: provide a capability to set environment variables | |
| via 'global(environment="var=val")' config statement. | |
| This is most importantly for things like HTTP_PROXY. | |
| see also https://github.com/rsyslog/rsyslog/issues/1209 | |
| - lookup tables: improved error checking | |
| Thanks to Janmejay Singh for the patch. | |
| - queue subsystem: add configuration parameter "queue.samplinginterval" | |
| Supports sampling of messages (as often used in data transmission). | |
| Thanks to Zachary M. Zhao for the patch. | |
| - bugfix core: errmsg.LogError now switches to dfltErrLogger just before shutdown | |
| Thanks to Janmejay Singh for the patches. | |
| - bugfix core: fixed un-freed memory in non-transactional action using string-passing | |
| closes https://github.com/rsyslog/rsyslog/issues/968 | |
| Thanks to Janmejay Singh for the patches. | |
| - rsgtutil: option to specify KSI publications file certificate constraints | |
| see also https://github.com/rsyslog/rsyslog/issues/1207 | |
| - omprog: bugfixes and enhancements | |
| - omprog resource leak fix (fd leak) | |
| - omprog got ability to force-kill process if it doesn't die in 5 seconds | |
| (linux specific) | |
| - child-process lifecycle debugging aid (in form of logs) (mainLoop and | |
| omprog cleanup both log pid at child-death, mainLoop reaping is now | |
| visible to user, as opposed to being a mystery, because omprog didn't | |
| seem to anticipate it in terms of code) | |
| Thanks to Janmejay Singh for the patches. | |
| see also https://github.com/rsyslog/rsyslog/pull/1155 | |
| - bugfix imfile: ReopenOnTruncate processing, file open processing | |
| This fixes | |
| * ReopenOnTrucate was only honored when a state file existed | |
| see https://github.com/rsyslog/rsyslog/issues/1090 | |
| * open processing could run into a loop | |
| see https://github.com/rsyslog/rsyslog/issues/1174 | |
| This is done via refactoring the open file handling, which provides | |
| overall cleaner and easier-to-follow code. | |
| Thanks to Owen Smith for analyzing the problem and providing a | |
| prototype PR which greatly helped towards the final solution. | |
| - bugfix omlibdbi: libdbi-driver-sqlite3/2 requires to provide a path to | |
| database split into two strings: | |
| * absolute path, where the database file sits | |
| * database filename itself. | |
| This was previously not done. | |
| Thanks to github user aleek for the patch. | |
| - bugfix RainerScript: issue in prifilt() function | |
| Initialize func-data(and to-be-freed flag) correctly for prifilt | |
| function-node | |
| Thanks to Janmejay Singh for the patch. | |
| - bugfix omrelp: invalid module name imrelp was used in some error messages | |
| Thanks to Chris Pilkington for the patch. | |
| - bugfix core: abort when persisting object state | |
| This causes a segfault. It happens whenever an object state larger | |
| than 4095 byte is persisted. Then, incorrectly a try to rollover to | |
| a new state file is tried, which will lead to a division by zero | |
| as the necessary variables for this operation are not set because we | |
| are NOT in circular mode. | |
| This problem can happen wherever state files are written. It has been | |
| experienced with imfile and queue files. | |
| Many thanks to github user mostolog for his help in reproducing the issue, | |
| which was very important to finally nail down this long-standing bug. | |
| closes https://github.com/rsyslog/rsyslog/issues/1239 | |
| closes https://github.com/rsyslog/rsyslog/issues/1162 | |
| closes https://github.com/rsyslog/rsyslog/issues/1074 | |
| - bugfix: segfault if hostname is unset on system | |
| happens when gethostname() returns empty string. This will cause | |
| the createon of the localhostname prop_t to fail, which in turn | |
| leads to a NULL pointer dereference when emitting local messages. | |
| As we emit a startup message by default, this had usually lead | |
| to a segfault on startup. | |
| Thanks to Erik Potter and github user mostolog for their help | |
| in analyzing this problem. | |
| closes https://github.com/rsyslog/rsyslog/issues/1040 | |
| closes https://github.com/rsyslog/rsyslog/issues/335 | |
| - bugfix external module perl skeleton: did not work properly | |
| Thanks to github user Igorjan666 for the patch. | |
| - bugfix build system: Fix detection of pthread_setschedparam() on platforms | |
| such as FreeBSD | |
| see also https://github.com/rsyslog/rsyslog/pull/1147 | |
| Thanks to Matthew Seaman for the patch. | |
| - bugfix omelasticsearch: modifies constant memory under some circumstances | |
| Function computeBaseUrl may modify its serverParam parameter, but | |
| this may contain the constant string "localhost". Depending on the | |
| platform, this can lead to a segfault. | |
| Noticed while working on compiler warnings, not seen in practice. | |
| - "bugfix": theoretical queue file corruption when more than MAX_INT files | |
| closes https://github.com/rsyslog/rsyslog/issues/1202 | |
| - bug fix/KSI: LOGSIG11 missing in the beginning of KSI log signature file | |
| When logging with KSI is not working properly for whatever reason, an | |
| empty .ksisig file is created (which by itself is not an issue). However, | |
| later it looks like this file is re-used, but it is not checked whether it | |
| already contains the magic LOGSIG11 in the beginning of the file. This leads | |
| to a log signature file which has correct content but is missing the | |
| LOGSIG11 magic in the beginning. | |
| - bugfix template processor: missing escaping of backslash in json mode | |
| Thanks to github user mostolog for providing the patch. | |
| - build environment: --enable-debug now defaults to new "auto" mode | |
| previously, DEBUG mode (and thus assert() macros) was disabled by default | |
| and explicitly needed to be enabled by providing the --enable-debug | |
| ./configure switch. Now, a new --enable-debug=auto mode has been added | |
| and made the default. It enables DEBUG mode if we build from git and only | |
| disables it when a release build is done (from the release tarball). This | |
| aims at better error checking during testbench runs and developer testing. | |
| - testbench improvements | |
| * improved testbench file generation tool | |
| Thanks to Pascal Withopf for the patch. | |
| * added some plumbing for extended tests which work by overriding OS APIs | |
| * imfile ReopenOnTruncate option is now being tested | |
| * the CI environment now runs most tests in debug mode, but some in | |
| release mode to cover potential release-mode races | |
| * template json option is now being tested | |
| * object state persisting received a basic test | |
| * added test for empty hostnames | |
| * added tests for omprog | |
| ------------------------------------------------------------------------------ | |
| Version 8.22.0 [v8-stable] 2016-10-04 | |
| - ompgsql: add template support | |
| Thanks to Radu Gheorghe for implementing this. | |
| - generate somewhat better error message on config file syntax error | |
| a common case (object at invalid location) has received it's own error | |
| message; for the rest we still rely on the generic flex/bison handler | |
| - bugfix:omhiredis reconnects after failure | |
| previously it could loose messages under such conditions. | |
| Thanks to Bob Gregory for the patch. | |
| - general cleanup and code improvement | |
| mostly guided by compiler warnings induced by newer opensuse builbot | |
| environment | |
| ------------------------------------------------------------------------------ | |
| Version 8.21.0 [v8-stable] 2016-08-23 | |
| - CHANGE OF BEHAVIOUR: | |
| by default, internal messages are no longer logged via the internal | |
| bridge to rsyslog but via the syslog() API call [either directly or | |
| via liblogging). For the typical single-rsyslogd-instance installation this | |
| is mostly unnoticable (except for some additional latency). If multiple | |
| instances are run, only the "main" (the one processing system log messages) | |
| will see all messages. To return to the old behaviour, do either of those | |
| two: | |
| 1) add in rsyslog.conf: | |
| global(processInternalMessages="on") | |
| 2) export the environment variable RSYSLOG_DFLT_LOG_INTERNAL=1 | |
| This will set a new default - the value can still be overwritten via | |
| rsyslog.conf (method 1). Note that the environment variable must be | |
| set in your **startup script**. | |
| For more information, please visit | |
| http://www.rsyslog.com/rsyslog-error-reporting-improved/ | |
| - slightly improved TLS syslog error messages | |
| - queue subsystem: improved robustness | |
| The .qi file is now persisted whenever an existing queue file is fully | |
| written and a new file is begun. This helps with rsyslog aborts, including | |
| the common case where the OS issues kill -9 because of insufficiently | |
| configured termination timout (this is an OS config error, but a frequent | |
| one). Also, a situation where an orphaned empty file could be left in the | |
| queue work directory has been fixed. We expect that this change causes | |
| fewer permanent queue failures. | |
| - bugfix: build failed on some platforms due to missing include files | |
| ------------------------------------------------------------------------------ | |
| Version 8.20.0 [v8-stable] 2016-07-12 | |
| - NEW BUILD REQUIREMENT: librelp, was 1.2.5, now is 1.2.12 | |
| This is only needed if --enable-relp is used. The new version is needed | |
| to support the new timeout parameter in omrelp. | |
| - NEW BUILD SUGGESTION: libfastjson 0.99.3 | |
| while not strictly necessary, previous versions of libfastjson have a bug | |
| in unicode processing that can result in non US-ASCII characters to be | |
| improperly encoded and may (very unlikely) also cause a segfault. | |
| This version will become mandatory in rsyslog 8.23.0 | |
| - omrelp: add configurable connection timeout | |
| Thanks to Nathan Brown for implementing this feature. | |
| - pmrfc3164: add support for slashes in hostname | |
| added parameter "permit.slashesinhostname" to support this, off by default | |
| [Note that the RFC5424 always supported this, as 5424 is a different | |
| standard] | |
| - bugfix omfile: handle chown() failure correctly | |
| If the file creation succeeds, but chown() failed, the file was | |
| still writen, even if the user requested that this should be treated | |
| as a failure case. This is corrected now. | |
| Also, some refactoring was done to create better error messages. | |
| - omfile now better conveys status of unwritable files back to core | |
| - config files recursively including themselfes are now detected | |
| and an error message is emitted in that case; Previously, this | |
| misconfiguration resulted in rsyslog loop and abort during startup. | |
| closes https://github.com/rsyslog/rsyslog/issues/1058 | |
| - refactored code to not emit compiler warnings in "strict mode" | |
| We changed the compiler warning settings to be rather strict and cleaned up | |
| the code to work without generating any warning messages. | |
| This results in an overall even more improved code quality, which will now | |
| also be enforced via our CI systems. | |
| - bugfix: fix some issues with action CommitTransaction() handling | |
| An action that returns an error from CommitTransaction() caused a | |
| loop in rsyslog action processing. Similarly, retry processing was not | |
| properly handled in regard to CommitTransaction(). | |
| This is a first shot at fixing the situation. It solves the | |
| immediate problems, but does not implement the full desired | |
| functionality (like error file). | |
| see also https://github.com/rsyslog/rsyslog/issues/974 | |
| see also https://github.com/rsyslog/rsyslog/issues/500 | |
| - bugfix omqmqp1: connecting to the message bus fails on nonstandard port | |
| Thanks to Ken Giusti for the patch. | |
| see also: https://github.com/rsyslog/rsyslog/pull/1064 | |
| - testbench/CI enhancements | |
| * new tests for RELP components | |
| * new tests for core action processing and retry | |
| * travis tests now also run against all unstable versions of supporting | |
| libraries. This helps to track interdependency problems early. | |
| * new tests for hostname parsing | |
| * new tests for RainerScript comparisons | |
| ------------------------------------------------------------------------------ | |
| Version 8.19.0 [v8-stable] 2016-05-31 | |
| - NEW BUILD REQUIREMENT: autoconf-archive | |
| - omelasticsearch: add option to permit unsigned certs (experimentally) | |
| This adds plumbing as suggested by Joerg Heinemann and Radu Gheorghe, | |
| but is otherwise untested. Chances are good it works. If you use it, | |
| please let us know your experience and most importantly any bug | |
| reports you may have. | |
| closes https://github.com/rsyslog/rsyslog/issues/89 | |
| - imrelp: better error codes on unvailablity of TLS options | |
| Most importantly, we will tell the user in clear words if specific TLS | |
| options are not available due to too-old GnuTLS. | |
| closes https://github.com/rsyslog/rsyslog/issues/1019 | |
| - default stack size for inputs has been explicitely set to 4MiB | |
| for most platforms, this means a reduction from the default of 10MiB, hower | |
| it may mean an increas for micro-libc's (some may have as low as 80KiB by | |
| default). | |
| - testbench: We are now using libfaketime instead of faketime command line | |
| tool. Make sure you have installed the library and not just the binary! | |
| - refactor stringbuf | |
| * use only a single string buffer | |
| ... both for the internal representation as well as the C-String one. | |
| The module originally tried to support embedded NUL characters, which | |
| over time has prooven to be not necessary. Rsyslog always encodes | |
| NUL into escape sequences. | |
| Also, the dual buffers were used inconsistently, which could lead to | |
| subtle bugs. With the single buffer, this does no longer happen and | |
| we also get some improved performance (should be noticable) | |
| and reduced memory use (a bit). | |
| closes https://github.com/rsyslog/rsyslog/issues/1033 | |
| * removed no longer used code | |
| * internal API changes to reflect new needs | |
| * performance improvements | |
| * miscellaneous minor cleanup | |
| - fix: potential misadressing in template config processing | |
| This could cause segfault on startup. Happens when template name shorter | |
| than two chars and outname is not set. Once we are over startup, things | |
| work reliably. | |
| - bugfix omfile: async output file writing does not respect flushing | |
| neither parameter flushInterval nor flushOnTXEnd="on" was respected. | |
| closes https://github.com/rsyslog/rsyslog/issues/1054 | |
| - bugfix imfile: corrupted multi-line message when state data was persisted | |
| see also https://github.com/rsyslog/rsyslog/issues/874 | |
| Thanks to Magnus Hyllander for the analysis and a patch suggestion. | |
| - bugfix imfile: missing newline after first line of multiline message | |
| see also https://github.com/rsyslog/rsyslog/issues/843 | |
| Thanks to Magnus Hyllander for the patch. | |
| - bugfix: dynstats unusedMetricTtl bug | |
| Thanks to Janmejay Singh for fixing this. | |
| - bugfix build system: build was broken on SunOS | |
| Thanks to Filip Hajny for the patch. | |
| - bugfix: afterRun entry point not correctly called | |
| The entry point was called at the wrong spot, only when the thread | |
| had not already terminated by itself. This could cause various | |
| cleanup to not be done. This affected e.g. imjournal. | |
| closes https://github.com/rsyslog/rsyslog/issues/882 | |
| - bugfix dynstats: do not leak file handles | |
| Thanks to Janmejay Singh for the patch. | |
| - bugfix omelasticsearch: disable libCURL signal handling | |
| previously, this could lead to segfaults on connection timeout | |
| see also https://github.com/rsyslog/rsyslog/pull/1007 | |
| Thanks to Sai Ke WANG for the patch. | |
| - bugfix omelasticsearc: some regressions were fixed | |
| * error file was no longer written | |
| * fix for some potential misaddressings | |
| - improved wording: gnutls error message points to potential cause | |
| What GnutTLS returns us is very unspecific and somehwat misleading, so | |
| we point to what it most probably is (broken connect). | |
| see also https://github.com/rsyslog/rsyslog/issues/846 | |
| - some general code improvements | |
| * "fixed" cosmetic memory leaks at shutdown | |
| - build system bugfix: configure can't find gss_acquire_cred on Solaris | |
| Thanks to github user vlmarek for the patch. | |
| - improvements to the CI environment | |
| * improvements on the non-raciness of some tests | |
| * imdiag: avoid races in detecting queue empty status | |
| This reslolves cases where the testbench terminated rsyslog too early, | |
| resulting in potential message loss and test failure. | |
| * omkafka has now dynamic tests | |
| Thanks to Janmejay Singh for implementing them. | |
| * try to merge PR to master and run tests; this guards against cross-PR | |
| regressions and wasn't caught previously. Note that we skip this test | |
| if we cannot successfully merge. So this is not a replacement for a | |
| daily full "all-project integration test run". | |
| * travis has finally enabled elasticsearch tests | |
| ES was unfortunately not being regularly tested for quite a while due to | |
| missing environment. This lead to some regressions becoming undetected. | |
| These were now discovered thanks to the new support on travis. Also, this | |
| guards against future regressions. | |
| * imfile has now additional tests and overall better coverage | |
| * omfile has now additional tests | |
| ------------------------------------------------------------------------------ | |
| Version 8.18.0 [v8-stable] 2016-04-19 | |
| - testbench: When running privdrop tests testbench tries to drop | |
| user to "rsyslog", "syslog" or "daemon" when running as root and | |
| you don't explict set RSYSLOG_TESTUSER environment variable. | |
| Make sure the unprivileged testuser can write into tests/ dir! | |
| - templates: add option to convert timestamps to UTC | |
| closes https://github.com/rsyslog/rsyslog/issues/730 | |
| - omjournal: fix segfault (regression in 8.17.0) | |
| - imptcp: added AF_UNIX support | |
| Thanks to Nathan Brown for implementing this feature. | |
| - new template options | |
| * compressSpace | |
| * date-utc | |
| - redis: support for authentication | |
| Thanks to Manohar Ht for the patch | |
| - omkafka: makes kafka-producer on-HUP restart optional | |
| As of now, omkafka kills and re-creates kafka-producer on HUP. This | |
| is not always desirable. This change introduces an action param | |
| (reopenOnHup="on|off") which allows user to control re-cycling of | |
| kafka-producer. | |
| It defaults to on (for backward compatibility). Off allows user to | |
| ignore HUP as far as kafka-producer is concerned. | |
| Thanks to Janmejay Singh for implementing this feature | |
| - imfile: new "FreshStartTail" input parameter | |
| Thanks to Curu Wong for implementing this. | |
| - omjournal: fix libfastjson API issues | |
| This module accessed private data members of libfastjson | |
| - ommongodb: fix json API issues | |
| This module accessed private data members of libfastjson | |
| - testbench improvements (more tests and more thourough tests) | |
| among others: | |
| - tests for omjournal added | |
| - tests for KSI subsystem | |
| - tests for priviledge drop statements | |
| - basic test for RELP with TLS | |
| - some previously disabled tests have been re-enabled | |
| - dynamic stats subsystem: a couple of smaller changes | |
| they also involve the format, which is slightly incompatible to | |
| previous version. As this was out only very recently (last version), | |
| we considered this as acceptable. | |
| Thanks to Janmejay Singh for developing this. | |
| - foreach loop: now also iterates over objects (not just arrays) | |
| Thanks to Janmejay Singh for developing this. | |
| - improvements to the CI environment | |
| - enhancement: queue subsystem is more robst in regard to some corruptions | |
| It is now detected if a .qi file states that the queue contains more | |
| records than there are actually inside the queue files. Previously this | |
| resulted in an emergency switch to direct mode, now the problem is only | |
| reported but processing continues. | |
| - enhancement: Allow rsyslog to bind UDP ports even w/out specific | |
| interface being up at the moment. | |
| Alternatively, rsyslog could be ordered after networking, however, | |
| that might have some negative side effects. Also IP_FREEBIND is | |
| recommended by systemd documentation. | |
| Thanks to Nirmoy Das and Marius Tomaschewski for the patch. | |
| - cleanup: removed no longer needed json-c compatibility layer | |
| as we now always use libfastjson, we do not need to support old | |
| versions of json-c (libfastjson was based on the newest json-c | |
| version at the time of the fork, which is the newest in regard | |
| to the compatibility layer) | |
| - new External plugin for sending metrics to SPM Monitoring SaaS | |
| Thanks to Radu Gheorghe for the patch. | |
| - bugfix imfile: fix memory corruption bug when appending @cee | |
| Thanks to Brian Knox for the patch. | |
| - bugfix: memory misallocation if position.from and position.to is used | |
| a negative amount of memory is tried to be allocated if position.from | |
| is smaller than the buffer size (at least with json variables). This | |
| usually leads to a segfault. | |
| closes https://github.com/rsyslog/rsyslog/issues/915 | |
| - bugfix: fix potential memleak in TCP allowed sender definition | |
| depending on circumstances, a very small leak could happen on each | |
| HUP. This was caused by an invalid macro definition which did not rule | |
| out side effects. | |
| - bugfix: $PrivDropToGroupID actually did a name lookup | |
| ... instead of using the provided ID | |
| - bugfix: small memory leak in imfile | |
| Thanks to Tomas Heinrich for the patch. | |
| - bugfix: double free in jsonmesg template | |
| There has to be actual json data in the message (from mmjsonparse, | |
| mmnormalize, imjournal, ...) to trigger the crash. | |
| Thanks to Tomas Heinrich for the patch. | |
| - bugfix: incorrect formatting of stats when CEE/Json format is used | |
| This lead to ill-formed json being generated | |
| - bugfix omfwd: new-style keepalive action parameters did not work | |
| due to being inconsistently spelled inside the code. Note that legacy | |
| parameters $keepalive... always worked | |
| see also: https://github.com/rsyslog/rsyslog/issues/916 | |
| Thanks to Devin Christensen for alerting us and an analysis of the | |
| root cause. | |
| - bugfix: memory leaks in logctl utility | |
| Detected by clang static analyzer. Note that these leaks CAN happen in | |
| practice and may even be pretty large. This was probably never detected | |
| because the tool is not often used. | |
| - bugfix omrelp: fix segfault if no port action parameter was given | |
| closes https://github.com/rsyslog/rsyslog/issues/911 | |
| - bugfix imtcp: Messages not terminated by a NL were discarded | |
| ... upon connection termination. | |
| Thanks to Tomas Heinrich for the patch. | |
| ------------------------------------------------------------------------------ | |
| Version 8.17.0 [v8-stable] 2016-03-08 | |
| - NEW REQUIREMENT: libfastjson | |
| see also: | |
| http://blog.gerhards.net/2015/12/rsyslog-and-liblognorm-will-switch-to.html | |
| - new testbench requirement: faketime command line tool | |
| This is used to generate a controlled environment for time-based tests; if | |
| not available, tests will gracefully be skipped. | |
| - improve json variable performance | |
| We use libfastjson's alternative hash function, which has been | |
| proven to be much faster than the default one (which stems | |
| back to libjson-c). This should bring an overall performance | |
| improvement for all operations involving variable processing. | |
| closes https://github.com/rsyslog/rsyslog/issues/848 | |
| - new experimental feature: lookup table suport | |
| Note that at this time, this is an experimental feature which is not yet | |
| fully supported by the rsyslog team. It is introduced in order to gain | |
| more feedback and to make it available as early as possible because many | |
| people consider it useful. | |
| Thanks to Janmejay Singh for implementing this feature | |
| - new feature: dynamic statistics counters | |
| which may be changed during rule processing | |
| Thanks to Janmejay Singh for suggesting and implementing this feature | |
| - new contributed plugin: omampq1 for AMQP 1.0-compliant brokers | |
| Thanks to Ken Giusti for this module | |
| - new set of UTC-based $now family of variables ($now-utc, $year-utc, ...) | |
| - simplified locking when accessing message and local variables | |
| this simlifies the code and slightly increases performance if such | |
| variables are heavily accessed. | |
| - new global parameter "debug.unloadModules" | |
| This permits to disable unloading of modules, e.g. to make valgrind | |
| reports more useful (without a need to recompile). | |
| - timestamp handling: guard against invalid dates | |
| We do not permit dates outside of the year 1970..2100 | |
| interval. Note that network-receivers do already guard | |
| against this, so the new guard only guards against invalid | |
| system time. | |
| - imfile: add "trimlineoverbytes" input paramter | |
| Thanks to github user JindongChen for the patch. | |
| - ommongodb: add support for extended json format for dates | |
| Thanks to Florian Bücklers for the patch. | |
| - omjournal: add support for templates | |
| see also: https://github.com/rsyslog/rsyslog/pull/770 | |
| Thanks to github user bobthemighty for the patch | |
| - imuxsock: add "ruleset" input parameter | |
| - testbench: framework improvement: configs can be included in test file | |
| they do no longer need to be in a separate file, which saves a bit | |
| of work when working with them. This is supported for simple tests with | |
| a single running rsyslog instance | |
| Thanks to Janmejay Singh for inspiring me with a similar method in | |
| liblognorm testbench. | |
| - imptcp: performance improvements | |
| Thanks to Janmejay Singh for implementing this improvement | |
| - made build compile (almost) without warnings | |
| still some warnings are suppressed where this is currently required | |
| - improve interface definition in some modules, e.g. mmanon, mmsequence | |
| This is more an internal cleanup and should have no actual affect to | |
| the end user. | |
| - solaris build: MAXHOSTNAMELEN properly detected | |
| - build system improvement: ability to detect old hiredis libs | |
| This permits to automatically build omhiredis on systems where the | |
| hiredis libs do not provide a pkgconfig file. Previsouly, this | |
| required manual configuration. | |
| Thanks to github user jaymell for the patch. | |
| - rsgtutil: dump mode improvements | |
| * auto-detect signature file type | |
| * ability to dump hash chains for log extraction files | |
| - build system: fix build issues with clang | |
| clang builds often failed with a missing external symbol | |
| "rpl_malloc". This was caused by checks in configure.ac, | |
| which checked for specific GNU semantics. As we do not need | |
| them (we never ask malloc for zero bytes), we can safely | |
| remove the macros. | |
| Note that we routinely run clang static analyer in CI and | |
| it also detects such calls as invalid. | |
| closes https://github.com/rsyslog/rsyslog/issues/834 | |
| - bugfix: unixtimestamp date format was incorrectly computed | |
| The problem happened in leap year from March til then end | |
| of year and healed itself at the begining of the next year. | |
| During the problem period, the timestamp was 24 hours too low. | |
| fixes https://github.com/rsyslog/rsyslog/issues/830 | |
| - bugfix: date-ordinal date format was incorrectly computed | |
| same root cause aus for unixtimestamp and same triggering | |
| condition. During the affected perido, the ordinal was one | |
| too less. | |
| - bugfix: some race when shutting down input module threads | |
| this had little, if at all, effect on real deployments as it resulted | |
| in a small leak right before rsyslog termination. However, it caused | |
| trouble with the testbench (and other QA tools). | |
| Thanks to Peter Portante for the patch and both Peter and Janmejay | |
| Singh for helping to analyze what was going on. | |
| - bugfix tcpflood: did not handle connection drops correct in TLS case | |
| note that tcpflood is a testbench too. The bug caused some testbench | |
| instability, but had no effect on deplyments. | |
| - bugfix: abort if global parameter value was wrong | |
| If so, the abort happened during startup. Once started, | |
| all was stable. | |
| - bugfix omkafka: fix potential NULL pointer addressing | |
| this happened when the topic cache was full and an entry | |
| needed to be evicted | |
| - bugfix impstats: @cee cookie was prefixed to wrong fromat (json vs. cee) | |
| Thanks to Volker Fröhlich for the fix. | |
| - bugfix imfile: fix race during startup that could lead to some duplication | |
| If a to-be-monitored file was created after inotify was initialized | |
| but before startup was completed, the first chunk of data from this | |
| file could be duplicated. This should have happened very rarely in | |
| practice, but caused occasional testbench failures. | |
| see also: https://github.com/rsyslog/rsyslog/issues/791 | |
| - bugfix: potential loss of single message at queue shutdown | |
| see also: https://github.com/rsyslog/rsyslog/issues/262 | |
| - bugfix: potential deadlock with heavy variable access | |
| When making havy use of global, local and message variables, a deadlock | |
| could occur. While it is extremly unlikely to happen, we have at least | |
| seen one incarnation of this problem in practice. | |
| - bugfix ommysql: on some platforms, serverport parameter had no effect | |
| This was caused by an invalid code sequence which's outcome depends on | |
| compiler settings. | |
| - bugfix omelasticsearch: invalid pointer dereference | |
| The actual practical impact is not clear. This came up when working | |
| on compiler warnings. | |
| Thanks to David Lang for the patch. | |
| - bugfix omhiredis: serverport config parameter did not reliably work | |
| depended on environment/compiler used to build | |
| - bugfix rsgtutil: -h command line option did not work | |
| Thanks to Henri Lakk for the patch. | |
| - bugfix lexer: hex numbers were not properly represented | |
| see: https://github.com/rsyslog/rsyslog/pull/771 | |
| Thanks to Sam Hanes for the patch. | |
| - bugfix TLS syslog: intermittent errors while sending data | |
| Regression from commit 1394e0b. A symptom often seen was the message | |
| "unexpected GnuTLS error -50 in nsd_gtls.c:530" | |
| - bugfix imfile: abort on startup if no slash was present in file name param | |
| Thanks to Brian Knox for the patch. | |
| - bugfix rsgtutil: fixed abort when using short command line options | |
| Thanks to Henri Lakk | |
| - bugfix rsgtutil: invalid computation of log record extraction file | |
| This caused verification to fail because the hash chain was actually | |
| incorrect. Depended on the input data set. | |
| closes https://github.com/rsyslog/rsyslog/issues/832 | |
| - bugfix build system: KSI components could only be build if in default path | |
| ------------------------------------------------------------------------------ | |
| Version 8.16.0 [v8-stable] 2016-01-26 | |
| - rsgtutil: Added extraction support including loglines and hash chains. | |
| More details on how to extract loglines can be found in the rsgtutil | |
| manpage. See also: https://github.com/rsyslog/rsyslog/issues/561 | |
| - clean up doAction output module interface | |
| We started with char * pointers, but used different types of pointers | |
| over time. This lead to alignment warnings. In practice, I think this | |
| should never cause any problems (at least there have been no reports | |
| in the 7 or so years we do this), but it is not clean. The interface is | |
| now cleaned up. We do this in a way that does not require modifications | |
| to modules that just use string parameters. For those with message | |
| parameters, have a look at e.g. mmutf8fix to see how easy the | |
| required change is. | |
| - new system properties for $NOW properties based on UTC | |
| This permits to express current system time in UTC. | |
| See also https://github.com/rsyslog/rsyslog/issues/729 | |
| - impstats: support broken ElasticSearch JSON implementation | |
| ES 2.0 no longer supports valid JSON and disallows dots inside names. | |
| This adds a new "json-elasticsearch" format option which replaces | |
| those dots by the bang ("!") character. So "discarded.full" becomes | |
| "discarded!full". | |
| This is a workaroud. A method that will provide more control over | |
| replacements will be implemented some time in the future. For | |
| details, see below-quoted issue tracker. | |
| closes https://github.com/rsyslog/rsyslog/issues/713 | |
| - omelasticsearch: craft better URLs | |
| Elasticsearch is confused by url's ending in a bare '?' or '&'. While | |
| this is valid, those are no longer produced. | |
| Thanks to Benno Evers for the patch. | |
| - imfile: add experimental "reopenOnTruncate" parameter | |
| Thanks to Matthew Wang for the patch. | |
| - bugfix imfile: proper handling of inotify initialization failure | |
| Thanks to Zachary Zhao for the patch. | |
| - bugfix imfile: potential segfault due to improper handling of ev var | |
| This occurs in inotify mode, only. | |
| Thanks to Zachary Zhao and Peter Portante for the patch. | |
| closes https://github.com/rsyslog/rsyslog/issues/718 | |
| - bugfix imfile: potential segfault under heavey load. | |
| This occurs in inotify mode when using wildcards, only. | |
| The root cause is dropped IN_IGNOPRED inotify events which be dropped | |
| in circumstance of high input pressure and frequent rotation, and | |
| according to wikipeida, they can also be dropped in other conditions. | |
| Thanks to Zachary Zhao for the patch. | |
| closes https://github.com/rsyslog/rsyslog/issues/723 | |
| - bugfix ommail: invalid handling of server response | |
| if that response was split into different read calls. Could lead to | |
| error-termination of send operation. Problem is pretty unlikely to | |
| occur in standard setups (requires slow connection to SMTP server). | |
| Thank to github user haixingood for the patch. | |
| - bugfix omelasticsearch: custom serverport was ignored on some platforms | |
| Thanks to Benno Evers for the patch. | |
| - bugfix: tarball did not include some testbench files | |
| Thanks to Thomas D. (whissi) for the patch. | |
| - bugfix: memory misadressing during config parsing string template | |
| This occurred if an (invalid) template option larger than 63 characters | |
| was given. | |
| Thanks to git hub user c6226 for the patch. | |
| - bugfix imzmq: memory leak | |
| Thanks to Jeremy Liang for the patch. | |
| - bugfix imzmq: memory leak | |
| Thanks to github user xushengping for the patch. | |
| - bugfix omzmq: memory leak | |
| Thanks to Jack Lin for the patch. | |
| - some code improvement and cleanup | |
| ------------------------------------------------------------------------------ | |
| Version 8.15.0 [v8-stable] 2015-12-15 | |
| - KSI Lib: Updated code to run with libksi 3.4.0.5 | |
| Also libksi 3.4.0.x is required to build rsyslog if ksi support | |
| is enabled. New libpackages have been build as well. | |
| - KSI utilities: Added option to ser publication url. | |
| Since libksi 3.4.0.x, there is no default publication url anymore. | |
| The publication url has to be set using the --publications-server | |
| Parameter, otherwise the ksi signature cannot be verified. UserID | |
| and UserKey can also be set by parameter now. | |
| Closes https://github.com/rsyslog/rsyslog/issues/581 | |
| - KSI Lib: Fixed wrong TLV container for KSI signatures from 0905 to 0906. | |
| closes https://github.com/rsyslog/rsyslog/issues/587 | |
| - KSI/GT Lib: Fixed multiple issues found using static analyzer | |
| - performance improvement for configs with heavy use of JSON variables | |
| Depending on the config, this can be a very big gain in performance. | |
| - added pmpanngfw: contributed module for translating Palo Alto Networks logs. | |
| see also: https://github.com/rsyslog/rsyslog/pull/573 | |
| Thanks to Luigi Mori for the contribution. | |
| - testbench: Changed valgrind option for imtcp-tls-basic-vg.sh | |
| For details see: https://github.com/rsyslog/rsyslog/pull/569 | |
| - pmciscoios: support for asterisk before timestamp added | |
| thanks to github user c0by for the patch | |
| see also: https://github.com/rsyslog/rsyslog/pull/583 | |
| - solr external output plugin much enhanced | |
| see also: https://github.com/rsyslog/rsyslog/pull/529 | |
| Thanks to Radu Gheorghe for the patch. | |
| - omrabbitmq: improvements | |
| thanks to Luigi Mori for the patch | |
| see also: https://github.com/rsyslog/rsyslog/pull/580 | |
| - add support for libfastjson (as a replacement for json-c) | |
| - KSI utilities: somewhat improved error messages | |
| Thanks to Henri Lakk for the patch. | |
| see also: https://github.com/rsyslog/rsyslog/pull/588 | |
| - pmciscoios: support for some format variations | |
| Thanks to github user c0by for the patch | |
| - support grok via new contributed module mmgrok | |
| Thanks to 饶琛琳 (github user chenryn) for the contribution. | |
| - omkafka: new statistics counter "maxoutqsize" | |
| Thanks to 饶琛琳 (github user chenryn) for the contribution. | |
| - improvments for 0mq modules: | |
| * omczmq - suspend / Retry handling - the output plugin can now recover | |
| from some error states due to issues with plugin startup or message sending | |
| * omczmq - refactored topic handling code for ZMQ_PUB output to be a little | |
| more efficient | |
| * omczmq - added ability to set a timeout for sends | |
| * omczmq - set topics can be in separate frame (default) or part of message | |
| frame (configurable) | |
| * omcmzq - code cleanup | |
| * imczmq - code cleanup | |
| * imczmq - fixed a couple of cases where vars could be used uninitialized | |
| * imczmq - ZMQ_ROUTER support | |
| * imczmq - Fix small memory leak from not freeing sockets when done with them | |
| * allow creation of on demand ephemeral CurveZMQ certs for encryption. | |
| Clients may specify clientcertpath="*" to indicate they want an on | |
| demand generated cert. | |
| Thanks to Brian Knox for the contributions. | |
| - cleanup on code to unset a variable | |
| under extreme cases (very, very unlikely), the old code could also lead | |
| to errornous processing | |
| - omelasticsearch: build on FreeBSD | |
| Thanks to github user c0by for the patch | |
| - pmciscoios: fix some small issues clang static analyzer detected | |
| - testbench: many improvements and some new tests | |
| note that there still is a number of tests which are somewhat racy | |
| - overall code improvements thanks to clang static analyzer | |
| - gnutls fix: Added possible fix for gnutls issue #575 | |
| see also: https://github.com/rsyslog/rsyslog/issues/575 | |
| Thanks to Charles Southerland for the patch | |
| - bugfix omkafka: restore ability to build on all platforms | |
| Undo commit aea09800643343ab8b6aa205b0f10a4be676643b | |
| because that lead to build failures on various important platforms. | |
| This means it currently is not possible to configure the location | |
| of librdkafka, but that will affect far fewer people. | |
| closes: https://github.com/rsyslog/rsyslog/issues/596 | |
| - bugfix omkafka: fix potentially negative partition number | |
| Thanks to Tait Clarridge for providing a patch. | |
| - bugfix: solve potential race in creation of additional action workers | |
| Under extreme circumstances, this could lead to segfault. Note that we | |
| detected this problem thanks to ASAN address sanitzier in combination | |
| with a very exterme testbench test. We do not think that this issue | |
| was ever reported in practice. | |
| - bugfix: potential memory leak in config parsing | |
| Thanks to github user linmujia for the patch | |
| - bugfix: small memory leak in loading template config | |
| This happened when a plugin was used inside the template. Then, the | |
| memory for the template name was never freed. | |
| Thanks to github user xushengping for the fix. | |
| - bugfix: fix extra whitespace in property expansions | |
| Address off-by-one issues introduced in f3bd7a2 resulting in extra | |
| whitespace in property expansions | |
| Thanks to Matthew Gabeler-Lee for the patch. | |
| - bugfix: mmfields leaked memory if very large messages were processed | |
| detected by clang static analyzer | |
| - bugfix: mmfields could add garbagge data to field | |
| this happened when very large fields were to be processed. | |
| Thanks to Peter Portante for reporting this. | |
| - bugfix: omhttpfs now also compiles with older json-c lib | |
| - bugfix: memory leak in (contributed) module omhttpfs | |
| Thanks to git hub user c6226 for the patch. | |
| - bugfix: parameter mismatch in error message for wrap() function | |
| - bugfix: parameter mismatch in error message for random() function | |
| - bugfix: divide by zero if max() function was provided zero | |
| - bugfix: invalid mutex handling in omfile async write mode | |
| could lead to segfault, even though highly unlikely (caught by | |
| testbench on a single platform) | |
| - bugfix: fix inconsistent number processing | |
| Unfortunately, previous versions of the rule engine tried to | |
| support oct and hex, but that wasn't really the case. | |
| Everything based on JSON was just dec-converted. As this was/is | |
| the norm, we fix that inconsistency by always using dec. | |
| Luckly, oct and hex support was never documented and could | |
| probably only have been activated by constant numbers. | |
| - bugfix: timezone() object: fix NULL pointer dereference | |
| This happened during startup when the offset or id parameter was not | |
| given. Could lead to a segfault at startup. | |
| Detected by clang static analyzer. | |
| - bugfix omfile: memory addressing error if very long outchannel name used | |
| Thanks to github user c6226 for the patch. | |
| ------------------------------------------------------------------------------ | |
| Version 8.14.0 [v8-stable] 2015-11-03 | |
| - Added possibility to customize librdkafka location | |
| see also: https://github.com/rsyslog/rsyslog/pull/502 | |
| Thanks to Matthew Wang for the patch. | |
| - add property "rawmsg-after-pri" | |
| - bugfix: potential misadresseing in imfile | |
| Could happen when wildcards were used. | |
| see also https://github.com/rsyslog/rsyslog/issues/532 | |
| see also https://github.com/rsyslog/rsyslog/issues/534 | |
| Thanks to zhangdaoling for the bugfix. | |
| - bugfix: re_extract RainerScript function did not work | |
| Thanks to Janmejay Singh for the patch | |
| ------------------------------------------------------------------------------ | |
| Version 8.13.0 [v8-stable] 2015-09-22 | |
| - ZeroMQ enhancements: | |
| * Added the ability to set a static publishing topic per action as an | |
| alternative to constructing topics with templates | |
| Contributor: Luca Bocassi | |
| * ZMQ_PUB socket now defaults to bind and ZMQ_SUB socket now defaults to | |
| connect - Contributor: Luca Bocassi | |
| - Redis enhancements: | |
| * Can now LPUSH to a Redis list in "queue" mode - Contributor: Brian Knox | |
| * Can now PUBLISH to a Redis channel in "publish" mode | |
| Contributor: Brian Knox | |
| - build requirement for rsyslog/mmnormalize is now liblognorm 1.1.2 or above | |
| - mmnormalize: liblognorm error messages are now emitted via regular | |
| rsyslog error reporting mechanism (aka "are now logged") | |
| This is possible due to a new API in liblognorm 1.1.2; | |
| Note that the amount of error messages depends on the version of | |
| liblognorm used. | |
| - add support for TCP client side keep-alives | |
| Thanks to github user tinselcity for the patch. | |
| - bugfix: imtcp/TLS hangs on dropped packets | |
| see also https://github.com/rsyslog/rsyslog/issues/318 | |
| Thanks to github user tinselcity for the patch. | |
| - bugfix testbench: some tests using imptcp are run if module is disabled | |
| Thanks to Michael Biebl for reporting this | |
| see also https://github.com/rsyslog/rsyslog/issues/524 | |
| - bugfix omkafka: Fixes a bug not accepting new messages anymore. | |
| see also: https://github.com/rsyslog/rsyslog/pull/472 | |
| Thanks to Janmejay Singh | |
| - bugfix: Parallel build issue "cannot find ../runtime/.libs/librsyslog.a: | |
| No such file or directory" (#479) fixed. | |
| Thanks to Thomas D. (Whissi) for the patch. | |
| - bugfix: Added missing mmpstructdata testfiles into makefile. | |
| see also: https://github.com/rsyslog/rsyslog/issues/484 | |
| - bugfix: Reverted FIX for issue #392 as it had unexpected side effects. | |
| The new fix duplicates the Listener object for static files (like | |
| done for dynamic files already), resolving issue #392 and #490. | |
| see also https://github.com/rsyslog/rsyslog/pull/490 | |
| - bugfix: issues in queue subsystem if syncqueuefiles was enabled | |
| * Error 14 was generated on the .qi file directory handle. | |
| As the .qi filestream does not have a directory set, fsync | |
| was called on an empty directory causing a error 14 in debug log. | |
| * When queue files existed on startup, the bSyncQueueFiles | |
| strm property was not set to 1. This is now done in the | |
| qqueueLoadPersStrmInfoFixup function. | |
| - bugfix/testbench: tcpflood tool could abort when random data was added | |
| see also: https://github.com/rsyslog/rsyslog/issues/506 | |
| Thanks to Louis Bouchard for the fix | |
| - rscryutil: Added support to decrypt a not closed log file. | |
| Thanks to wizard1024 for the patch. | |
| ------------------------------------------------------------------------------ | |
| Version 8.12.0 [v8-stable] 2015-08-11 | |
| - Harmonize resetConfigVariables values and defaults | |
| see also https://github.com/rsyslog/rsyslog/pull/413 | |
| Thanks to Tomas Heinrich for the patch. | |
| - GT/KSI: fix some issues in signature file format and add conversion tool | |
| The file format is incompatible to previous format, but tools have been | |
| upgraded to handle both and also an option been added to convert from | |
| old to new format. | |
| - bugfix: ommysql did not work when gnutls was enabled | |
| as it turned out, this was due to a check for GnuTLS functions | |
| with the side-effect that | |
| AC_CHECK_LIB, by default, adds the lib to LIBS, if there is no | |
| explicit action, what was the case here. So everything was now | |
| linked against GnuTLS, which in turn made ommysql fail. | |
| Thanks to Thomas D. (whissi) for the analysis of the ommysql/gnutls | |
| problem and Thomas Heinrich for pointing out that AC_CHECK_LIB might | |
| be the culprit. | |
| - bugfix omfile: potential memory leak on file close | |
| see also: https://github.com/rsyslog/rsyslog/pull/423 | |
| Thanks to Robert Schiele for the patch. | |
| - bugfix omfile: potential race in dynafile detection/creation | |
| This could lead to a segfault. | |
| Thanks to Tomas Heinrich for the patch. | |
| - bugfix omfile: Fix race-condition detection in path-creation code | |
| The affected code is used to detect a race condition in between | |
| testing for the existence of a directory and creating it if it didn't | |
| exist. The variable tracking the number of attempts wasn't reset for | |
| subsequent elements in the path, thus limiting the number of | |
| reattempts to one per the whole path, instead of one per each path | |
| element. | |
| This solution was provided by Martin Poole. | |
| - bugfix parser subsystem: potential misadressing in SanitizeMsg() | |
| could lead to a segfault | |
| Thanks to Tomas Heinrich for the patch. | |
| - imfile: files moved outside of directory are now (properly) handled | |
| - bugfix: imfile: segfault when using startmsg.regex if first log line | |
| doesn't match | |
| Thanks to Ciprian Hacman for the patch. | |
| - bugfix imfile: file table was corrupted when on file deletion | |
| This could happen when a file that was statically configured (not via an | |
| wildcard) was deleted. | |
| - bugfix ompgsql: transaction were improperly handled | |
| now transaction support is solidly disabled until we have enough requests | |
| to implement it again. Module still works fine in single insert mode. | |
| closes https://github.com/rsyslog/rsyslog/issues/399 | |
| - bugfix mmjsonparse: memory leak if non-cee-json message is processed | |
| see also https://github.com/rsyslog/rsyslog/pull/383 | |
| Thanks to Anton Matveenko for the patch | |
| - testbench: remove raciness from UDP based tests | |
| - testbench: added bash into all scripts makign it mandatory | |
| - bugfix testbench: Fixed problem building syslog_caller util when | |
| liblogging-stdlog is not available. | |
| Thanks to Louis Bouchard for the patch | |
| - bugfix rscryutil.1: Added fix checking for generate_man_pages condition | |
| Thanks to Radovan Sroka for the patch | |
| - bugfix freebsd console: \n (NL) is prepended with \r (CR) in console | |
| output on freebsd only. For more details see here: | |
| https://github.com/rsyslog/rsyslog/issues/372 | |
| Thanks to AlexandreFenyo for the patch | |
| ------------------------------------------------------------------------------ | |
| Version 8.11.0 [v8-stable] 2015-06-30 | |
| - new signature provider for Keyless Signature Infrastructure (KSI) added | |
| - build system: re-enable use of "make distcheck" | |
| - add new signature provider for Kesless Signature Infrastructure (KSI) | |
| This has also been added to existing tooling; KSI is kind of v2 of | |
| the Guardtime functionality and has been added in the appropriate | |
| places. | |
| - bugfix imfile: regex multiline mode ignored escapeLF option | |
| Thanks to Ciprian Hacman for reporting the problem | |
| closes https://github.com/rsyslog/rsyslog/issues/370 | |
| - bugfix omkafka: fixed several concurrency issues, most of them related | |
| to dynamic topics. | |
| Thanks to Janmejay Singh for the patch. | |
| - bugfix: execonlywhenpreviousissuspended did not work correctly | |
| This especially caused problems when an action with this attribute was | |
| configured with an action queue. | |
| - bugfix core engine: ensured global variable atomicity | |
| This could lead to problems in RainerScript, as well as probably in other | |
| areas where global variables are used inside rsyslog. I wouldn't outrule | |
| it could lead to segfaults. | |
| Thanks to Janmejay Singh for the patch. | |
| - bugfix imfile: segfault when using startmsg.regex because of empty log line | |
| closes https://github.com/rsyslog/rsyslog/issues/357 | |
| Thanks to Ciprian Hacman for the patch. | |
| - bugfix: build problem on Solaris | |
| Thanks to Dagobert Michelsen for reporting this and getting us up to | |
| speed on the openCWS build farm. | |
| - bugfix: build system strndup was used even if not present | |
| now added compatibility function. This came up on Solaris builds. | |
| Thanks to Dagobert Michelsen for reporting the problem. | |
| closes https://github.com/rsyslog/rsyslog/issues/347 | |
| - bugfix imjournal: do not pass empty messages to rsyslog core | |
| this causes a crash of the daemon | |
| see also https://github.com/rsyslog/rsyslog/pull/412 | |
| Thanks to Tomas Heinrich for the patch. | |
| - bugfix imjournal: cosmetic memory leak | |
| very small and an shutdown only, so did not affect operations | |
| see also https://github.com/rsyslog/rsyslog/pull/411 | |
| Thanks to Tomas Heinrich for the patch. | |
| ------------------------------------------------------------------------------ | |
| Version 8.10.0 [v8-stable] 2015-05-19 | |
| - imfile: add capability to process multi-line messages based on regex | |
| input parameter "endmsg.regex" was added for that purpose. The new | |
| mode provides much more power in processing different multiline-formats. | |
| - pmrfc3164: add new parameters | |
| * "detect.yearAfterTimestamp" | |
| This supports timestamps as generated e.g. by some Aruba Networks | |
| equipment. | |
| * "permit.squareBracesInHostname" | |
| Permits to use "hostnames" in the form of "[127.0.0.1]"; also seen in | |
| Aruba Networks equipment, but we strongly assume this can also happen | |
| in other cases, especially with IPv6. | |
| - supplementary groups are now set when dropping privileges | |
| closes https://github.com/rsyslog/rsyslog/issues/296 | |
| Thanks to Zach Lisinski for the patch. | |
| - imfile: added brace glob expansion to wildcard | |
| Thanks to Zach Lisinski for the patch. | |
| - zmq: add the ability for zeromq input and outputs to advertise their | |
| presence on UDP via the zbeacon API. | |
| Thanks to Brian Knox for the contribution. | |
| - added omhttpfs: contributed module for writing to HDFS via HTTP | |
| Thanks to sskaje for the contribution. | |
| - Configure option "--disable-debug-symbols" added which is disabled per | |
| default. If you set the new option, configure won't set the appropriate | |
| compiler flag to generate debug symbols anymore. | |
| - When building from git source we now require rst2man and yacc (or a | |
| replacement like bison). | |
| That isn't any new requirement, we only added missing configure checks. | |
| - Configure option "--enable-generate-man-pages" is now disabled for non git | |
| source builds per default but enforced when building from git source. | |
| - mmpstrucdata: some code cleanup | |
| removed lots of early development debug outputs | |
| - bugfix imuxsock: fix a memory leak that happened with large messages | |
| ... when annotation was enabled. | |
| Thanks to github user c6226 for the patch | |
| - bugfix omhttpfs: memory leak | |
| Thanks to github user c6226 for the patch | |
| - bugfix imuxsock: fix a crash when setting a hostname | |
| Setting a hostname via the legacy directive would lead to a crash | |
| during shutdown caused by a double-free. | |
| Thanks to Tomas Heinrich for the patch. | |
| - bugfix: memory leak in mmpstrucdata | |
| Thanks to Grégoire Seux for reporting this issue. | |
| closes https://github.com/rsyslog/rsyslog/issues/310 | |
| - bugfix (minor): default action name: assigned number was one off | |
| see also https://github.com/rsyslog/rsyslog/pull/340 | |
| Thanks to Tomas Heinrich for the patch. | |
| - bugfix: memory leak in imfile | |
| A small leak happened each time a new file was monitored based on | |
| a wildcard. Depending on the rate of file creation, this could result | |
| in a serious memory leak. | |
| ------------------------------------------------------------------------------ | |
| Version 8.9.0 [v8-stable] 2015-04-07 | |
| - omprog: add option "hup.forward" to forwards HUP to external plugins | |
| This was suggested by David Lang so that external plugins (and other | |
| programs) can also do HUP-specific processing. The default is not | |
| to forward HUP, so no change of behavior by default. | |
| - imuxsock: added capability to use regular parser chain | |
| Previously, this was a fixed format, that was known to be spoken on | |
| the system log socket. This also adds new parameters: | |
| - sysSock.useSpecialParser module parameter | |
| - sysSock.parseHostname module parameter | |
| - useSpecialParser input parameter | |
| - parseHostname input parameter | |
| - 0mq: improvements in input and output modules | |
| See module READMEs, part is to be considered experimental. | |
| Thanks to Brian Knox for the contribution. | |
| - imtcp: add support for ip based bind for imtcp -> param "address" | |
| Thanks to github user crackytsi for the patch. | |
| - bugfix: MsgDeserialize out of sync with MsgSerialize for StrucData | |
| This lead to failure of disk queue processing when structured data was | |
| present. Thanks to github user adrush for the fix. | |
| - bugfix imfile: partial data loss, especially in readMode != 0 | |
| closes https://github.com/rsyslog/rsyslog/issues/144 | |
| - bugfix: potential large memory consumption with failed actions | |
| see also https://github.com/rsyslog/rsyslog/issues/253 | |
| - bugfix: omudpspoof: invalid default send template in RainerScript format | |
| The file format template was used, which obviously does not work for | |
| forwarding. Thanks to Christopher Racky for alerting us. | |
| closes https://github.com/rsyslog/rsyslog/issues/268 | |
| - bugfix: size-based legacy config statements did not work properly | |
| on some platforms, they were incorrectly handled, resulting in all | |
| sorts of "interesting" effects (up to segfault on startup) | |
| - build system: added option --without-valgrind-testbench | |
| ... which provides the capability to either enforce or turn off | |
| valgrind use inside the testbench. Thanks to whissi for the patch. | |
| - rsyslogd: fix misleading typos in error messages | |
| Thanks to Ansgar Püster for the fixes. | |
| ------------------------------------------------------------------------------ | |
| Version 8.8.0 [v8-stable] 2015-02-24 | |
| - omkafka: add support for dynamic topics and auto partitioning | |
| Thanks to Tait Clarridge for the patches. | |
| - imtcp/imptcp: support for broken Cisco ASA TCP syslog framing | |
| - omfwd: more detailled error messages in case of UDP send error | |
| - TLS syslog: enable capability to turn on GnuTLS debug logging | |
| This provides better diagnostics in hard-to-diagnose cases, | |
| especially when GnuTLS is extra-picky about certificates. | |
| - bugfix: $AbortOnUncleanConfig did not work | |
| - improve rsyslogd -v output and error message with meta information | |
| version number is now contained in error message and build platform in | |
| version output. This helps to gets rid of the usual "which version" | |
| question on mailing list, support forums, etc... | |
| - bugfix imtcp: octet-counted framing cannot be turned off | |
| - bugfix: build problems on Illuminos | |
| Thanks to Andrew Stormont for the patch | |
| - bugfix: invalid data size for iMaxLine global property | |
| It was defined as int, but inside the config system it was declared as | |
| size type, which uses int64_t. With legacy config statements, this could | |
| lead to misadressing, which usually meant the another config variable was | |
| overwritten (depending on memory layout). | |
| closes https://github.com/rsyslog/rsyslog/issues/205 | |
| - bugfix: negative values for maxMessageSize global parameter were permitted | |
| ------------------------------------------------------------------------------ | |
| Version 8.7.0 [v8-stable] 2015-01-13 | |
| - add message metadata "system" to msg object | |
| this permits to store metadata alongside the message | |
| - imfile: add support for "filename" metadata | |
| this is useful in cases where wildcards are used | |
| - imptcp: make stats counter names consistent with what imudp, imtcp uses | |
| - added new module "omkafka" to support writing to Apache Kafka | |
| - omfwd: add new "udp.senddelay" parameter | |
| - mmnormalize enhancements | |
| Thanks to Janmejay Singh for the patch. | |
| - RainerScript "foreach" iterator and array reading support | |
| Thanks to Janmejay Singh for the patch. | |
| - now requires liblognorm >= 1.0.2 | |
| - add support for systemd >= 209 library names | |
| - BSD "ntp" facility (value 12) is now also supported in filter | |
| Thanks to Douglas K. Rand of Iteris, Inc. for the patch. | |
| Note: this patch was released under ASL 2.0 (see email-conversation). | |
| - bugfix: global(localHostName="xxx") was not respected in all modules | |
| - bugfix: emit correct error message on config-file-not-found | |
| closes https://github.com/rsyslog/rsyslog/issues/173 | |
| - bugfix: impstats emitted invalid JSON format (if JSON was selected) | |
| - bugfix: (small) memory leak in omfile's outchannel code | |
| Thanks to Koral Ilgun for reporting this issue. | |
| - bugfix: imuxsock did not deactivate some code not supported by platform | |
| Among potential other problemns, this caused build failure under Solaris. | |
| Note that this build problem just made a broader problem appear that so | |
| far always existed but was not visible. | |
| closes https://github.com/rsyslog/rsyslog/issues/185 | |
| ------------------------------------------------------------------------------ | |
| Version 8.6.0 [v8-stable] 2014-12-02 | |
| NOTE: This version also incorporates all changes and enhancements made for | |
| v8.5.0, but in a stable release. For details see immediately below. | |
| - configuration-setting rsyslogd command line options deprecated | |
| For most of them, there are now proper configuration objects. Some few | |
| will be completely dropped if nobody insists on them. Additional info at | |
| http://blog.gerhards.net/2014/11/phasing-out-legacy-command-line-options.html | |
| - new and enhanced plugins for 0mq. These are currently experimantal. | |
| Thanks to Brian Knox who contributed the modules and is their author. | |
| - empty rulesets have been permitted. They no longer raise a syntax error. | |
| - add parameter -N3 to enable config check of partial config file | |
| Use for config include files. Disables checking if any action exists at | |
| all. | |
| - rsyslogd -e option has finally been removed | |
| It is deprectated since many years. | |
| - testbench improvements | |
| Testbench is now more robust and has additional tests. | |
| - testbench is now by default disabled | |
| To enable it, use --enable-testbench. This was done as the testbench now | |
| does better checking if required modules are present and this in turn | |
| would lead to configure error messages where non previously were if we | |
| would leave --enable-testbench on by default. Thus we have turned it off. | |
| This should not be an issue for those few testbench users. | |
| - add new RainerScript functions warp() and replace() | |
| Thanks to Singh Janmejay for the patch. | |
| - mmnormalize can now also work on a variable | |
| Thanks to Singh Janmejay for the patch. | |
| - new property date options for day ordinal and week number | |
| Thanks to github user arrjay for the patch | |
| - remove --enable-zlib configure option, we always require it | |
| It's hard to envision a system without zlib, so we turn this off | |
| closes https://github.com/rsyslog/rsyslog/issues/76 | |
| - slight source-tree restructuring: contributed modules are now in their | |
| own ./contrib directory. The idea is to make it clearer to the end user | |
| which plugins are supported by the rsyslog project (those in ./plugins). | |
| - bugfix: imudp makes rsyslog hang on shutdown when more than 1 thread used | |
| closes https://github.com/rsyslog/rsyslog/issues/126 | |
| - bugfix: not all files closed on auto-backgrounding startup | |
| This could happen when not running under systemd. Some low-numbered | |
| fds were not closed in that case. | |
| - bugfix: typo in queue configuration parameter | |
| made parameter unusable | |
| Thanks to Bojan Smojver for the patch. | |
| - bugfix: unitialized buffer off-by-one error in hostname generation | |
| The DNS cache used uninitialized memory, which could lead to | |
| invalid hostname generation. | |
| Thanks to Jarrod Sayers for alerting us and provinding analysis and | |
| patch recommendations. | |
| - bugfix imuxsock: possible segfault when SysSock.Use="off" | |
| Thanks to alexjfisher for reporting this issue. | |
| closes https://github.com/rsyslog/rsyslog/issues/140 | |
| - bugfix: RainerScript: invalid ruleset names were accepted | |
| during ruleset defintion, but could of course not be used when | |
| e.g. calling a ruleset. | |
| IMPORTANT: this may cause existing configurations to error out on start, | |
| as they invalid names could also be used e.g. when assigning rulesets. | |
| - bugfix: some module entry points were not called for all modules | |
| callbacks like endCnfLoad() were primarily being called for input | |
| modules. This has been corrected. Note that this bugfix has some | |
| regression potential. | |
| - bugfix omlibdbi: connection was taken down in wrong thread | |
| this could have consequences depending on the driver being used. In | |
| general, it looks more like a cosmetic issue. For example, with | |
| MySQL it lead to a small memory but also an annoying message about | |
| a thread not properly torn down. | |
| - imttcp was removed because it was an incompleted experimental module | |
| - pmrfc3164sd because it was a custom module nobody used | |
| We used to keep this as a sample inside the tree, but whoever wants | |
| to look at it can check in older versions inside git. | |
| - omoracle was removed because it was orphaned and did not build/work | |
| for quite some years and nobody was interested in fixing it | |
| --------------------------------------------------------------------------- | |
| Version 8.5.0 [v8-stable] 2014-10-24 | |
| - imfile greatly refactored and support for wildcards added | |
| - PRI-handling code refactored for more clarity and robustness | |
| - ommail: add support for RainerScript config system [action() object] | |
| This finally adds support for the new config style. Also, we now permit | |
| to set a constant subject text without the need to create a template for | |
| it. | |
| - refactored the auto-backgrounding method | |
| The code is now more robust and also offers possibilities for enhanced | |
| error reporting in the future. This is also assumed to fix some races | |
| where a system startup script hang due to "hanging" rsyslogd. | |
| - make gntls tcp syslog driver emit more error messages | |
| Messages previously emitted only to the debug log are now emitted as | |
| syslog error messages. It has shown that they contain information | |
| helpful to the user for troubleshooting config issues. Note that this | |
| change is a bit experimental, as we are not sure if there are situations | |
| where large amounts of error messages may be emitted. | |
| - bugfix: imfile did not complain if configured file did not exist | |
| closes https://github.com/rsyslog/rsyslog/issues/137 | |
| - bugfix: build failure on systems which don't have json_tokener_errors | |
| Older versions of json-c need to use a different API (which don't exists | |
| on newer versions, unfortunately...) | |
| Thanks to Thomas D. for reporting this problem. | |
| - imgssapi: log remote peer address in some error messages | |
| Thanks to Bodik for the patch. | |
| --------------------------------------------------------------------------- | |
| Version 8.4.3 [v8-stable] 2014-10-?? | |
| - ommail: minor bugfixes & improvements | |
| * timestamps were 1 hour out when using daylight saving times when | |
| viewing emails in most email clients due to incorrect date format | |
| * X-Mailer header had a typo in it | |
| * To: header was duplicated once per recipient (this is permitted, | |
| but an address list is a better choice nowadays) | |
| Thanks to github user cacheus for the patches. | |
| - bugfix imkmsg: infinite loop on OpenVZ VMs | |
| Thanks to github user PaulSD for the patch | |
| closes https://github.com/rsyslog/rsyslog/pull/138 | |
| - bugfix: typo in queue configuration parameter made parameter unusable | |
| Thanks to Bojan Smojver for the patch. | |
| - bugfix: unitialized buffer off-by-one error in hostname generation | |
| The DNS cache used uninitialized memory, which could lead to | |
| invalid hostname generation. | |
| Thanks to Jarrod Sayers for alerting us and provinding analysis and | |
| patch recommendations. | |
| - bugfix imfile: segfault on startup in "inotify" mode | |
| A segfault happened when more than one file was monitored. | |
| - bugfix imfile: could make rsyslog exit in inotify mode | |
| - bugfix: rsgtutil sometimes crashed in verify mode if file did not exist | |
| - bugfix imklog: pri was miscalculated | |
| actually, the pri was totally off the real value for PRIs > 9 | |
| - bugfix imfile:file processing in inotify mode was stalled sometimes | |
| closes https://github.com/rsyslog/rsyslog/issues/134 | |
| - bugfix: imjournal did not build properly | |
| The build succeeded, but the module did not load due to a type in | |
| a support function name, which kept unresolved during load. | |
| - bugfix: mmcount did no longer build | |
| note that this is untested -- users of this module should file a bug if | |
| the new (trivial) code is broken [if there are any users, thus I did not | |
| invest time in testing...] | |
| closes https://github.com/rsyslog/rsyslog/issues/129 | |
| - bugfix imuxsock: possible segfault when SysSock.Use="off" | |
| Thanks to alexjfisher for reporting this issue. | |
| closes https://github.com/rsyslog/rsyslog/issues/140 | |
| --------------------------------------------------------------------------- | |
| Version 8.4.2 [v8-stable] 2014-10-02 | |
| - bugfix: the fix for CVE-2014-3634 did not handle all cases | |
| This is corrected now. | |
| see also: CVE-2014-3683 | |
| - fixed a build problem on some platforms | |
| Thanks to Olaf for the patch | |
| - behaviour change: "msg" of messages with invalid PRI set to "rawmsg" | |
| When the PRI is invalid, the rest of the header cannot be valid. So | |
| we move all of it to MSG and do not try to parse it out. Note that | |
| this is not directly related to the security issue but rather done | |
| because it makes most sense. | |
| --------------------------------------------------------------------------- | |
| Version 8.4.1 [v8-stable] 2014-09-30 | |
| - imudp: add for bracketing mode, which makes parsing stats easier | |
| - permit at-sign in variable names | |
| closes: https://github.com/rsyslog/rsyslog/issues/110 | |
| - bugfix: fix syntax error in anon_cc_numbers.py script | |
| Thanks to github user anthcourtney for the patch. | |
| closes: https://github.com/rsyslog/rsyslog/issues/109 | |
| - bugfix: ompgsql: don't loose uncomitted data on retry | |
| Thanks to Jared Johnson and Axel Rau for the patch. | |
| - bugfix: imfile: if a state file for a different file name was set, | |
| that different file (name) was monitored instead of the configured | |
| one. Now, the state file is deleted and the correct file monitored. | |
| closes: https://github.com/rsyslog/rsyslog/issues/103 | |
| - bugfix: omudpspoof: source port was invalid | |
| Thanks to Pavel Levshin for the patch | |
| - bugfix: build failure on systems which don't have json_tokener_errors | |
| Older versions of json-c need to use a different API (which don't exists | |
| on newer versions, unfortunately...) | |
| Thanks to Thomas D. for reporting this problem. | |
| - bugfix: omelasticsearch does not work with broken/changed ES 1.0+ API | |
| closes: https://github.com/rsyslog/rsyslog/issues/104 | |
| - bugfix: mmanon did not properly anonymize IP addresses starting with '9' | |
| Thanks to defa-at-so36.net for reporting this problem. | |
| closes: http://bugzilla.adiscon.com/show_bug.cgi?id=529 | |
| - bugfix: build problems on SuSe Linux | |
| Thanks Andreas Stieger for the patch | |
| - bugfix: omelasticsearch error file did not work correctly on ES 1.0+ | |
| due to a breaking change in the ElasticSearch API. | |
| see also: https://github.com/rsyslog/rsyslog/issues/104 | |
| - bugfix: potential abort when a message with PRI > 191 was processed | |
| if the "pri-text" property was used in active templates, this could | |
| be abused to a remote denial of service from permitted senders | |
| see also: CVE-2014-3634 | |
| --------------------------------------------------------------------------- | |
| Version 8.4.0 [v8-stable] 2014-08-18 | |
| - this is the new stable branch, which incorporates all enhancements of | |
| rsyslog 8.3. | |
| --------------------------------------------------------------------------- | |
| Version 8.3.5 [v8-devel] 2014-08-05 | |
| - mmjsonparse: support selectable cookie and target containers | |
| This permits to put different meanings into a json formatted syslog | |
| message, e.g. the "traditional" cee or cim data. | |
| - bugfix: mmjsonparse did not build with json-c < 0.10 | |
| This was a regression introduced some time in the past in order to | |
| support API changes in json-c. Now we check for the version and use | |
| proper code. | |
| - omprog: emit error message via syslog() if loading binary fails | |
| This happens after forking, so omprog has no longer access to rsyslog's | |
| regular error reporting functions. Previously, this meant any error | |
| message was lost. Now it is emitted via regular syslog (which may end up | |
| in a different instance, if multiple instances run...) | |
| - couple of patches imported from v7-stable (7.6.4) | |
| --------------------------------------------------------------------------- | |
| Version 8.3.4 [v8-devel] 2014-07-11 | |
| - new pmciscoios parser supporting various Cisco IOS formats | |
| - RFC3164 timestamp parser now accepts timezones and subsecond resolution | |
| ... at least for some common formats and where we could do so without | |
| running risk of breaking proper formats (or introducing regressions) | |
| - new parser config object -- permits to define custom parser definitions | |
| - new tzinfo config object -- permits to define time zone offsets | |
| This is a utility object that currently is being used by some parsers. | |
| - bugfix: mishandling of input modules not supporting new input instances | |
| If they did not support this, accidently the output module part of the | |
| module union was written, leading to unpredictable results. Note: all | |
| core modules do support this interface, but some contributed or very | |
| old ones do not. | |
| - bugfix: double-free when ruleset() parser parameters were used | |
| While unlikely, this could cause stability issues even after the | |
| config phase. | |
| --------------------------------------------------------------------------- | |
| Version 8.3.3 [v8-devel] 2014-06-26 | |
| - unify input object naming | |
| imudp now supports "name" parameter, as other inputs do. "inputname" has | |
| been deprecated, but can still be used. Same applies to "appendport" | |
| subparameter". Thanks to "Nick Syslog" for the suggestion. | |
| - made the missing (contributed) modules build under v8 [import from 8.2.2] | |
| Modules: | |
| * mmrfc5424addhmac | |
| * omrabbitmq | |
| * omgssapi | |
| * omhdfs | |
| * omzmq3 | |
| - added a cleanup process (janitor); permits to close omfile files after a | |
| timeout | |
| - make omgssapi build under v8.3 [import vom v8.2] | |
| note that we could do this to the stable, because there is NO regression | |
| chance at all: only omgssapi was changed, and this module did NOT work | |
| previously. | |
| - removed obsolete --disable-fsstnd configure option | |
| Thanks to Thomas D. for alerting us. | |
| Closes: https://github.com/rsyslog/rsyslog/issues/72 | |
| --------------------------------------------------------------------------- | |
| Version 8.3.2 [v8-devel] 2014-05-02 | |
| - new template options for date extraction: | |
| - year | |
| - month | |
| - day | |
| - wday | |
| - hour | |
| - minute | |
| - second | |
| - tzoffshour | |
| - tzoffsmin | |
| - tzoffsdirection | |
| - wdayname | |
| For string templates, these are property options and they are | |
| prefixed with "date-" (e.g. "date-year", "date-month", ...) | |
| see also: https://github.com/rsyslog/rsyslog/issues/65 | |
| - bugfix: mmexternal remove framing char before processing JSON reply | |
| This did not have any real bad effects, but caused unnecessary | |
| processing, as empty replies were not properly detected. Otherwise, | |
| the bug was not noticible from the user's PoV. | |
| - bugfix: mmexternal segfault due to invalid free in non-json input mode | |
| closes: https://github.com/rsyslog/rsyslog/issues/70 | |
| - bugfix: mmexternal segfault when external plugin sent invalid reply | |
| ... or no reply at all. This happened if the reply was imporper JSON. | |
| Now, we emit an error message in those cases. | |
| see also: https://github.com/rsyslog/rsyslog/issues/69 | |
| - bugfix: mmexternal did potentially pass incomplete data to restarted | |
| external plugin | |
| This could happen if EPIPE was returned "too late", in which case the | |
| beginning of the data could be lost. | |
| - bugfix: mmexternal did not properly process messages over 4KiB | |
| The data to be passed to the external plugin was truncated after 4KiB. | |
| see: https://github.com/rsyslog/rsyslog/issues/64 | |
| - imrelp: added support for per-listener ruleset and inputname | |
| see: https://github.com/rsyslog/rsyslog/pull/63 | |
| Thanks to bobthesecurityguy github user for the patch | |
| --------------------------------------------------------------------------- | |
| Version 8.3.1 [v8-devel] 2014-04-24 | |
| - external message modification interface now support modifying message PRI | |
| - "jsonmesg" property will include uuid only if one was previously generated | |
| This is primarily a performance optimization. Whenever the message uuid | |
| is gotten, it is generated when not already present. As we used the | |
| regular setter, this means that always the uuid was generated, which is | |
| quite time-consuming. This has now been changed so that it only is | |
| generated if it already exists. That also matches more closly the | |
| semantics, as "jsonmesg" should not make modifications to the message. | |
| Note that the same applies to "fulljson" passing mode for external | |
| plugins. | |
| - added plugin to rewrite message facility and/or severity | |
| Name: fac-sever-rewrite.py | |
| - permits to build against json-c 0.12 | |
| Unfortunately, json-c had an ABI breakage, so this is necessary. Note | |
| that versions prior to 0.12 had security issues (CVE-2013-6370, | |
| CVE-2013-6371) and so it is desirable to link against the new version. | |
| Thanks to Thomas D. for the patch. Note that at least some distros | |
| have fixed the security issue in older versions of json-c, so this | |
| seems to apply mostly when building from sources. | |
| - bugfix: using UUID property could cause segfault | |
| - bugfix/mmexternal: memory leak | |
| - bugfix: memory leak when using "jsonmesg" property | |
| - bugfix: mmutf8fix did not detect two invalid sequences | |
| Thanks to Axel Rau for the patch. | |
| - bugfix: build problems with lexer.l on some platforms | |
| For some reason, the strdup() prototype and others are missing. I admit | |
| that I don't know why, as this happens only in 8.3.0+ and there is no | |
| indication of changes to the affected files. In any case, we need to | |
| fix this, and the current solution works at least as an interim one. | |
| --------------------------------------------------------------------------- | |
| Version 8.3.0 [v8-devel] 2014-04-10 | |
| - new plugin for anonymizing credit card numbers | |
| Thanks to Peter Slavov for providing the code. | |
| - external message modification modules are now supported | |
| They are bound via the new native module "mmexternal". Also, a sample | |
| skeleton for an external python message modification module has been | |
| added. | |
| - new $jsonmesg property with JSON representation of whole message object | |
| closes: https://github.com/rsyslog/rsyslog/issues/19 | |
| - improved error message for invalid field extraction in string template | |
| see also: | |
| http://kb.monitorware.com/problem-with-field-based-extraction-t12299.html | |
| - fix build problems on Solaris | |
| - NOTE: a json-c API that we begun to use requires the compiler to be in | |
| c99 mode. By default, we select it automatically. If you modify this and | |
| use gcc, be sure to include "-std=c99" in your compiler flags. This seems | |
| to be necessary only for older versions of gcc. | |
| --------------------------------------------------------------------------- | |
| Version 8.2.3 [v8-stable] 2014-??-?? | |
| - bugfix: ommysql: handle/mem leak upon termination of worker thread | |
| This could become bad if the (instance) worker threads are often | |
| started and terminated. But it takes quite a while to show effect. | |
| --------------------------------------------------------------------------- | |
| Version 8.2.2 [v8-stable] 2014-06-02 | |
| - made the missing (contributed) modules build under v8 | |
| Note that we could do this to the stable, because there is NO regression | |
| chance at all: only the modules themselves were changed, and they did | |
| NOT work at all previously. Please also note that most of these modules | |
| did not yet receive real testing. As we don't have the necessary | |
| environments (easily enough available), we depend on users submitting | |
| error reports and helping to iron out any issues that may arise. | |
| Modules: | |
| * mmrfc5424addhmac | |
| * omrabbitmq | |
| * omgssapi | |
| * omhdfs | |
| * omzmq3 | |
| --------------------------------------------------------------------------- | |
| Version 8.2.1 [v8-stable] 2014-04-17 | |
| - permits to build against json-c 0.12 | |
| Unfortunately, json-c had an ABI breakage, so this is necessary. Note | |
| that versions prior to 0.12 had security issues (CVE-2013-6370, | |
| CVE-2013-6371) and so it is desirable to link against the new version. | |
| Thanks to Thomas D. for the patch. Note that at least some distros | |
| have fixed the security issue in older versions of json-c, so this | |
| seems to apply mostly when building from sources. | |
| - doc is no longer shipped as part of the rsyslog tarball | |
| Instead, the rsyslog-doc project creates its own tarball. This is the | |
| result of a mailing list discussion after the 8.2.0 release with a | |
| tarball-in-tarball approach, which was disliked by almost all distro | |
| maintainers. This move also has the advantage of de-coupling the | |
| release cycles of both projects a bit (which turned out to be a bit | |
| problematic in practice). | |
| - bugfix: mmutf8fix did not detect two invalid sequences | |
| Thanks to Axel Rau for the patch. | |
| --------------------------------------------------------------------------- | |
| Version 8.2.0 [v8-stable] 2014-04-02 | |
| This starts a new stable branch based on 8.1.6 plus the following changes: | |
| - we now use doc from the rsyslog-doc project | |
| As such, the ./doc subtree has been removed. Instead, a cache of the | |
| rsyslog-doc project's files has been included in ./rsyslog-doc.tar.gz. | |
| Note that the exact distribution mode for the doc is still under | |
| discussion and may change in future releases. | |
| This was agreed upon on the rsyslog mailing list. For doc issues | |
| and corrections, be sure to work with the rsyslog-doc project. It is | |
| currently hosted at https://github.com/rsyslog/rsyslog-doc | |
| - add support for specifying the liblogging-stdlog channel spec | |
| new global parameter "stdlog.channelspec" | |
| - add "defaultnetstreamdrivercertfile" global variable to set a default | |
| for the certfile. | |
| Thanks to Radu Gheorghe for the patch. | |
| - omelasticsearch: add new "usehttps" parameter for secured connections | |
| Thanks to Radu Gheorghe for the patch. | |
| - "action resumed" message now also specifies module type | |
| which makes troubleshooting a bit easier. Note that we cannot output all | |
| the config details (like destination etc) as this would require much more | |
| elaborate code changes, which we at least do not like to do in the | |
| stable version. | |
| - add capability to override GnuTLS path in build process | |
| Thanks to Clayton Shotwell for the patch | |
| - better and more consistent action naming, action queues now always | |
| contain the word "queue" after the action name | |
| - bugfix: ompipe did resume itself even when it was still in error | |
| See: https://github.com/rsyslog/rsyslog/issues/35 | |
| Thanks to github user schplat for reporting | |
| - bugfix: ompipe used invalid default template | |
| This is a regression from an old change (didn't track it down precisely, | |
| but over a year ago). It used the Forwarding template instead of | |
| the file template (so we have a full syslog header). This fix corrects | |
| it back to previous behaviour, but new scripts that used the wrong | |
| format may now need to have the RSYSLOG_ForwardingFormat template | |
| explicitely be applied. | |
| closes: https://github.com/rsyslog/rsyslog/issues/50 | |
| --------------------------------------------------------------------------- | |
| Version 8.1.6 [release candidate] 2014-02-20 | |
| - omfile: permit to set global defaults for action parameters | |
| Thanks to Nathan Brown for the patch. | |
| See also: https://github.com/rsyslog/rsyslog/pull/23 | |
| - add capability to escape control characters in the C way of doing it | |
| adds new global parameter "parser.escapeControlCharactersCStyle" | |
| Thanks to Nathan Brown for the patch. | |
| See also: https://github.com/rsyslog/rsyslog/pull/13 | |
| - parser global parameters can now be set using RainerScript global() | |
| Thanks to Nathan Brown for the patch. | |
| See also: https://github.com/rsyslog/rsyslog/pull/23 | |
| - omprog: guard program-to-be-executed against CTL-C | |
| This can frequently happen in debug mode, where rsyslog is terminated | |
| by ctl-c. In any case, SIGINT is not meant to control the child process, | |
| so it should be blocked. | |
| - omprog bugfix: parameter "forceSingleInstance" is NOT mandatory | |
| - add new jsonr property replacer option | |
| Thanks to Nathan Brown for the patch. | |
| - added external plugin interface | |
| - ommongodb: add authentication support (untested) | |
| Thanks to JT for the patch. | |
| See also: https://github.com/rsyslog/rsyslog/pull/17 | |
| - bugfix: json templates are improperly created | |
| Strings miss the terminating NUL character, which obviously can lead | |
| to all sorts of problems. | |
| See also: https://github.com/rsyslog/rsyslog/issues/27 | |
| Thanks to Alain for the analysis and the patch. | |
| - ompgsql bugfix: improper handling of auto-backgrounding mode | |
| If rsyslog was set to auto-background itself (default code behaviour, but | |
| many distros now turn it off for good reason), ompgsql could not | |
| properly connect. This could even lead to a segfault. The core reason | |
| was that a PG session handle was kept open over a fork, something that | |
| is explicitely forbidden in the PG API. | |
| Thanks to Alain for the analysis and the patch. | |
| - bugfix: ommongodb's template parameter was mandatory but should have | |
| been optional | |
| Thanks to Alain for the analysis and the patch. | |
| - bugfix: end of batch processing was not 100% correct. Could lead to | |
| outputs not properly wirting messages. At least omelasticsearch did not | |
| write anything to the database due to this bug. | |
| See: https://github.com/rsyslog/rsyslog/issues/10 | |
| Thanks to Radu Gheorghe for reporting the issue. | |
| --------------------------------------------------------------------------- | |
| Version 8.1.5 [devel] 2014-01-24 | |
| - omprog: ability to execute multiple program instances per action | |
| It can now execute one program instance per worker thread. This is | |
| generally a very good thing the have performance wise. Usually, this | |
| should cause no problems with the invoked program. For that reason, | |
| we have decided to make this the default mode of operation. If not | |
| desired, it can be turned off via the 'forceSingleInstance="on"' | |
| action parameter. | |
| CHANGE OF BEHAVIOUR: previous versions did always execute only one | |
| instance per action, no matter how many workers were active. If | |
| your program has special needs, you need to change your configuration. | |
| - imfile now supports inotify (but must be explicitely turned on) | |
| - imfile no longer has a limit on number of monitored files | |
| - added ProcessInternalMessages global system parameter | |
| This permits to inject rsyslog status messages into *another* main | |
| syslogd or the journal. | |
| - new dependency: liblogging-stdlog (for submitting to external logger) | |
| - bugfix: imuxsock input parameters were not accepted | |
| due to copy&paste error. Thanks to Andy Goldstein for the fix. | |
| --------------------------------------------------------------------------- | |
| Version 8.1.4 [devel] 2014-01-10 | |
| - add exec_template() RainerScript function | |
| - imrelp: support for TCP KEEPALIVE added | |
| - bumped librelp dependency to 1.2.2 to support new KEEPALIVE feature | |
| - Add directives for numerically specifying GIDs/UIDs | |
| The already present directives (FileOwner, FileGroup, DirOwner, | |
| DirGroup) translate names to numerical IDs, which depends on the user | |
| information being available during rsyslog's startup. This can fail if | |
| the information is obtained over a network or from a service such as | |
| SSSD. The new directives provide a way to specify the numerical IDs | |
| directly and bypass the lookup. | |
| Thanks to Tomas Heinrich for the patch. | |
| - bugfix: action commitTransaction() processing did not properly handle | |
| suspended actions | |
| - bugfix: omelasticsearch fail.es stats counter was improperly maitained | |
| --------------------------------------------------------------------------- | |
| Version 8.1.3 [devel] 2013-12-06 | |
| THIS VERSION CAN BE CONSIDERED A "NORMAL" DEVEL RELEASE. It's no longer | |
| highly experimental. This assertion is based on real-world feedback. | |
| - changes to the strgen module interface | |
| - new output module interface for transactional modules | |
| - performance improvements | |
| * reduced number of malloc/frees due to further changes to the | |
| output module interface | |
| * reduced number of malloc/frees during string template processing | |
| We now re-use once allocated string template memory for as long | |
| as the worker thread exists. This saves us from doing new memory | |
| allocs (and their free counterpart) when the next message is | |
| processed. The drawback is that the cache always is the size of | |
| the so-far largest message processed. This is not considered a | |
| problem, as in any case a single messages' memory footprint should | |
| be far lower than that of a whole set of messages (especially on | |
| busy servers). | |
| * used variable qualifiers (const, __restrict__) to hopefully help | |
| the compiler generate somewhat faster code | |
| - failed action detection more precisely for a number of actions | |
| If an action uses string parameter passing but is non-transactional | |
| it can be executed immediately, giving a quicker indicatio of | |
| action failure. | |
| - bugfix: limiting queue disk space did not work properly | |
| * queue.maxdiskspace actually initializes queue.maxfilesize | |
| * total size of queue files was not checked against | |
| queue.maxdiskspace for disk assisted queues. | |
| Thanks to Karol Jurak for the patch. | |
| --------------------------------------------------------------------------- | |
| Version 8.1.2 [experimental] 2013-11-28 | |
| - support for liblognorm1 added - results in performance improvements | |
| Thanks to Pavel Levshin for his work in this regard. | |
| - support for jemalloc added via --enable-jemalloc | |
| Thanks to Pavel Levshin for suggesting jemalloc | |
| Note that build system is experimental at this stage. | |
| - queue defaults have changed | |
| * high water mark is now dynamically 90% of queue size | |
| * low water makr is now dynamically 70% of queue size | |
| * queue.discardMark is now dynamically 98% of queue size | |
| * queue.workerThreadMinimumMessage set to queue.size / num workers | |
| For queues with very low queue.maxSize (< 100), "emergency" defaults | |
| will be used. | |
| - bugfix: disk queues created files in wrong working directory | |
| if the $WorkDirectory was changed multiple times, all queues only | |
| used the last value set. | |
| - bugfix: legacy directive $ActionQueueWorkerThreads was not honored | |
| - bugfix: mmrfc5424addhmac: "key" parameter was not properly processed | |
| --------------------------------------------------------------------------- | |
| Version 8.1.1 [experimental] 2013-11-19 | |
| - bugfix: STOP/discard(~) was mostly NOT honored | |
| This lead to execution of config code that was not meant to be executed. | |
| - bugfix: memory leak on worker thread termination | |
| - bugfix: potential segfault in omfile under heavy load | |
| Thanks to Pavel Levshin for alerting us. | |
| - bugfix: mmsequence: instance mode did not work | |
| Thanks to Pavel Levshin for the patch | |
| - bugfix: segfault on startup when certain script constructs are used | |
| e.g. "if not $msg ..." | |
| - omhiredis: now supports v8 output module interface and works again | |
| Thanks to Pavel Levshin for the patch | |
| - mmaudit: now supports v8 output module interface and work again | |
| - bugfix: potential abort on startup in debug mode | |
| This depends on template type being used. The root cause was a | |
| non-necessary debug output, which were at the wrong spot (leftover from | |
| initial testing). | |
| Thanks to Pavel Levshin for alerting us and providing a patch | |
| proposal. | |
| --------------------------------------------------------------------------- | |
| Version 8.1.0 [experimental] 2013-11-15 | |
| - rewritten core engine for higher performance and new features | |
| In detail: | |
| * completely rewritten rule execution engine | |
| * completely changed output module interface | |
| * remodelled output module interface | |
| * enabled important output modules to support full concurrent | |
| operation | |
| The core engine has been considerably changed and must be considered | |
| experimental at this stage. Note that it does not yet include all | |
| features planned for v8, but is close to this goal. In theory, the | |
| engine should perform much better, especially on complex configurations | |
| and busy servers. Most importantly, actions instances can now be called | |
| concurrently from worker threads and many important output modules | |
| support multiple concurrent action instances natively. | |
| - module omruleset is no longer enabled by default. | |
| Note that it has been deprecated in v7 and been replaced by the "call" | |
| statement. Also, it can still be build without problems, the option must | |
| just explicitely be given. | |
| --------------------------------------------------------------------------- | |
| Version 7.6.8 [v7.6-stable] 2014-10-?? | |
| - bugfix: typo in queue configuration parameter made parameter unusable | |
| Thanks to Bojan Smojver for the patch. | |
| - bugfix imuxsock: possible segfault when SysSock.Use="off" | |
| Thanks to alexjfisher for reporting this issue. | |
| closes https://github.com/rsyslog/rsyslog/issues/140 | |
| - bugfix: unitialized buffer off-by-one error in hostname generation | |
| The DNS cache used uninitialized memory, which could lead to | |
| invalid hostname generation. | |
| Thanks to Jarrod Sayers for alerting us and provinding analysis and | |
| patch recommendations. | |
| - remove zpipe (a testing tool) from --enable-diagtools | |
| This tool is no longer maintained and currently not used inside the | |
| testbench. We keep it in the source tree for the time being in case that | |
| it may be used in the future. | |
| - bugfix: imjournal did not build properly | |
| The build succeeded, but the module did not load due to a type in | |
| a support function name, which kept unresolved during load. | |
| - bugfix imklog: pri was miscalculated | |
| actually, the pri was totally off the real value for PRIs > 9 | |
| - bugfix rsgtutil: sometimes crashed in verify mode if file did not exist | |
| - bugfix rsgtutil: some errors/problems at end of file were not reported | |
| * The verification function in rsgtutil tool did not report deletion of | |