-
Notifications
You must be signed in to change notification settings - Fork 667
/
Copy pathChangeLog
16894 lines (16854 loc) · 954 KB
/
ChangeLog
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
----------------------------------------------------------------------------------------
Scheduled Release 8.2504.0 (aka 2025.04) 2025-04-??
- 2025-04-16: tcpsrv bugfix: input name was not properly propagated
As a result, it did not show up in pstats. Alos, we now use the input name
in worker thread to easily identify where they belong to. As thread names
have very limited length, the thread naming now is
"w<worker-number>/<input-name>".
Note: this bug was introduced 2025-03-01 (daily stable build)
- 2025-03-06: rsyslog core fix: buffer overflow when the argument to replace() is empty
This fixes the following situation:
We have these expressions in rsyslogd.conf:
set $!rsyslog_FileFormat = exec_template("RSYSLOG_FileFormat")
set $!localheader = re_extract($!rsyslog_FileFormat, "[^ ]+.* +port[0-9]", 0, 0, "");
set $!localpattern = re_extract($!rsyslog_FileFormat, " [^ ]+ +[^ ]+ +port[0-9]", 0, 0, "");
set $!localheader = replace($!localheader, $!localpattern, " ");
We have a message like this arriving.
<30>Feb 24 22:08:21 hostname port03 'label' RXDATA: \n
It was observed that when 2 of these messages arrive in a row, rsyslogd
crashes. This is clearly due to memory corruption, as the crash comes
from within calloc.
Thanks to Lincoln Ramsay for the patch.
- 2025-03-05: several small to cosmetic leak fixes based on Coverity Scan
- 2025-03-04: perctile_stats: do not use uninitialized values on error
Thanks to Attila Lakatos for the patch.
- 2025-03-04: omfwd bufgix: potential segfault in UDP forwarding
A segfault could occur if the configured port could not be bound to the interface.
Thanks to Markel Azpeitia Loiti for the patch.
- 2025-03-01: imtcp: major multithreading and performance improvements
This commit significantly enhances imtcp by introducing a fully
functional worker thread pool, enabling true multi-threaded
processing for better scalability under high loads. This is
particularly beneficial when using TLS connections.
Notable changes:
- Implemented a complete worker pool for imtcp.
- Introduced the `workerthreads` config parameter for tuning
concurrency.
- Improved epoll efficiency by enabling edge-triggered mode.
- Added starvation handling via `starvationProtection.maxReads`.
- Refactored session accept logic and optimized network object
handling.
- Removed an obsolete network driver layer for event notification.
- Fixed multiple issues related to message timing, EPOLLERR
handling, and tests.
- Improved performance in poll() mode by reducing redundant
allocations.
- Introduced new CI tests for imtcp without epoll.
- Allowed disabling imtcp tests via a new configure switch.
- Added new impstats counters for worker thread pool statistics.
Details:
- The worker pool replaces an outdated experimental
implementation.
- If `workerthreads=1`, no worker pool is created to minimize
context switches.
- Moves worker pool variables inside `tcpsrv` instance to
prevent conflicts.
- Extracts session `accept()` logic into a dedicated function
for clarity.
- Fixes message ordering inconsistencies in multi-threaded
scenarios.
- Properly handles `EPOLLERR` notifications to improve error
resilience.
- Optimizes poll() mode by avoiding unnecessary reallocation
of file descriptors.
- Replaces the old network driver layer for event notification
with a streamlined solution.
- Now uses **conditional compilation** to select the best
method (epoll or poll) at build time.
- This significantly reduces code complexity, improves
maintainability, and boosts performance.
- The previous "thread pool" was a rough experiment that did
not perform significantly better than single-threaded mode.
- The **new implementation** allows multiple worker threads
on platforms with `epoll`.
- On non-epoll systems, an optimized **poll() based
single-threaded approach** is used, which is expected to
perform better than the old "thread pool."
- Adds `pthread_setname_np` only when available to improve
portability.
- Fixes test cases that assumed strict message timing, which
was unreliable.
- Reduces test parallelism for TSAN CI runs to prevent
resource exhaustion.
- Moves a test case to `imdiag` to ensure stable execution.
- Provides a new CI environment to verify `imtcp` behavior
without epoll.
- Introduces `--enable-imtcp-tests` configure switch for test
flexibility.
- Improves debug logging and adds better error handling for
worker pool startup.
New configuration parameters:
- `workerthreads`: Defines the number of worker threads for
imtcp. If set to 1, no worker pool is created.
- `starvationProtection.maxReads`: Defines the maximum number
of consecutive reads a worker can perform before being
interrupted to allow other sessions to be processed.
New impstats counters (emitted only when `workerthreads > 1`):
- `runs`: Number of times the worker thread has been invoked.
- `read`: Number of read calls performed by the worker.
For TLS, this includes read/write calls.
- `accept`: Number of `accept()` calls handled by the worker.
- `starvation_protect`: Number of times a socket was sent
back to the queue due to reaching the maximum number of
consecutive requests, ensuring fair scheduling of sessions.
These changes significantly enhance rsyslog’s TCP handling
performance and stability, particularly in high-volume
environments.
Closes #5529, #5532, #5578, #5580.
- 2025-03-01: "fixed" clang static analyzer false positives
- 2025-03-01: fixed clang static analyzer detected very unlikely but possible bugs
----------------------------------------------------------------------------------------
Scheduled Release 8.2502.0 (aka 2025.02) 2025-02-18
- 2025-02-16: gnutls bugfix: Avoid blocking sockets during TLS handshake
When forwarding logs to a TLS server, using a blocking socket
can lead to indefinite waiting during the gnutls_handshake()
call if the server does not respond as expected.
This commit modifies the behavior to use non-blocking sockets,
ensuring that the rsyslog client does not hang indefinitely
waiting for a response.
Thanks to Attila Lakatos for the patch.
- 2025-02-13: CI: add compile check with atomic operations disabled
The platforms that do not have support for atomic operations become
more and more exoctic. Even more so, only Solaris builders now test
that inside the rsyslog CI. To "harden" the CI system against a
potential (temporary) unavailability of Solaris, we now also do
a build where we intentionally disable atomics. This will trigger
build issues on all platforms. This method also permits us to detect
problems quicker, as the solaris builders are slow.
- 2025-02-13: omfwd bugfix: variable was not properly synced across threads
While this could lead to some inefficiency, it should not have caused
any real harm. But with data races it is never sure if more severe
issues occur. However, here only very strange use cases can be
envisioned where this might be the case.
In any case, the issue is now solved.
This also fixes some TSAN CI "flakes".
- 2025-02-12: testbench: prevent flake by better sync of snmp test tool startup
In order to prevent flakes on some environments, we have enhanced the SNMP trap
receiver testing tool so that it indicates successful startup via a pid file, which
now can be used to hold rsyslog start until the test tool is fully up and running.
- 2025-02-12: CI: add Testruns under Ubuntu 24.04
Note: we temporarily disable omamqp1 testing while we search for
the missing dependency.
We also add a special setting that permits us to control if
known-(very)-flaky test shall be run.
closes https://github.com/rsyslog/rsyslog/issues/5546
- 2025-02-11: testbench: Add new version of snmptrapreceiver.py for python 3.11+
The system packages on Ubuntu 24 appear to be broken for python3-pysnmp4.
And when we update the package using
pip install pyasn1 pysnmp --break-system-packages --upgrade
We need adapt the code, so a new version snmptrapreceiverv2.py
has been added which will automatically been chosen if Python is >3.10
closes: https://github.com/rsyslog/rsyslog/issues/5554
- 2025-02-10: ossl netstream driver: allow ephemeral Diffie-Hellman key exchange
Use well known DH parameters that have built-in support in OpenSSL.
From the man page: If "auto" DH parameters are switched on then
the parameters will be selected to be consistent with the size of
the key associated with the server's certificate. If there is no
certificate (e.g. for PSK ciphersuites), then it it will be
consistent with the size of the negotiated symmetric cipher key.
Thanks to Attila Lakatos for the patch.
- 2025-01-28: testbench: reduce flakiness for omhttp-batch-retry-metadata.sh test
The test results were not deterministic. Sometimes it passed, sometimes
it failed. The omhttp-validate-response.py script tried to parse the
json input line by line, but sometimes the message and response fields
are on separate lines, thus it failed to parse. This is the case
for both files used to control the checking.
This patch updates the check program so that it reads in complete
lines when parsing fails, thus removing this condition for flakiness.
HOWEVER, after doing so the test still sometimes fails. This may
point into a problem with the contributed omhttp module. Unfortunately,
I do not have enough insight into that module (nor time) to try to
address the root cause.
closes https://github.com/rsyslog/rsyslog/issues/5439
- 2025-01-27: testbench fix: OpenSSL 3.x depreceated warnings in tcpflood
- 2025-01-27: omfwd: align some parameters with imtcp names
Parameter names have historically evolved and are sometimes
inconsistent between modules (while still having the same
semantics and similiar name).
This commit creates three aliases in omfwd to keep stream
driver parameter names consistent with the names used in
imtcp. Note that we do not create an alias for
"streamdriverpermittedpeers" because the name differ more
considerably and we would also need to create an alias
in imtcp as well. We will do this only on request.
Note: aliases help, but are not a great solution. They may
cause confusion if both names are used together in a single
config. So care must be taken when using an alias.
- 2025-01-27: testbench: remove ElasticSearch 6.0 based tests
EleasticSearch 6 is heavily outdated. Testing against that
platform is no longer needed.
- 2025-01-27: omsnmp: replaced inet_aton with inet_pton for IPv4 address conversion
inet_pton is part of POSIX, whilst inet_aton is not. Moreover the
function is marked as forbidden according to rpmlint:
Forbidden function symbols found: inet_aton
Thanks to Attila Lakatos for the patch.
- 2025-01-27: tcpflood test tool: enable multi threaded connection open
Connections were originally opened sequentially. This commit adds the
capability to open them via multiple threads in parallel.
This is needed at least for cases where we need to test multiple
pending connection requests, which can be accepted with an
"accept() loop".
It can also be useful for speeding up parts of the testbench.
However, the additional concurrency makes it potentially harder
to debug if bugs are detected. So tests may use multiple threads
for connect during regular operations but should also offer the
option (by editing) to revert to a single thread for easier
debugging.
closes https://github.com/rsyslog/rsyslog/issues/5535
- 2025-01-23: imtcp: add "socketBacklog" parameter to configure TCP backlog size
A new "socketBacklog" parameter has been added to the imtcp module, allowing
users to override the default TCP SYN backlog size. Previously, the backlog
was set to roughly 10% of the configured max sessions, which remains the
default if the parameter is not specified. This enhancement enables better
configuration for high-performance servers. The parameter name aligns with
the "socketBacklog" parameter in imptcp for consistency.
The "socketBacklog" parameter should be set based on the anticipated connection
rate and the server's ability to handle incoming connections. For high-performance
environments with heavy traffic, a larger value may be needed to avoid dropped
connections during bursts. If unsure, leave the parameter unset to use the default
(10% of max sessions), which is suitable for typical workloads.
- 2025-01-23: testbench: add new and adapt imtcp tests for large connection counts
Updated the imtcp test suite to set the "SocketBacklog" parameter to a higher
value to accommodate tests with a large number of concurrent connections.
This change addresses potential CI flakes caused by SYN queue overflows during
test runs and significantly reduces test runtime by avoiding delays during
session setup.
The adjusted SocketBacklog parameter ensures smoother handling of high
connection counts, improving test reliability and performance.
- 2025-01-23: imptcp: Increase default SYN backlog for imptcp server to 64
Increased the default TCP SYN backlog for the imptcp server from 5 to 64 to
address potential connection instability and performance issues under high
connection load scenarios. The previous default of 5 was too low for many
workloads, particularly in scenarios involving a rapid influx of client
connections. This could lead to SYN queue overflow, delayed connection
establishment, and eventual RSTs or retransmissions during subsequent phases
of the connection lifecycle.
The new default of 64 provides better handling of typical workloads while
allowing further customization via the "SocketBacklog" configuration option.
Note: The "SocketBacklog" configuration option can be adjusted to handle
specific workloads such as high rates of concurrent connection openings, burst
traffic, or other resource-intensive scenarios.
- 2025-01-23: testbench: adapt imptcp tests for large connection counts
Updated the imptcp test suite to set the "SocketBacklog" parameter to a higher
value to accommodate tests with a large number of concurrent connections.
This change addresses potential CI flakes caused by SYN queue overflows during
test runs and significantly reduces test runtime by avoiding delays during
session setup.
The adjusted SocketBacklog parameter ensures smoother handling of high
connection counts, improving test reliability and performance.
- 2025-01-16: netstrm TLS driver: move TLS retry handling to main driver
This simplifies the driver interface. It is also kind of prep-work
for implementing real multi-threading support for imtcp (to be done
in a separate set of commits).
see also https://github.com/rsyslog/rsyslog/issues/5532
- 2024-12-31: build: support for compile with -std=gnu23 gcc option
Note: The upcoming gnu23 C standard is overdoing it with type-safety. Inside
rsyslog, we historically have method tables for generic calls, which
keeps the code small and easy to understand. This would not decently be
possible with the new type-safety requirements.
So this commit works around these warning in a way that pretends to
provide more type safety. We have done this in the least intrusive
way to reduce the risk for regressions in code that works well in
since decades. Also note that the code already does parameter
validation.
There would have been more elaborate ways to make gnu23 compile happy,
e.g. by using a union of structs to provide the data element. Some folks
consider this type safe. In reality, it is not a bit better than
traditional C without types at all. Because the caller still needs to
ensure it picks the right struct from the union. As this approach
would also have larger regeression potential, we have not used it.
Right now, we have suppressed some of the new warnings, as working
around them would have required an even larger time budget and
potentially larger regression potential. In the long term we may
want to look into enabling them, as they would potentially be
beneficial for new code not involving method tables.
Some nits, however, were detected and have been fixed.
- 2024-12-04: omazureeventhubs bugfix: URL escaping
Some characters were not properly escaped, leading to malfunction on submission.
----------------------------------------------------------------------------------------
Scheduled Release 8.2412.0 (aka 2024.12) 2024-12-03
- 2024-12-02: rainerscript bugfix: do not try to call a function if it does not exist
This could previously cause a segfault when the function was called.
Thanks to Attila Lakatos for the patch.
- 2024-12-02: Fix legacy $ActionQueueDiscardMark parameter
If the $ActionQueueSize legacy parameter was configured
with a much value higher than the default, the queueDiscardMark
option was not automatically adjusted to represent 98% of
the actual queue size. This caused a misalignment issue,
which does not occur when using the RainerScript syntax.
Thanks to Attila Lakatos for the patch.
Note: We usually do NOT change legacy parameters, but in this case it makes
sense to adjust to recent developments.
- 2024-11-25: improvements in CI system
added some support for Ubuntu 24.04 - stil more work to do
- 2024-11-25: small nitfixes
mostly to address nits detected by newer compilers due to CI-system modernization
- 2024-11-19: core bugfix: potential stability issue with corrupt queue file resolved
Thanks to Flos Lonicerae for the patch.
- 2024-11-19: core bugfix: prevent segfault on use of $ControlCharacterEscapePrefix
When this obsolete legacy directive was used, rsyslog most probably segfaulted during
startup. This did not affect it's modern counterpart.
Thanks to Flos Lonicerae for the patch.
- 2024-11-14: core bugfix: rsyslog could segfault if immediately stopped during startup
systemd restart rsyslog in the early start of OS will let rsyslog segmentation
fault. This cmd will send sigTerm to rsylogd, and rsyslogd will handle the
signal in rsyslogdDoDie. If the rsyslogd havn't parse the conf, the runConf
will be NULL So check the pointer before reference it.
see also: https://github.com/rsyslog/rsyslog/issues/5453
thanks to Wang Haitao for the patch.
- 2024-11-14: imhiredis cleanup: fix incorrect variable definitions and missing prototype
Thanks to Théo Bertin for the patch.
- 2024-11-14: config processing bugfix: typo and missing space when file can't be accessed
When using the backtick feature with cat and a file that does not exist,
an error message is placed where the file content would be. This error
message contained a typo in could as well as a missing space between
see and error, which have now been fixed.
Thanks to Tobias Kantusch for the patch.
- 2024-11-14: build fix: resolved issue when building on musl
Thanks to Quincy Fleming for the patch.
- 2024-11-14: testbench bugfix: typo caused invalid test file to be used
... which went unnoticed during regular make check runs, but lead to
"left-over files after distclean" after make distcheck.
This could also potentially be responsible for some flakes that were
not fully explainable.
- 2024-11-14: nsd_ptcp regression fix: remove debugging messages emited to stderr
fix regression introduced by 9ac56b286. This spits out a debug message
to stderr. That message is removed by this patch here.
closes https://github.com/rsyslog/rsyslog/issues/5485
----------------------------------------------------------------------------------------
Scheduled Release 8.2410.0 (aka 2024.10) 2024-10-22
- network subsystem: improve connection failure error message
If we try to connect via TCP and the connections fails, we now
tell inside the error message how long the connection attempt
took. This is useful to find out if targets connect very
slowly.
- regression fix: build issue on Solaris 10
Solaris 10 does not support SO_REUSEPORT, which we need for some
tests. It is used in minitcpsrvr.c, and will now not be used if
not available.
Note that tests requiring that option will also need to be disabled
for that platform.
commit which introduced regression: 1c0f9bb
- testbench: correctly apply socket options in minitcpsrvr
The option name in setsockopts is not a bitmask, so SO_REUSEADDR and
SO_REUSEPORT can't be ORed together. Instead apply the options via
separate calls.
Fixes: #5456
Thanks: Chris Hofstaedtler <zeha@debian.org> and Michael Biebl
----------------------------------------------------------------------------------------
Scheduled Release 8.2408.0 (aka 2024.08) 2024-08-20
- 2024-08-16: omfwd: implement native load balancing - phase 1
This patch implements a simple round-robin load balancer
for omfwd. It provides equal distribution of load to a pool
of target servers.
The code currently has no different modes and no special tuning
for the load balancer. However, it works very well in the most
common use cases. Furthermore, it provides a solid base on which
more elaborate functionality can be build if there is need to.
The new functionality is fully backwards compatible with previous
configuration settings.
New action() config params:
* pool.resumeinterval
New/"changed" rstats counters
Each target receives its own set of pstats counters. Most
importantly this is the case for byte counts. That counter retains
the same naming, but there may now be multiple of these counters,
one for each target ip, port tuple.
New pstats message count to target
Among others, this can be used for checking that the load balancer
works as intended. The so-far byte count emitted does not provide
a clear indication of how many messages the targets had actually
processed.
For obvious reasons, this message count makes most sense in
advanced load balancing scenarios, but also provides additional
insight into round-robin. Non-matches indicate that targets
went offline, and we can now evaluate the impact this had
on processing.
- re-design rebind functionality
This now works at the transaction level. It causes a rebind of all
pool members. Previous code did not work 100% correct since for a
couple of years now (after output batching integration).
As cleanup, rebindInterval support has been removed from tcpClt,
because omfwd is the only user. This permits a cleaner code path.
We also noticed a bug with rebindInterval: it caused some mild
message duplication for quite some time. This went unnoticed.
To address that efficiently, rebindInterval in the future will
be considered once per batch. That means up to (maxBatchSize - 1)
messages may be transmitted more than the rebindinterval is.
That's the cleanest mode of operation and should not make any
difference for real deployments.
Some additional work done in this commit:
- netstream: harden component against upper-layer logic errors
- network subsystem: better handle API errors and provide more info
- omfwd: add new parameter "iobuffer.maxsize"
- add new global parameter debug.abortoninternalerror and use it
- This parameter permits to make test runs fail when an internal error
- is detected and gracefully handled by rsyslog. While it is great to
have it gracefully handled in practice, we should not accept this
during testing. The new parameter permits to abort in this case and
emits the related error message beforehand. It is turned on by
default in our regular tests.
- add dedicated error code for "hard" program errors
- omfwd: some cleanup + error message fix + new debug level messages
- imptcp: improve error messages
- add omfwd option to NOT do extended connection check
- also output wrkr id in some omfwd messages (primarily debugging aid)
- better debug info via LogMsg() interface
- improve messages regarding imptcp and omfwd suspension / thread IDs
- refactor and enchance minitcpsrvr for mimicing died servers
- new global (debugging) option, correction of an informational msg
- add global option allmessagestostderr
- add new tests
- 2024-08-16: omkafka enhance: write Kafka log with level ERROR or higher into errorfile
closes: https://github.com/rsyslog/rsyslog/issues/5425
- 2024-08-16: net subsusytem bugfix: some config statemens were not accepted
The following parameters were not always excepted, even though they were documented
and the supporting code was also present. This has been corrected:
* streamdriver.cafile
* streamdriver.crlfile
* streamdriver.keyfile
* streamdriver.certfile
Thanks to Qiumiao Zhang for the fix.
- 2024-08-13: new "ossl" crypto provider
Thanks to Attila Lakatos for the patch.
- 2024-08-05: imtcp: improve connection error messages
They now contain remote peer information whereever possible.
----------------------------------------------------------------------------------------
Scheduled Release 8.2406.0 (aka 2024.06) 2024-07-02
- 2024-07-01: ompgsql bugfix: unexpected and unnecessary message loss
In case of PG not available lets retry instead of dropping messages
Thanks to Nickolai Novik for the patch.
- 2024-07-01: tls subsystem/ossl driver
Several small fixes/additions
* output all loaded ciphers and engines into debug log
* Add new global option "defaultopensslengine" to customize the default openssl
engine. If not defined, openssl will handle the default engine automatically.
Sample: global(defaultopensslengine="rdrand")
* Add simple openssl performance test (imtcp-tls-ossl-basic-stress.sh) with
defaultopensslengine set to rdrand (Intel).
* removed unneeded testcase files in runtime folder.
- 2024-04-18: [o|i]mprog/mmexternal bugfix: invalid command line parameter passing
Quoted command line parameters were incorrectly passed to the called program. This
resulted in unexpected behaviour. For example, in
bash -c "echo test"
bash got passed parameters '-c" and '"echo test"'. Note the double quotes in
'"echo test"'. These needed to be removed by Unix/Linux standards but were instead
passed to bash. That in turn resulted in invalid execution. With the fix, the
behaviour is now correct. Now, bash gets passed '-c' and 'echo test'.
This actually is a BREAKING CHANGE. However, it was outright wrong behaviour from
the beginning. We assume that people either never noticed it (because they did not
use quoted parameters) or used some workaroud, likely a "starter script", which
performed the right task. As such, we do expect that only a very limited set of
installations might be affected by the change.
Even more important, we would need to change the default behaviour in any case,
because the previous handling was obviosuly unacceptable. As such, there was no
way to keep rsyslog perform the previous action.
Thanks to Paul Fertser for the patch.
- 2024-04-12: bugfixes related to time_t on 32 bit platforms (y38k isue)
Thanks to Michael Biebl for the patch.
----------------------------------------------------------------------------------------
Scheduled Release 8.2404.0 (aka 2024.04) 2024-04-02
- 2024-04-02: omhttp patches and enhancement
New omhttp plugin configuration parameters added:
* restpathtimeout - configures a timeout value for an omhttp restpath, and retry again
* httpretrycodes (list) - configurable list of HTTP status codes that should be
retried by omhttp plugin. The default behavior is to retry any non 2xx status code.
* httpignorablecodes (list) that can specify HTTP status codes that should be ignored
(as failures). status codes specified as ignorable code will not be retried.
* proxyhost, proxyport - configures a proxy, for which omhttp can use to set up a
HTTP proxy tunnel connection. Also works by reading environment variable if
configured HTTP_PROXY. For more details see libcurl docs on CURLOPT_PROXY
thanks to @erenwh for the contribution!
* retry.addmetadata - when this option is enabled, omhttp will add the
response metadata to: $!omhttp!response. There are 3 response metadata added:
code, body, batch_index.
More statistics omhttp curl statistic counters:
* requests.count
* requests.status.0xx
* requests.status.1xx
* requests.status.2xx
* requests.status.3xx
* requests.status.4xx
* requests.status.5xx
* requests.bytes
* requests.time_ms
No longer discard 3xx, 4xx errors
see also https://github.com/rsyslog/rsyslog/issues/4636
Thanks to github user n2yen for the patches.
- 2024-04-02: remove CAP_IPC_LOCK capability
Does not seem to be necessary.
Thanks to Michael Biebl for the patch.
- 2024-03-28: TLS bugfix: TLS drivers did not properly load under some conditions
This was a regeression from the DTLS implementation
- 2024-03-28: mmdblookup bugfix: do not suspend on incompatible IP lookup
Mmdblookup module used to suspend after erroring on an IPv6 IP lookup on IPv4-only
DBs. The suspension of the module is now replaced by a simple log message,
allowing it to keep working for future lookups.
Thanks to Théo Bertin (frikilax) for the patch.
- 2024-03-28: tarball fix: dist tarball was missing a textbench file
This made the testbench fail.
----------------------------------------------------------------------------------------
Scheduled Release 8.2402.0 (aka 2024.02) 2024-02-27
- 2024-02-26: add DTLS support
This version comes with the initial implementation of imdtls and omdtls.
These modules permit secure message exchange over UDP.
- 2024-02-26: testbench: make omusrmsg-noabort test more reliable
The previous test did not always detect an abort of rsyslog/omusrmsg.
The detection method has now been improved, so it is far more
probable that an abort is detected.
While doing this, we noticed that the omusrmsg-noabort-legacy test was
now a 100% duplicate. There is no need any longer to check pure legacy
syntax, and so that test has been removed.
We also added a valgrind-based test ofr omusrmsg-noabort, which furthers
strengthens bug detection. Most importantly, it helps us to detect
potentially new memory leaks on all CI platforms (in case the lib
behaves differently depending on os/distro).
see also https://github.com/rsyslog/rsyslog/issues/5294
- 2024-02-26: omusrmsg bugfix: potential double free, which can cause segfault
omusrmsg frees a string which points to OS/system library memory. When
the os/libs clean up, it frees the memory as well. This results in a
double free. This bug interestingly seems to go unnoticed in many cases.
But it can cause a segfault or hard-to-trace memory corruptions which
could lead to other problems later on. The outcome of this bug most
probably depdns on os/library versions.
closes https://github.com/rsyslog/rsyslog/issues/5294
- 2024-02-26: ommysql bugfix: potential segfault on database error
Due to an invalid code path, ommysql may cause a segfault if database
transactions fail into a specific way. The main trigger is a totally
irrecoverrable database error which can lead to premature connection
close, which is not checked for in all recover code.
This was detected in a setting where a stored procedure is called that
rolls back a transaction in itself.
This patch fixes the issue.
closes https://github.com/rsyslog/rsyslog/issues/5288
- 2024-02-26: omfile: do not carry out actual action when writing to /dev/null
In some use cases omfile is configured to write to /dev/null. This seems
primarily be done because of statistics gathering but maybe some other
scenarios. We now add conditional logic to not do any actual omfile
action when the target file is /dev/null.
Note: this check only works on static file names. When /dev/null is
evaluated as part of dynafile, it will be handled just in the regular
case like before this patch.
----------------------------------------------------------------------------------------
Scheduled Release 8.2312.0 (aka 2023.12) 2023-12-12
- 2023-12-11: imjournal: Add new input module parameter 'defaulttag'
The DefaultTag option specifies the default value for the tag field.
In imjournal, this can happen when one of the following is missing:
* identifier string provided by the application (SYSLOG_IDENTIFIER)
* name of the process the journal entry originates from (_COMM)
Thanks to Attila Lakatos for the patch.
- 2023-12-08: core bugfix: rsyslog messages may not always have FQDN
Even if hostname FQDN is configured, rsyslog internal messages generated
after rsyslog startup and before the first HUP will not necessarily have
FQDN but instead only the shortname of the local host. This commit
fixes the situation.
Special thanks to github user eciii for doing a great bug analysis
and helping us considerably to fix the issue.
closes https://github.com/rsyslog/rsyslog/issues/5218
- 2023-12-08: omlibdbi regression fix: database path was not properly used
Commit 4a072d6c93015a63716c49a6c7756df22750086a caused a regression that made
the database path unreliable to use. Depending on platform/libc version the
basename was improperly extracted, which made access to the database of sqllite
impossible.
Thanks to Flávio Tapajós for the patch.
closes: https://github.com/rsyslog/rsyslog/issues/5282
- 2023-12-06: mazureeventhubs: Corrected handling of transport closed failures
- Added test for connection interrupts (requires root)
- Corrected handling of PN_TRANSPORT_CLOSED.
- Make sure Connection is being reestablished trough tryResume
- Enhanced Debug log output
closes: https://github.com/rsyslog/rsyslog/issues/5269
- 2023-11-24: imkmsg: add params "readMode" and "expectedBootCompleteSeconds"
These parameters permit to control when imkmsg reads the full
kernel log upon startup.
Parameter "readMode" provides the following options:
* full-boot - (default) read full klog, but only "immediately" after
boot. "Immediately" is hereby meant in seconds of system
uptime given in "expectedBootCompleteSeconds"
* full-always - read full klog on every rsyslog startup. Most
probably causes messag duplication
* new-only - never emit existing kernel log message, read only
new ones.
Note that some message loss can happen if rsyslog is stopped
in "full-boot" and "new-only" read mode. The longer rsyslog is
inactive, the higher the message loss probability and potential
number of messages lost. For typical restart scenarios, this
should be minimal. On HUP, no message loss occurs as rsyslog
is not actually stopped.
The default value for "expectedBootCompleteSeconds" is 90.
see also https://github.com/rsyslog/rsyslog/issues/5161
- 2023-11-10: imkmsg: add module param parseKernelTimestamp
The parameter permits to select whether or not and when kernel
timestamps shall parsed, that is be used as the actual time a
log message occurs.
This permits to work around problems with the way kernel
timestamps are represented. The reasoning is given in a sysklogd
commit by Joachim Wiberg, which we reproduce below ("QUOTE") to
have a stable reference.
The commit itself can be found for example at:
https://github.com/troglobit/sysklogd/commit/9f6fbb3301e571d8af95f8d771469291384e9e95
The new parameter parseKernelTimestamp has three possible modes:
"startup" - uses the kernel time stamp during the initial read
loop of /dev/kmsg, but replaced it later ignores it for later reads.
This is the DEFAULT setting.
"on" - kernel timestamps are always used and no correction is tried
"off" - kernel timestamps are never used, system time is used instead
Note that there this is a slightly breaking change. Previously, imkmsg
reported similar to "off" mode, now it reports by default in "startup"
mode. We consider this acceptable, as "off" mode timestamps are not
correct for startup. After startup, the behaviour is correct. All in
all, the new default is kind of a bugfix.
============== QUOTE ===============
The spec[1] says the /dev/kmsg timestamp is a monotonic clock and in
microseconds. After a while you realize it's also relative to the boot
of the system, that fact was probably too obvious to be put in the spec.
However, what's *not* in the spec, and what takes a while to realize, is
that this monotonic time is *not* adjusted for suspend/resume cycles ...
On a frequently used laptop this can manifest itself as follows. The
kernel is stuck on Nov 15, and for the life of me I cannot find any to
adjust for this offset:
$ dmesg -T |tail -1; date
[Mon Nov 15 01:42:08 2021] wlan0: Limiting TX power to 23 (23 - 0) dBm as advertised by 18:e8:29:55:b0:62
Tue 23 Nov 2021 05:20:53 PM CET
Hence this patch. After initial "emptying" of /dev/kmsg when syslogd
starts up, we raise a flag (denoting done with backlog), and after this
point we ignore the kernel's idea of time and replace it with the actual
time we have now, the same that userspace messages are logged with.
Sure, there will be occasions where there's a LOT of kernel messages to
read and we won't be able to keep track. Yet, this patch is better than
the current state (where we log Nov 15).
[1]: https://www.kernel.org/doc/Documentation/ABI/testing/dev-kmsg
===========END QUOTE ===============
closes https://github.com/rsyslog/rsyslog/issues/4561
closes https://github.com/rsyslog/rsyslog/issues/5161
- 2023-11-07: imfile bugfix: remove state file on file delete
The state file would remain in the working directory
after shutdown, even though deleteStateOnfileDelete is
set to "on" and the monitored file was removed.
closes https://github.com/rsyslog/rsyslog/issues/5258
Thanks to Attila Lakatos for the patch.
- 2023-10-31: TLS subsystem: fix small memory leak on startup
This was a one-time leak of the file name that hapened if a certificate file
was not accessible. It had no operational issues, but could confuse automatted
testing. As not only a side-effect, certificate load failures are now somewhat
more verbosely reported, which we consider helpful to the user.
Thanks to Attila Lakatos for the patch.
- 2023-10-31: imklog bugfix: keepKernelTimestamp=off config param did not work
... at least not as expected. It was only honored for kernel-level
messages and only when parseKernelTimestamp was "on". Otherwise, the
kernel timestamp was always kept inside the message.
closes https://github.com/rsyslog/rsyslog/issues/5160
- 2023-10-26: TLS subsystem: add remote hostname to error reporting
This provides richer and easier to process logs for error and warning
cases. One goal is to enable automatic operations without the need
to consolidate multiple message to a single information.
This improves one situation in gtls driver and provides a more
generic approach in ossl driver for OpenSSL error reporting.
There is probably still room for improvement, however this patch
is at least a good starting point for further work. Please
provide feedback if you need more!
closes https://github.com/rsyslog/rsyslog/issues/5244
- 2023-10-24: imjournal: add the ability to run multiple journal inputs
This may be useful to de-couple journal processing.
Thanks to Willy Tu for the patch.
- 2023-10-24: regression fix: forking rsyslogd on BSD did not work
Actually, this was an issue for all platforms that do not provide open file handle
detection via the /proc file system.
Tech details: After fork if the child process uses close_range to close open file
descriptors it has no way to exempt the parentPipeFD causing a failure to signal
successful startup to the parent process. This causes failures on all systems that
aren't Linux that implement close_range.
Thanks to Nathan Huff for the patch.
- 2023-10-24: omusrmsg: use logind instead of utmp for wall messages with systemd
Future SUSE versions will get rid of utmp due to a 32bit time_t counter
overflow in 2038.
See details at:
https://github.com/thkukuk/utmpx/blob/main/Y2038.md
On systemd based systems logind is an alternative to utmp.
Thanks to github user tblume for the patch.
- 2023-10-24: cleanup: rm no longer used --with-systemdsystemunitdir configure switch
This is a clean up following the removal of the service unit in
cfd07503ba055100a84d75d1a78a5c6cceb9fdab
- 2023-10-23: testbench: bump zookeeper version to match current offering
Older version can no longer be downloaded. It also makes sense to
test with mainstream version.
----------------------------------------------------------------------------------------
Scheduled Release 8.2310.0 (aka 2023.10) 2023-10-10
- 2023-10-04: Add CAP_NET_RAW capability due to the omudpspoof module
The CAP_NET_RAW ensures the use of RAW and PACKET sockets,
which is utilized by the omudpspoof module, more precisely
the libnet_init function.
Thanks to Attila Lakatos for the patch.
- 2023-10-04: Add new global config option "libcapng.enable"
Defines whether rsyslog should drop capabilities at startup or not.
By default, it is set to "on". Until this point, if the project was
compiled with --enable-libcap-ng option, capabilities were
automatically dropped. This is configurable now.
Thanks to Attila Lakatos for the patch.
- 2023-10-04: tcp net subsystem: handle data race gracefully
It may happen that a socket file descriptor has been closed either
while setting up poll() et al or while being inside the system call.
This was previously treated as error and caused abort in debug
builds. However, it was essentially ignored in production builds.
This has now been fixed and now is always gracefully ignored. This
most importantly fixes some flakes in CI runs (which were caused
by this situation).
- 2023-09-29: imrelp bufgifx: avoid crash on restart in imrelp SIGTTIN handler
While existing, if at specific time rsyslog receives a SIGTTIN, it
crashes due to 2 issues.
1. debug.unloadModules="off" a double free of pRelpEngine
2. debug.unloadModules="on" it crashes because the signal handler has
been unmapped from memory.
This patch covers both issues.
Thanks to Ali Abdallah for the patch.
- 2023-09-28: fix startup issue on modern systemd systems
When we startup AND are told to auto-background ourselfs, we must
close all unneeded file descriptors. Not doing this has some
security implications. Traditionally, we do this by iterating
over all possible file descriptor values. This is fairly compatible,
because we need no OS-specific method. However, modern systemd configs
tend to not limit the number of fds, so there are potentially 2^30(*)
fds to close. While this is OKish, it takes some time and makes
systemd think that rsyslog did not properly start up.
We have now solved this by using the /proc filesystem to obtain our
currently open fds. This works for Linux, as well as Cygwin, NetBSD,
FreeBDS and MacOS. Where not available,and close_range() is available
on the (build) platform, we try to use it. If that fails as well, we
fall back to the traditional method. In our opionion, this fallback
is unproblematic, as on these platforms there is no systemd and in
almost all cases a decent number of fds to close.
Very special thanks go out to Brennan Kinney, who clearly described
the issue to us on github and also provided ample ways to solve it.
What we did is just implement what we think is the best fit from
rsyslog's PoV.
(*) Some details below on the number of potentially to close fds.
This is directly from a github posting from Brennan Kinney.
Just to clarify, by default since systemd v240 (2018Q4), that
should be `1024:524288` limit. As in the soft limit is the expected
`1024`.
The problem is other software shipping misconfiguration in systemd
services that overrides this to something silly like
`LimitNOFILE=infinity`.
- Which will map to the sysctl `fs.nr_open` (_a value systemd
v240 also raises from `2^20` to 2^30`, some distro like Debian are
known to opt-out via patch for the `fs.nr_open` change_).
- With the biggest issue there being that the soft limit was also
set to `infinity` instead of their software requesting to raise
the soft limit to a higher value that the hard limit permits.
`infinity` isn't at all sane though.
- The known source of this misconfiguration is container software such
as Docker and `containerd` (_which would often sync with the
systemd `.service` config from the Docker daemon `dockerd.service`_).
closes https://github.com/rsyslog/rsyslog/issues/5158
- 2023-09-13: Add the 'batchsize' parameter to imhiredis
Parameter set to allow configuring the amount of entries imhiredis debatches at once.
Default value of '10' has been kept to avoid any side effect on existing
configurations.
Thanks to Jérémie Jourdin for the patch.
- 2023-09-13: omprog bugfix: Add CAP_DAC_OVERRIDE to the bounding set
The omprog module uses the execve() function to execute
a third party program. Some required capabilities were not
preserved in the bounding set [1]. This caused problems, e.g.
the program could not write to files even if rsyslog was
executed as root and privileges were not dropped. As of now,
only the CAP_DAC_OVERRIDE capability is added to the bounding
set. Others could be added later, if there is justification
behind that.
[1] The capability bounding set is a security mechanism that
can be used to limit the capabilities that can be gained
during an execve(2). During an execve, the capability
bounding set is ANDed with the file permitted capability
set, and the result of this operation is assigned to the
thread's permitted capability set. The capability
bounding set thus places a limit on the permitted
capabilities that may be granted by an executable file.
Thanks to Attila Lakatos for the patch.
- 2023-09-13: tcpflood bugfix: plain tcp send error not properly reported
The error code when plain tcp sending failed was improperly returned,
resulting in no meaningful error message.
Note: tcpflood is a testbench tool, not part of production rsyslog.
----------------------------------------------------------------------------------------
Scheduled Release 8.2308.0 (aka 2023.08) 2023-08-15
- 2023-08-07: crypto subsystem bugfix: potential undefined behaviour
The is some potential undefined behaviour when initializting the IV for locally
encrypting log files. The issue cancels itself out, but at least causes
some confusion when using undefined behaviour sanitizer (UBSAN). However,
UBSAN seems not to detect the issue on all platforms and/or in all versions
(we were not able to reproduce this issue in our CI).
Please also note that the functionality where this can happen is extremely
rarely being used.
Thanks to Jeffrey Walton for providing the patch.
- 2023-08-02: lookup tables: fix static analyzer issue
If something goes really wrong, a lookup table's name would not
be set. That could lead to a NULL pointer access. HOWEVER, this
would require serious bugs in config parameter parsing, as the
lookup table name is a required parameter and the parser will
error out if not set.
So the bug is mostly cosmetic - but it does not hurt to handle
this case, of course.
- 2023-08-02: lookup tables bugfix: reload on HUP did not work when backgrounded
Lookup tables were only reloaded on HUP if the -n option was given
and rsyslog no backgrounded. This patch fixes the issue.
closes: https://github.com/rsyslog/rsyslog/issues/4813
- 2023-07-30: testbench: make test more reliable
There was a race between tcpflood and rsyslog in imptpc_maxsessions.sh.
We now use the new -A tcpflood option to make the timing more
predictable, hopefully fixing test flakiness.
Note: if that does not help, we need to introduce a wait on the number
of error messages and maybe a delay before tcpflood termination. The
theory behind the latter is that rsyslog possibly does not fully
iniaitlize session which are quickly aborted before rsyslog receives
the related OS notification! We just record this info in case we
need it and are positive that this change will fix the situation.
- 2023-07-28: openssl: make connection setup more reliable by use of newer lib feature
Replaced depreceated method SSLv23_method with TLS_method.
In OpenSSL 1.1.0 and higher, SSLv23_method causes some errors
in TLS handshake from time to time. As this method is depreceated
since 1.1.0, I have replaced it with the follow up method
TLS_method which is the most generic one.
It fixes the random test failures in tests like
- sndrcv_tls_ossl_anon_rebind.sh
Also added some debug output in OpenSSL error handling, which is
useful when analysing debug files.
closes: https://github.com/rsyslog/rsyslog/issues/5201
- 2023-07-28: testbench improvement: define state file directories for imfile tests
Not all imfile tests have state file directories or a global working
directory defined. This results in usage of the default location.
While state file names should be sufficiently different, there is still
some riks of using the same name in different tests. That becomes
problematic if tests are run in parallel (and they are run in
parallel inside the regular CI).
NOTE: NOT YET COMPLETED FOR ALL TESTS! We are considering if it makes
sense to deliberately keep some as-is.
- 2023-07-28: tcpflood bugfix: TCP sending was not implemented properly
Note: tcpflood is a testbench tool. This bug could lead to testbench
false positives. No way it can affect production deployments.
The tcpflood tool did improperly assume that a TCP sendto() call
would send messages of any size in a single shot. This is not the
case. It has now been corrected to proper behavior.
As a side-activity, some int variables which acutally needed to be
size_t have been fixed as well.
- 2023-07-28: testbench: make waiting for HUP processing more reliable
The previous approach was more or less delay based. We have now
changed the code to enable imdiag to detect if HUP is underway
and wait until it is completed. The new method still employs some
kind of timeout, but is now quite reliable. Most importantly,
it works great with long-running HUP processing, which can happen
e.g. when querying the system name takes long or some actions need
longer time to persist their HUP processing.
The new approach will most likely reduce CI flakes and also speed
up testbench runs. The speedup happens from not having to wait a
full delay in cases where we detect HUP is completed (plus reduced
timeout when we cannot clearly detect this - see code comments why
the new method is still considered more reliable than the old one).
Code note: we needed to slightly re-structure the way actual HUP
processing and the "HUP mutex" is handled. After best analysis,
this does not affect the reliability or speed in production
settings.
closes https://github.com/rsyslog/rsyslog/issues/5192
- 2023-07-27: build system: make rsyslogd execute when --disable-inet is configured
This option is mostly useless, as network functionality depends on the
modules loaded by the config. The only real, and important, effect it
has is to control auto-load of omfwd - a feature almost all installations
depend in (backward compatibility).
This has been clarified in ./configure -help
Also, when --disable-inet is given, rsyslog now executes successfully.
The reason for the abort was that previously building of the lmnet
component was prevented, but that component is also needed by rsyslog
startup itself to query its own (correct) hostname.
Note that --disable-inet still does not compile some networking
libraries. So do not use it if you intend to load standard networking
modules like omfwd, imtcp or imudp.
closes https://github.com/rsyslog/rsyslog/issues/5188
- 2023-07-26: testbench/CI: update zookeper download to newer version
Old version is no longer available.
- 2023-07-24: openssl: add support for new-version init function
- 2023-07-07: add CRL support for network (TLS) drivers
Thanks to Darren J Moffat for implementing the OpenSSL part.
- 2023-07-07: omazureeventhubs: Initial implementation of new output module
The output module uses Apache "Qpid Proton C API" which is a solid
AMQP protocol library implementation that can be integrated
very well into the rsyslog dev environment.
- Implemented Delivery with submitted and accepted state checking
- saving of failed messages in a failed list with support of saving
and restoring.
- Add testcases (requires ENV variables) to testbench
- Using application/octect-stream (binary) to send messages based on
Microsoft Code Sample:
https://docs.microsoft.com/en-us/azure/event-hubs/event-hubs-c-getstarted-send
* Note original Microsoft Samplecode is not working anymore, we are using
* QPID Proton Proactor based on
https://github.com/apache/qpid-proton/blob/main/c/examples/send.c
- requires QPID-PROTON Version 0.13 or higher because of the proactor API
- Add EventProperties configuration parameters
- Slow down when sender credit reaches zero (10ns).
- Add support for static library linking of qpid-proton
This is needed to build the module from source and remove
library package dependencies.
- adjusted valgrind suppressions
- 2023-07-04: core bugfix: action.resumeintervalmax parameter was not respected
Unfortunately, defining action.resumeintervalmax in the configration
did not have any effect at all. Instead, the default value was used,
which is 1800. This was caused by not having all the letters in
lower-case.
Fixes https://github.com/rsyslog/rsyslog/issues/5132
Thanks to Attila Lakatos for the patch.
- 2023-06-29: core bugfix: do not try to drop capabilities when we don't have any
In case the process does not have any capabilities, e.g. running as regular user then
we do not have to force capability dropping. The capng_have_capabilities() returns
none if that's the case.
Fixes https://github.com/rsyslog/rsyslog/issues/5091
Thanks to Attila Lakatos for the patch.
- 2023-06-29: imhiredis bugfix: Restore compatiblity with hiredis < v1.0.0
RESP3 protocol wasn't implemented yet, some types weren't
available (REDIS_REPLY_DOUBLE)
Thanks to Théo Bertin (frikilax) for the patch.
- 2023-06-23: testbench: use newer zookeeper version in tests
- 2023-06-23: build system: more precise error message on too-old lib
When libcap-ng was enabled, the lib was present but did not meet the minimum version
dependency during configure, it was reported as "missing". We now emit a message
telling that it is present, but the version too old.
----------------------------------------------------------------------------------------
Scheduled Release 8.2306.0 (aka 2023.06) 2023-06-20
- 2023-06-19: mmnormalize bugfix: if msg cannot be parsed, parser chain is stopped
When an parser is not able to parse a message, it should indicate this
to rsyslog core, which then activates the next parser(s) inside the
configured parser chain.
Unfortunatley, mmnormalize always tells core "success", and so no
other parsers are activated.
closes https://github.com/rsyslog/rsyslog/issues/5148
- 2023-06-19: [i/o]mhiredis: various fixes and enhancements
please see the change log for details. Among others, suspending of the modules
has been fixed. Also a new "stream" mode has been added.
Thanks to Théo Bertin (frikilax) for the patch.
- 2023-06-19: testbench/bug: mmexternal-SegFault-empty-jroot-vg.sh fails due to typo
Fix the typo that makes the test fail.
Thanks to Paul Fertser for the patch.
- 2023-06-16: imjournal: Add FileCreateMode module parameter
FileCreateMode allows to set the default file mode bits
when creating new files. As of now, it has only impact on the state file.
Add test suite as well.
Minor indentation fix in run_journal.yml
Thanks to Attila Lakatos for the patch.
- 2023-06-16: core bugfix: potential segfault on busy systems
This was discovered by Konstantin J. Chernov in a practicaly deployment.
Here, msg object tag processing caused sporadic segfaults. We did not
hear from similiar cases, but there clearly is potential for problems
because a mutex lock had insufficient range, thus leading to a potential
race.
The patch is directly from Konstantin J. Chernov, thanks for that.
Please note that the mutex lock could be minimized as it is not strictly
needed for the pM == NULL case, but this cause is extremely exotic
and the resulting code would be harder to understand. Thus we opt
to do the locking on funtion level (as usual).
Descriptiond edited by Rainer Gerhards
closes: https://github.com/rsyslog/rsyslog/issues/5110
- 2023-06-16: Add new global config option "libcapng.default"
Defines how rsyslog should behave in case something went wrong
when capabilities were to be dropped. Default value is "on",
in which case rsyslog exits on a libcapng related error.
Thanks to Attila Lakatos for the patch.
Closes https://github.com/rsyslog/rsyslog/issues/5096
- 2023-06-05: imfile bugfix: file handle leak, primarily in kubernetes context
At this point there is a code imfile.c#L919 that adds an inotify observer to the
parent of the symbolic link target. But there is no such code that removes this
observer in the case when inotify events do not occur in the directory tree above.
This may be if the directory tree of the symbolic link target and the directory tree
of the symbolic link itself are divided into different subtrees somewhere at the levels
above.
For example, in the rsyslog configuration, an imfile with the
template /var/log/containers/*.log is configured and there is the following directory
tree:
/var/log/pods/pod-1/a/0.log
/var/log/containers/pod-1-a-0.log -> /var/log/pods/pod-1/a/0.log
In this example, kubernetes cron jobs will permanently delete directories at the
/var/log/pods/pod-* level. And thus, inotify observer on the parent object of the
symbolic link target (/var/log/pods/pod-1/a/0.log) looking at the directory
/var/log/pods/pod-1/a will constantly leak.
This is due to the fact that the list of active objects in the edge with path
/var/log/containers, where the parent object of the target symbolic link is added,
is not checked. Verification and deletion will occur only in the case of an inotify
event in the upper nodes of the directory tree, in /var/log and above.
Thanks to Sergey Kacheev for the patch!
- 2023-06-05: GNUTls Driver: Fix memory leaks in gtlsInitCred
Missing CA Certificate or multiple Connections caused
a memory leak in pThis->xcred as it was allocated each time in
gtlsInitCred by gnutls_certificate_allocate_credentials
closes: https://github.com/rsyslog/rsyslog/issues/5135
- 2023-05-24: CI: update base ubuntu image for github actions
----------------------------------------------------------------------------------------
Scheduled Release 8.2304.0 (aka 2023.04) 2023-04-18
- 2023-04-17: imptcp bugfix: spam log on oversize message
If an oversize message was received by imptcp, imptcp reported
one error message for EACH oversize character. This could
result in a potentially very large number of similar (and