forked from balabit/syslog-ng-3.5
/
NEWS
682 lines (516 loc) · 25.4 KB
/
NEWS
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
3.5.2
Fri, 29 Nov 2013 17:09:40 +0100
This is the first bug-fix release for the 3.5.x series, with a
number of significant fixes over the original 3.5.1 release.
Upgrading to this release is highly recommended.
Bugfixes
========
* The system() source has been fixed on Solaris and HP-UX: it
correctly recognises the system on Solaris, and uses
/dev/log (and not /dev/pipe) on HP-UX.
* The system() source has much stricter requirements to select
/dev/kmsg on Linux, so it won't choose it over /proc/kmsg
unless the kernel really does support it.
* The syslog-format() module has been fixed to only set the
PROGRAM to "kernel" if all of the previous conditions apply,
and the message is a local message.
* The linux-kmsg-format parser (and system() on Linux 3.5+ as
a consequence) will correctly calculate the timestamp of
kernel messages on 32-bit systems now.
* The sun-streams() source has been fixed, it compiles and
works again.
* A very rare and short race condition has been fixed in the
pipe() source and destination modules, which could leave the
pipe available with elevated permissions for a short time.
* The SQL, MongoDB and SMTP modules were fixed to not cause a
segmentation fault when loaded before the socket module.
* A crash was fixed in the no-tls build of the socket
destination driver.
* It is now possible to specify the host() and port()
parameters for the redis() destination.
* Various stability fixes, such as:
- A use-after-free condition was corrected in the queue
backlog acknowledgement code.
- Always use setsid() when detaching, even if we're not
detaching from a TTY.
- Memory leaks were fixed in the persist state handling
code.
- Changing stats-level() and reloading syslog-ng produced a
crash, which has been fixed.
Credits:
========
syslog-ng is developed as a community project, and as such it relies
on volunteers to do the work necessarily to produce syslog-ng.
Reporting bugs, testing changes, writing code or simply providing
feedback are all important contributions, so please if you are a
user of syslog-ng, contribute.
These people have helped in this release:
Alexandre Lecuyer
Balazs Scheidler <bazsi@balabit.hu>
Daniel Gados <gdani@balabit.hu>
Gergely Nagy <algernon@balabit.hu>
Howard Tsai <htsai@skytap.com>
Imre Lazar <imre@balabit.hu>
Jakub Wilk <jwilk@debian.org>
Laszlo Budai <lbudai@balabit.hu>
Martin Bagge / brother <brother@bsnet.se>
Matyas Koszik <koszik@atw.hu>
Oscar Muñoz
Otto Berger <otto@bergerdata.de>
Viktor Juhasz <jviktor@balabit.hu>
Viktor Tusa <tusa@balabit.hu>
3.5.1
Mon, 4 Nov 2013 14:49:00 +0100
This is the first stable release in the 3.5 series, adding a
number of features compared to 3.4, a result of about eight
months of development. This release includes all the fixes of
the recent 3.4.5 release, and a host of new features.
Bugfixes since 3.5.0rc1
=======================
* A race condition in log message reference counting code that
sometimes led to crashes was fixed. [#255]
* A use-after-free error that sometimes happened after a
reload, and caused memory corruption was also fixed. [#253]
* patterndb was corrected not to create a new instance on
reload: this way, the old one is not leaked, and db-parser()
does not forget the correlation state, nor its idea of
current time on reload. [#257]
* The syslog-ng.spec file does not try to install the
long-removed ChangeLog file anymore.
Incompatible changes
====================
* Template escaping was changed in an incompatible way:
previously, both the lexer and the template compiler used
the '\' character for escaping, which was confusing. The
template compiler uses '$$' to escape the '$' char, and '@@'
to escape '@'.
If a non-numeric value follows '@', a warning will be
printed.
* The replace() transformation function of value-pairs() was
renamed to replace-prefix() to better reflect what it
actually does. The old name remains as an obsolete alias,
however.
* The username() and password() options of the MongoDB
destination were removed, they never did anything before,
either.
Major features since 3.4
========================
Multi-line support
------------------
A major feature in the 3.5 release is the inclusion of support
for multi-line messages, a feature that has been available in
syslog-ng PE for a good while.
With this, both indented and regexp-based multi-line support
becomes available. For further information and examples, see
the following blog post:
http://asylum.madhouse-project.org/blog/2013/09/05/what-is-cooking-in-syslog-ng-3.5/#f:multiline
STOMP destination
-----------------
Joining the message-queue club, this new destination makes
syslog-ng able to send events to any STOMP-capable message
queuing server, such as Apache ActiveMQ.
For further information about the destination, see this commit
message:
https://github.com/balabit/syslog-ng-3.5/commit/82d19a4d4ad3f91e8214006f6ea7a47baabf48ef
Redis destination
-----------------
Developed during the Google Summer of Code 2013 program, this
destination driver makes it possible to easily send commands
to a Redis server.
For further information and examples, please see the following
blog post:
http://tichygsoc.blogspot.hu/2013/09/the-road-so-far.html
Template type hinting
---------------------
While syslog-ng supported sending events to various datastores
and queues for a while now (SQL first, MongoDB, JSON, and AMQP
later), even if those supported other types of data, syslog-ng
only ever sent strings.
With template type hinting, it became possible to tell
syslog-ng what type a certain template should be, so that the
drivers can use that information in whatever way they see fit.
This is currently implemented for the MongoDB driver and the
$(format-json) template function only.
For more information about type hinting and for examples, see
the following post:
http://asylum.madhouse-project.org/blog/2013/09/05/what-is-cooking-in-syslog-ng-3.5/#f:type-hinting
Template options honored everywhere
-----------------------------------
Until this release, there were situations where template
options were ignored, such as filter expressions that use the
comparsion operators, regexp based substitutions, incoming
templates for parsers, the new value rewrite rules, SMTP
values, some of the new drivers (stomp, amqp), and patterndb
context-id.
These all honor the global template options now, and
per-driver options such as frac-digits and local-time-zone are
available for drivers which did not have them before, like
MongoDB.
Support for unit suffixes in the configuration
----------------------------------------------
You no longer need to remember how many zeros to put after a
big number in the syslog-ng configuration file, you can use
unit suffixes, such as:
log-fifo-size(2GiB)
See the following post for more details:
http://asylum.madhouse-project.org/blog/2013/09/05/what-is-cooking-in-syslog-ng-3.5/#f:unit-suffix
The Incubator project
---------------------
Alongside the 3.5.1 release, we are pleased to announce the
existence of the syslog-ng incubator project, which hosts
additional modules and tools not merged into syslog-ng proper.
These serve both as examples and as a staging area, but also
makes it easier to try out new modules without patching or
upgrading your syslog-ng version.
The project's homepage is:
https://github.com/balabit/syslog-ng-incubator
Other features
==============
* in-list() filter: this new filter function allows one to
easily implement file-based white- and blacklists with a
simple syntax:
filter f_white { in-list("/path/to/file", value("HOST")); };
* A set of new string-related template functions are
available: $(uppercase STRINGS...), $(lowercase STRINGS...)
and $(replace-delimiter DELIMITERS NEW-DELIMITER TEXT).
The first two do exactly what their names suggest, while the
last one replaces all occurrences of any DELIMITERS within
TEXT with the NEW-DELIMITER.
* There is also a new $(env VARIABLE...) template function,
which looks up the given variables in the environment. This
is similar to using backticks in the configuration file,
however the name of the environment variable with $(env) may
contain macros too.
* Support for Linux 3.5+'s /dev/kmsg was added, and the
system() source will automatically detect whether to use it
over /proc/kmsg.
* For every correlated message in patterndb, the
${.classifier.context_id} property is automatically set to
the context-id attribute of the matching rule.
* The build system was completely redone, it is much faster,
more reliable, and less verbose by default now.
* Several systemd-related enhancements were made, including
support for notification-based startup. This also means that
when systemd support is compiled in, libsystemd-daemon
becomes a new dependency.
Known Bugs
==========
* The afstreams module is broken, does not compile, and does
not work. This will be corrected in a later maintenance
release.
Credits:
========
syslog-ng is developed as a community project, and as such it relies
on volunteers to do the work necessarily to produce syslog-ng.
Reporting bugs, testing changes, writing code or simply providing
feedback are all important contributions, so please if you are a
user of syslog-ng, contribute.
These people have helped in this release:
Alexandre Biancalana <biancalana@gmail.com>
Andras Tim <tia@balabit.hu>
Anton Koldaev <koldaevav@gmail.com>
Attila M. Magyar <athos@balabit.hu>
Attila Nagy <bra@fsn.hu>
Attila Szalay <sasa@balabit.hu>
Balazs Scheidler <bazsi@balabit.hu>
Balint Kovacs <blint@balabit.hu>
Chris Johnson <chris.johnson3@hp.com>
Cy Schubert <Cy.Schubert@komquats.com>
Evan Rempel <erempel@uvic.ca>
Fabien Wernli <cpan@faxm0dem.org>
Gergely Nagy <algernon@balabit.hu>
Gonzalo Paniagua <gonzalo.paniagua+slng1@acquia.com>
Jose Pedro Oliveira <jpo@di.uminho.pt>
Laszlo Budai <lbudai@balabit.hu>
Lucas McLane <lucas@clicksecurity.com>
Marc Falzon <marc.falzon@cloudwatt.com>
Martin <bmartin@lavabit.com>
Michal Privoznik <miso.privoznik@gmail.com>
Michael Sterrett <mr_bones_@gentoo.org>
Nicolas Szalay <nico@rottenbytes.info>
Oscar Muñoz
Paul Dann <pdgiddie+balabit@gmail.com>
Peter Czanik <czanik@balabit.hu>
Peter Gyongyosi <gyp@balabit.hu>
Robert Fekete <frobert@balabit.hu>
Ryan Frederick
Sergey Shuman
Tamas Pal <folti@balabit.hu>
Tibor Benke <btibi@balabit.hu>
Tihamer Petrovics <tihameri@gmail.com>
Valentijn Sessink <valentijn@sessink.nl>
Viktor Juhasz <jviktor@balabit.hu>
Viktor Tusa <tusa@balabit.hu>
Vincent Brillault <spam@lerya.net>
3.5.0rc1
Thu, 24 Oct 2013 10:48:51 +0200
This is the first Release Candidate of the syslog-ng 3.5
development branch, with stability and reliability
improvements. Being a release candidate, this release should
be very close to production ready.
Bugfixes
========
* The program() source and destination now properly terminates
child processes on reload and shutdown.
* The AMQP, SMTP, MongoDB, Redis and STOMP destinations now
work correctly across reloads.
* A rare race condition in the log message reference cache was
corrected, fixing a crash.
* The JSON parser now understands null-typed values, and
ignores the whole key silently when it encounters one.
* Using the subst() rewrite rule when the same source is
present in multiple logpaths does not crash anymore.
Miscellaneous
=============
* The autogen.sh script is included in the distributed
tarball.
Known Bugs
==========
* The afstreams module is broken, does not compile, and does
not work. This will be corrected before the final 3.5.1
release.
Credits:
========
syslog-ng is developed as a community project, and as such it relies
on volunteers to do the work necessarily to produce syslog-ng.
Reporting bugs, testing changes, writing code or simply providing
feedback are all important contributions, so please if you are a
user of syslog-ng, contribute.
These people have helped in this release:
Balazs Scheidler <bazsi@balabit.hu>
Evan Rempel <erempel@uvic.ca>
Gergely Nagy <algernon@balabit.hu>
Jose Pedro Oliveira <jpo@di.uminho.pt>
Michal Privoznik <miso.privoznik@gmail.com>
Nicolas Szalay <nico@rottenbytes.info>
Ryan Frederick
3.5.0beta3
Fri, 11 Oct 2013 10:21:38 +0200
This is the third BETA release of the syslog-ng 3.5
development branch, with stability fixes and other
enhancements. While running in production is still not
advised, testing is most appreciated.
Incompatible changes
====================
Template escaping
-----------------
Template escaping was changed in an incompatible way:
previously, both the lexer and the template compiler used the
'\' character for escaping, which was confusing. The template
compiler uses '$$' to escape the '$' char, and '@@' to escape
'@'.
If a non-numeric value follows '@', a warning will be printed.
Bugfixes
========
* syslog-ng should compile again on non-Linux platforms.
* The flush() and fsync() options of the file destination were
fixed, and they should work the same way now as they do in
3.3 and 3.4.
* The hiredis library should be detected on the default
include paths, so one does not necessarily need to specify
--with-libhiredis non non-Debian systems. Said option was
also made to work.
* A memory corruption was fixed in the @STRING@ parser of
db-parser().
* Excludes now work properly with value-pairs(), and they do
not get ignored if the value to exclude is in the default
set.
Known Bugs
==========
* The afstreams module is broken, does not compile, and does
not work. This will be corrected before the final 3.5.1
release.
Credits:
========
syslog-ng is developed as a community project, and as such it relies
on volunteers to do the work necessarily to produce syslog-ng.
Reporting bugs, testing changes, writing code or simply providing
feedback are all important contributions, so please if you are a
user of syslog-ng, contribute.
These people have helped in this release:
Balazs Scheidler <bazsi@balabit.hu>
Cy Schubert <Cy.Schubert@komquats.com>
Evan Rempel <erempel@uvic.ca>
Gergely Nagy <algernon@balabit.hu>
Lucas McLane <lucas@clicksecurity.com>
Oscar Muñoz
Peter Czanik <czanik@balabit.hu>
Sergey Shuman
Viktor Juhasz <jviktor@balabit.hu>
3.5.0beta2
Fri, 27 Sep 2013 10:06:52 +0200
This is the second BETA release of the syslog-ng 3.5
development branch, introducing new features and a handful of
fixes since the last beta version. While running in production
is still not advised, testing is most appreciated.
Major Features
==============
Redis destination
-----------------
Developed during the Google Summer of Code 2013 program, this
destination driver makes it possible to easily send commands
to a Redis server.
For further information and examples, please see the following
blog post:
http://tichygsoc.blogspot.hu/2013/09/the-road-so-far.html
Template type hinting
---------------------
In 3.5.0beta1, type hinting was implemented for the MongoDB
destination, with this release, it is also available for the
$(format-json) template function.
Template options honored everywhere
-----------------------------------
Until this release, there were situations where template
options were ignored, such as filter expressions that use the
comparsion operators, regexp based substitutions, incoming
templates for parsers, the new value rewrite rules, SMTP
values, some of the new drivers (stomp, amqp), and patterndb
context-id.
These all honor the global template options now, and
per-driver options such as frac-digits and local-time-zone are
available for drivers which did not have them before, like
MongoDB.
Bugfixes
========
* The AMQP destination has been fixed, and using it does not
lead to a crash anymore.
* Several memory leaks in the STOMP destination have been
fixed.
* The --no-framing option of loggen has been fixed, and works
as advertised now.
* The in-list() filter ignores empty lines in its input file,
to avoid matching on empty values.
* Compilation on FreeBSD has been fixed.
Credits:
========
syslog-ng is developed as a community project, and as such it relies
on volunteers to do the work necessarily to produce syslog-ng.
Reporting bugs, testing changes, writing code or simply providing
feedback are all important contributions, so please if you are a
user of syslog-ng, contribute.
These people have helped in this release:
Balazs Scheidler <bazsi@balabit.hu>
Gergely Nagy <algernon@balabit.hu>
Peter Czanik <czanik@balabit.hu>
Tibor Benke <btibi@balabit.hu>
Tihamer Petrovics <tihameri@gmail.com>
Viktor Tusa <tusa@balabit.hu>
3.5.0beta1
Thu, 19 Sep 2013 11:03:15 +0200
This is a BETA release of the syslog-ng 3.5 development
branch, with a lot of internal rework and even more new
features included. Being a beta release, testing is most
appreciated, but running it in production is not advised.
The release contains all the fixes from the 3.4 branch, the
changes below are relative to the 3.4.3 release.
Major Features
==============
Multi-line support
------------------
A major feature in the 3.5 release is the inclusion of support
for multi-line messages, a feature that has been available in
syslog-ng PE for a good while.
With this, both indented and regexp-based multi-line support
becomes available. For further information and examples, see
the following blog post:
http://asylum.madhouse-project.org/blog/2013/09/05/what-is-cooking-in-syslog-ng-3.5/#f:multiline
STOMP destination
-----------------
Joining the message-queue club, this new destination makes
syslog-ng able to send events to any STOMP-capable message
queuing server, such as Apache ActiveMQ.
For further information about the destination, see this commit
message:
https://github.com/balabit/syslog-ng-3.5/commit/82d19a4d4ad3f91e8214006f6ea7a47baabf48ef
Template type hinting
---------------------
While syslog-ng supported sending events to various datastores
and queues for a while now (SQL first, MongoDB, JSON, and AMQP
later), even if those supported other types of data, syslog-ng
only ever sent strings.
With template type hinting, it became possible to tell
syslog-ng what type a certain template should be, so that the
drivers can use that information in whatever way they see fit.
This is currently implemented for the MongoDB driver only.
For more information about type hinting and for examples, see
the following post:
http://asylum.madhouse-project.org/blog/2013/09/05/what-is-cooking-in-syslog-ng-3.5/#f:type-hinting
Other features
==============
* in-list() filter: this new filter function allows one to
easily implement file-based white- and blacklists with a
simple syntax:
filter f_white { in-list("/path/to/file", value("HOST")); };
* You no longer need to remember how many zeros to put after a
big number in the syslog-ng configuration file, you can use
unit suffixes, such as:
log-fifo-size(2GiB)
See the following post for more details:
http://asylum.madhouse-project.org/blog/2013/09/05/what-is-cooking-in-syslog-ng-3.5/#f:unit-suffix
* A set of new string-related template functions are
available: $(uppercase STRINGS...), $(lowercase STRINGS...)
and $(replace-delimiter DELIMITERS NEW-DELIMITER TEXT).
The first two do exactly what their names suggest, while the
last one replaces all occurrences of any DELIMITERS within
TEXT with the NEW-DELIMITER.
* There is also a new $(env VARIABLE...) template function,
which looks up the given variables in the environment. This is
similar to using backticks in the configuration file, however the
name of the environment variable with $(env) may contain macros
too.
* Support for Linux 3.5+'s /dev/kmsg was added, and the
system() source will automatically detect whether to use it
over /proc/kmsg.
* For every correlated message in patterndb, the
${.classifier.context_id} property is automatically set to
the context-id attribute of the matching rule.
Incompatible changes
====================
* The replace() transformation function of value-pairs() was
renamed to replace-prefix() to better reflect what it
actually does. The old name remains as an obsolete alias,
however.
* The username() and password() options of the MongoDB
destination were removed, they never did anything before,
either.
Miscellaneous changes
=====================
* The build system was completely redone, it is much faster,
more reliable, and less verbose by default now.
* Several systemd-related enhancements were made, including
support for notification-based startup. This also means that
when systemd support is compiled in, libsystemd-daemon
becomes a new dependency.
Credits:
========
syslog-ng is developed as a community project, and as such it relies
on volunteers to do the work necessarily to produce syslog-ng.
Reporting bugs, testing changes, writing code or simply providing
feedback are all important contributions, so please if you are a
user of syslog-ng, contribute.
These people have helped in this release:
Andras Tim <tia@balabit.hu>
Anton Koldaev <koldaevav@gmail.com>
Attila M. Magyar <athos@balabit.hu>
Attila Nagy <bra@fsn.hu>
Attila Szalay <sasa@balabit.hu>
Balazs Scheidler <bazsi@balabit.hu>
Balint Kovacs <blint@balabit.hu>
Chris Johnson <chris.johnson3@hp.com>
Evan Rempel <erempel@uvic.ca>
Fabien Wernli <cpan@faxm0dem.org>
Gergely Nagy <algernon@balabit.hu>
Juhasz Viktor <jviktor@balabit.hu>
Laszlo Budai <lbudai@balabit.hu>
Martin <bmartin@lavabit.com>
Michael Sterrett <mr_bones_@gentoo.org>
Michal Privoznik <miso.privoznik@gmail.com>
Paul Dann <pdgiddie+balabit@gmail.com>
Peter Czanik <czanik@balabit.hu>
Peter Gyongyosi <gyp@balabit.hu>
Robert Fekete <frobert@balabit.hu>
Tamas Pal <folti@balabit.hu>
Tibor Benke <btibi@balabit.hu>
Valentijn Sessink <valentijn@sessink.nl>
Viktor Tusa <tusa@balabit.hu>
Vincent Brillault <spam@lerya.net>