Set loglevels from config #4091

bazsi · 2022-07-29T10:55:15Z

This patch adds a feature to control the internal verbosity of syslog-ng (e.g. verbose, debug, trace) via the configuration file.

Example:

    options {
      log-level(debug);
    };

Possible values for log-level:
- default, just normal log messages
- verbose, normal + messages that get enabled via -v
- debug, verbose + messages that get enabled via -d
- trace, debug + messages that get enabled via -t

If the option is unset, we leave the settings as specified on the command line.

If we have log-level(default) it would override settings specified via the command line.

Rationale: in containerized environments, it is much easier to override log verbosity settings via the config file than via changing the command line options.

github-actions · 2022-07-29T10:55:30Z

No news file has been detected. Please write one, if applicable.

lgtm-com · 2022-07-29T11:19:33Z

This pull request introduces 1 alert when merging 20c4f4daafa815b8a178b47e1dd85fa983388641 into d8d8d0f - view on LGTM.com

new alerts:

1 for Comparison result is always the same

MrAnno · 2022-07-29T16:29:39Z

Shouldn't we work the other way around? Command line options are usually stronger than configuration file options as they are easier to modify. For example, starting syslog-ng with -d is much quicker than removing/changing the option in the config.

AFAIK other tools work similarly, for example, nginx has a command line flag to override a few directives inside the config file.

bazsi · 2022-07-29T21:22:21Z

Good point.

…

On Fri, Jul 29, 2022, 18:29 László Várady ***@***.***> wrote: Shouldn't we work the other way around? Command line options are usually stronger than configuration file options as they are easier to modify. For example, starting syslog-ng with -d is much quicker than removing/changing the option in the config. AFAIK other tools work similarly, for example, nginx has an option to override a few directives inside the config file. — Reply to this email directly, view it on GitHub <#4091 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAFOK5RUSHEABRG6WZXKY73VWQBH3ANCNFSM55ALLVVQ> . You are receiving this because you authored the thread.Message ID: ***@***.***>

kira-syslogng · 2022-08-04T18:57:34Z

Build FAILURE

kira-syslogng · 2022-08-07T06:58:20Z

Build FAILURE

kira-syslogng · 2022-08-07T11:46:57Z

Build FAILURE

kira-syslogng · 2022-08-24T11:20:00Z

Build FAILURE

bazsi · 2022-08-24T12:16:02Z

@kira-syslogng seems to build on the response from "syslog-ng-ctl debug" which was changed by this PR. Can anyone help me with fixing that dependency?

kira-syslogng · 2022-08-24T12:39:05Z

Build FAILURE

MrAnno · 2022-08-28T20:43:12Z

@bazsi Of course. I'll take a look at this next week.

kira-syslogng · 2022-09-01T15:07:49Z

Build FAILURE

mitzkia · 2022-09-02T07:56:38Z

@kira-syslogng test this please test branch=followup_log_level_changes; test=functions/tools/syslog-ng-ctl/;

kira-syslogng · 2022-09-02T08:11:14Z

Build FAILURE

mitzkia · 2022-09-02T14:49:05Z

@kira-syslogng test this please test branch=followup_log_level_changes; test=functions/tools/syslog-ng-ctl/;

kira-syslogng · 2022-09-02T15:04:33Z

Build FAILURE

MrAnno · 2022-09-02T15:10:54Z

@kira-syslogng test this please test branch=followup_log_level_changes;

kira-syslogng · 2022-09-02T15:33:08Z

Build FAILURE

MrAnno · 2022-09-02T16:48:01Z

I'm just thinking out loud:
This PR deliberately replaces our composable diagnostic flags -Fdevt with a more straightforward approach: log levels.

This means that the -Fdevt is not considered useful anymore as -Fet produces the same level.

In that case, we could say that the old syslog-ng-ctl commands are just for backward compatibility (deprecated), log-level should be used instead. I think the same is true for the command line flags: Since using -dvt is not intuitive anymore either, a --log-level flag would be really useful and could maintain a consistent feel.

MrAnno · 2022-09-09T11:06:47Z

@bazsi What do you think?

kira-syslogng · 2022-09-09T14:55:22Z

Build FAILURE

bazsi · 2022-09-09T14:55:38Z

@bazsi What do you think?

done.

bazsi · 2022-09-12T16:14:03Z

@kira-syslogng test this please test branch=followup_log_level_changes;

kira-syslogng · 2022-09-12T16:40:28Z

Build FAILURE

mitzkia · 2022-09-15T11:59:38Z

Just for the record about the latest failing Kira job.
Somehow Jenkins clones out not the PR itself, but the master branch, and because of this the test assertion fails.
In the other Kira test job everything is ok, new testcase assertion passes on PR changes.

mitzkia · 2022-09-16T09:01:48Z

@kira-syslogng test this please test branch=followup_log_level_changes;

kira-syslogng · 2022-09-16T09:35:03Z

Build FAILURE

mitzkia · 2022-09-16T09:39:31Z

@kira-syslogng test this please test branch=followup_log_level_changes;

mitzkia · 2022-09-16T11:07:38Z

Latest Kira-starter job runs correctly (I have fixed the Kira job):

it uses this PR as a source in both cases (cmake and autotools build), and
the correct topic branch for testdb

bazsi · 2022-09-18T05:04:35Z

Thanks @mitzkia

With that I think we can merge this once it gets a thumbs up from someone.

lib/messages.c

lib/cfg.c

lib/messages.c

lib/mainloop-control.c

Signed-off-by: Balazs Scheidler <bazsi77@gmail.com>

This patch adds a feature to control the internal verbosity of syslog-ng (e.g. verbose, debug, trace) via the configuration file. Example: options { log-level(debug); }; Possible values for log-level: - default, just normal log messages - verbose, normal + messages that get enabled via -v - debug, verbose + messages that get enabled via -d - trace, debug + messages that get enabled via -t The command line continues to dominate the log-level, e.g. if the command line contains options to control the level of logging, the configuration file would be ignored. Signed-off-by: Balazs Scheidler <bazsi77@gmail.com>

…-ctl to log levels syslog-ng-ctl is issuing separate commands for various log levels, e.g. LOG VERBOSE ON LOG DEBUG ON LOG TRACE ON which was operating on various verbosity flags exported by the messages module. This patch changes the implementation so while the interface continues to work, we are mapping the control commands to log levels. Whevever we turn ON verbose/debug/trace, we set the log level to that specific values. When we turn them OFF we set the log level one less than the specific value. LOG VERBOSE ON => verbose LOG VERBOSE OFF => default LOG DEBUG ON => debug LOG DEBUG OFF => verbose LOG TRACE ON => trace LOG TRACE OFF => debug This keeps compatibility with old syslog-ng-ctl binaries with a tiny change in functionality: the user will not be able to enable trace messages while not enabling debug at the same time. Signed-off-by: Balazs Scheidler <bazsi77@gmail.com>

…ate function Signed-off-by: Balazs Scheidler <bazsi77@gmail.com>

…tl commands Signed-off-by: Balazs Scheidler <bazsi77@gmail.com>

This patch adds "log-level" to syslog-ng-ctl while using the old control commands. Signed-off-by: Balazs Scheidler <bazsi77@gmail.com>

Signed-off-by: Balazs Scheidler <bazsi77@gmail.com>

bazsi · 2022-09-18T14:55:57Z

added some more unit tests to cover the cmdline/config overrides.

MrAnno · 2022-09-18T15:19:31Z

@kira-syslogng test this please test branch=followup_log_level_changes;

MrAnno

Thank you. LGTM.

We'll merge this PR on Monday together with the internal testdb modification.

MrAnno · 2022-09-18T15:12:39Z

lib/mainloop-control.c

+    {
+      gboolean on = g_str_equal(onoff, "ON");
+      if (!on)
+        ll = ll - 1;


One last note:

Asking our users to enable debug logging in production for a short period of time is quite usual, the support team does this frequently in the case of syslog-ng PE.

"Debug on" and then "debug off" now are not inverse functions.

I think this can be documented, and we can inform the affected parties about the change and the new intuitive log-level alternative.

bazsi force-pushed the set-loglevels-from-config branch from 20c4f4d to 7410142 Compare July 29, 2022 12:20

bazsi force-pushed the set-loglevels-from-config branch from 7410142 to a307bdb Compare August 4, 2022 18:38

bazsi force-pushed the set-loglevels-from-config branch from a307bdb to 4cb5d58 Compare August 7, 2022 06:33

bazsi force-pushed the set-loglevels-from-config branch 2 times, most recently from fd2ff11 to a60eed7 Compare August 24, 2022 10:58

bazsi force-pushed the set-loglevels-from-config branch from a60eed7 to e2fbe9e Compare August 24, 2022 12:15

MrAnno self-requested a review August 28, 2022 20:43

bazsi force-pushed the set-loglevels-from-config branch from e2fbe9e to 2b641e5 Compare September 1, 2022 14:44

bazsi force-pushed the set-loglevels-from-config branch from 2b641e5 to 5ec9099 Compare September 9, 2022 14:54

MrAnno requested changes Sep 18, 2022

View reviewed changes

bazsi force-pushed the set-loglevels-from-config branch 2 times, most recently from 891aa9a to 380bc79 Compare September 18, 2022 14:36

bazsi added 8 commits September 18, 2022 16:55

messages: allow setting verbosity levels via log levels

a202403

Signed-off-by: Balazs Scheidler <bazsi77@gmail.com>

mainloop-control: extract LOG <level> <onoff> processing into a separ…

58a2c3b

…ate function Signed-off-by: Balazs Scheidler <bazsi77@gmail.com>

mainloop-control: add LOG LEVEL command as an alternative to legacy c…

a06439b

…tl commands Signed-off-by: Balazs Scheidler <bazsi77@gmail.com>

syslog-ng-ctl: add the new log-level subcommand

df8b071

This patch adds "log-level" to syslog-ng-ctl while using the old control commands. Signed-off-by: Balazs Scheidler <bazsi77@gmail.com>

tests/copyright: added syslog-ng-ctl log-level related files

2fd17c0

Signed-off-by: Balazs Scheidler <bazsi77@gmail.com>

news: added news file

57c8544

Signed-off-by: Balazs Scheidler <bazsi77@gmail.com>

bazsi force-pushed the set-loglevels-from-config branch from 380bc79 to 57c8544 Compare September 18, 2022 14:55

bazsi requested a review from MrAnno September 18, 2022 14:56

MrAnno approved these changes Sep 18, 2022

View reviewed changes

MrAnno merged commit 79a7d9b into syslog-ng:master Sep 19, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Set loglevels from config #4091

Set loglevels from config #4091

bazsi commented Jul 29, 2022

github-actions bot commented Jul 29, 2022

lgtm-com bot commented Jul 29, 2022

MrAnno commented Jul 29, 2022 •

edited

bazsi commented Jul 29, 2022 via email

kira-syslogng commented Aug 4, 2022

kira-syslogng commented Aug 7, 2022

kira-syslogng commented Aug 7, 2022

kira-syslogng commented Aug 24, 2022

bazsi commented Aug 24, 2022

kira-syslogng commented Aug 24, 2022

MrAnno commented Aug 28, 2022

kira-syslogng commented Sep 1, 2022

mitzkia commented Sep 2, 2022

kira-syslogng commented Sep 2, 2022

mitzkia commented Sep 2, 2022

kira-syslogng commented Sep 2, 2022

MrAnno commented Sep 2, 2022

kira-syslogng commented Sep 2, 2022

MrAnno commented Sep 2, 2022

MrAnno commented Sep 9, 2022

kira-syslogng commented Sep 9, 2022

bazsi commented Sep 9, 2022

bazsi commented Sep 12, 2022

kira-syslogng commented Sep 12, 2022

mitzkia commented Sep 15, 2022 •

edited

mitzkia commented Sep 16, 2022

kira-syslogng commented Sep 16, 2022

mitzkia commented Sep 16, 2022

mitzkia commented Sep 16, 2022 •

edited

bazsi commented Sep 18, 2022

bazsi commented Sep 18, 2022

MrAnno commented Sep 18, 2022

MrAnno left a comment

MrAnno Sep 18, 2022 •

edited

Set loglevels from config #4091

Set loglevels from config #4091

Conversation

bazsi commented Jul 29, 2022

github-actions bot commented Jul 29, 2022

lgtm-com bot commented Jul 29, 2022

MrAnno commented Jul 29, 2022 • edited

bazsi commented Jul 29, 2022 via email

kira-syslogng commented Aug 4, 2022

kira-syslogng commented Aug 7, 2022

kira-syslogng commented Aug 7, 2022

kira-syslogng commented Aug 24, 2022

bazsi commented Aug 24, 2022

kira-syslogng commented Aug 24, 2022

MrAnno commented Aug 28, 2022

kira-syslogng commented Sep 1, 2022

mitzkia commented Sep 2, 2022

kira-syslogng commented Sep 2, 2022

mitzkia commented Sep 2, 2022

kira-syslogng commented Sep 2, 2022

MrAnno commented Sep 2, 2022

kira-syslogng commented Sep 2, 2022

MrAnno commented Sep 2, 2022

MrAnno commented Sep 9, 2022

kira-syslogng commented Sep 9, 2022

bazsi commented Sep 9, 2022

bazsi commented Sep 12, 2022

kira-syslogng commented Sep 12, 2022

mitzkia commented Sep 15, 2022 • edited

mitzkia commented Sep 16, 2022

kira-syslogng commented Sep 16, 2022

mitzkia commented Sep 16, 2022

mitzkia commented Sep 16, 2022 • edited

bazsi commented Sep 18, 2022

bazsi commented Sep 18, 2022

MrAnno commented Sep 18, 2022

MrAnno left a comment

Choose a reason for hiding this comment

MrAnno Sep 18, 2022 • edited

Choose a reason for hiding this comment

MrAnno commented Jul 29, 2022 •

edited

mitzkia commented Sep 15, 2022 •

edited

mitzkia commented Sep 16, 2022 •

edited

MrAnno Sep 18, 2022 •

edited