plugopts from config file not mentioned in "effective options" debug log

[pmoravec]

Currently, sos report prints a debug:

[sos.report:setup] effective options now: --batch --build --config-file /tmp/sos.conf --only-plugins qpid -vvv

before the config file is evaluated. We dont know what particular options were really used at the end.

Shall not we (maybe rename the "effective"? and ..) add a debug similar to above that will print options after all options processing is done, incl. config file?

Reproducer:

echo "[report]" > /tmp/sos.conf
echo "only-plugins = qpid" >> /tmp/sos.conf
python3 bin/sos report --conf /tmp/sos.conf -v -v -v --batch --build | grep ffect
[sos.report:setup] effective options now: --batch --build --config-file /tmp/sos.conf --only-plugins qpid -vvv

(or have the default /etc/sos/sos.conf of the same content)

[pmoravec]

I apologize for confusion - the bug is in not printing plugin options only, the above example shows a correct behaviour. Proper reproducer:

echo "[plugin_options]" > /tmp/sos.conf; echo "filesys.lsof = on" >> /tmp/sos.conf
python3 bin/sos report --conf /tmp/sos.conf -v -v -v --batch --build | grep effect
[sos.report:setup] effective options now: --batch --build --config-file /tmp/sos.conf -vvv

(same happens when updating the default /etc/sos/sos.conf file - other sections are shown in the effective options", but plugin_options` not)

[TurboTurtle]

Two findings:

1. This is caused by https://github.com/sosreport/sos/blob/main/sos/options.py#L103 - we need to instead hand a private copy of this value over here, otherwise we update the arg_default value when we later set the plugin options at https://github.com/sosreport/sos/blob/main/sos/options.py#L236

This can be solved by using self.arg_defaults.get(arg) on line 103. Easy enough.

2. However, if we have a plugin option set in the config file and we pass -k plugin.opt=foo on the cmdline, the config file value is overwritten. This is inline with how we handle options generally right now, but I wonder if this is actually desired for plugin options specifically? Should we continue to overwrite, or should the cmdline options be appended (since we allow multiple -k invocations on the cmdline)?

[pmoravec]

1. Whatever I tried on line 103 there, I got no improvement :( .

2. Good catch. That is a separate issue where I would lean to "append" behaviour (if needed, one can use -k plug.opt=Default to overwrite the config file setting). But it can be subjective and I am not against the current implementation.

[TurboTurtle]

Ah, ok - looks like I had a separate change in my local branch that actually resolved it. If in SoSOptions.__init__() you add self.plugopts = [], you should see it log properly. Again this appears to be caused by the update_from_conf() method actually updating arg_defaults for plugopts from the conf file, thus causing argify().has_value() to detect that it hasn't been changed.

$ git diff upstream/main 
diff --git a/sos/options.py b/sos/options.py
index a014a022..3b660939 100644
--- a/sos/options.py
+++ b/sos/options.py
@@ -105,6 +105,7 @@ class SoSOptions():
         for arg in kwargs.keys():
             self.arg_names.append(arg)
             setattr(self, arg, kwargs[arg])
+        self.plugopts = []
 
     @classmethod
     def from_args(cls, args, arg_defaults={}):

$ grep -C2 kernel /etc/sos/sos.conf
[plugin_options]
#rpm.rpmva = off
kernel.with-timer = on


$ sudo ./bin/sos report -o kernel -v --batch

$ grep effectiv sos_logs/sos.log 
2021-08-31 04:53:37,974 INFO: [sos.report:setup] effective options now: --batch --only-plugins kernel --plugopts kernel.with-timer=on -v

That being said, I don't think this is really the best fix for this on its face, but I am struggling to find a better approach.

[pmoravec]

Hmm.. that ignores cmdline options: try in your case "-k kernel.trace=yes":

# grep -C2 kernel /etc/sos/sos.conf

[plugin_options]
kernel.with-timer = on
# python3 bin/sos report -o kernel --batch -vvv -k kernel.trace=yes | grep effect
[sos.report:setup] effective options now: --batch --only-plugins kernel --plugopts kernel.with-timer=on -vvv
#