Author: Stefan Hundhammer shundhammer@suse.com
License: GPL V2
This contains some scripts for handling and analyzing y2logs as generated by YaST and the save_y2logs script.
They are provided in the hope that they may be useful, but there is absolutely no warranty and no support. Use at your own risk.
This merges all the parts of y2logs into one single file, so you no longer have
to deal with y2log, y2log-1.gz, y2log-2.gz etc. separately.
It generates one single file y2log-full.log from them.
Usage:
cd YaST2
y2log-merge
You will get a new file y2log-full.log.
This splits one y2log (by default y2log-full.log) into several parts, one for
each y2base call (i.e. YaST start). The resulting files include a numeric
part for the chronological order of invocation and the name of the YaST module
called.
Usage:
cd YaST2
y2log-merge
y2log-split
or
cd YaST2
y2log-split y2log
You will get files like
- y2-00-installation.log
- y2-01-menu.log
- y2-02-sw_single.log
- y2-03-repositories.log
- y2-04-repositories.log
- y2-05-repositories.log
- y2-06-repositories.log
- y2-07-dasd.log
- y2-08-dasd.log
- y2-09-dasd.log
This shows the headers (the full y2base call) from split logs (that were
created by y2log-split):
% y2log-headers
y2-00-installation.log: 2021-02-26 22:29:37 y2base called with ["installation", "--arg", "initial", "ncurses"]
y2-01-menu.log: 2021-02-26 17:42:37 y2base called with ["menu", "ncurses", "-name", "YaST2", "-icon", "yast"]
y2-02-sw_single.log: 2021-02-26 17:42:40 y2base called with ["sw_single", "ncurses", "-name", "YaST2", "-icon", "yast"]
y2-03-repositories.log: 2021-02-26 17:42:52 y2base called with ["repositories", "ncurses", "-name", "YaST2", "-icon", "yast"]
y2-04-repositories.log: 2021-02-26 17:48:14 y2base called with ["repositories", "ncurses", "-name", "YaST2", "-icon", "yast"]
y2-05-repositories.log: 2021-02-26 17:51:45 y2base called with ["repositories", "ncurses", "-name", "YaST2", "-icon", "yast"]
y2-06-repositories.log: 2021-02-26 17:58:07 y2base called with ["repositories", "ncurses", "-name", "YaST2", "-icon", "yast"]
y2-07-dasd.log: 2021-02-26 18:06:22 y2base called with ["dasd", "ncurses", "-name", "YaST2", "-icon", "yast"]
y2-08-dasd.log: 2021-02-27 00:07:52 y2base called with ["dasd", "ncurses", "-name", "YaST2", "-icon", "yast"]
y2-09-dasd.log: 2021-02-27 00:11:57 y2base called with ["dasd", "ncurses", "-name", "YaST2", "-icon", "yast"]
This shows y2base calls from the raw (gzipped and not split) y2log and
y2log-[1-9].gz files:
% y2log-y2base-calls
y2log-1.gz: 2021-02-26 22:29:37 y2base called with ["installation", "--arg", "initial", "ncurses"]
y2log: 2021-02-26 17:42:37 y2base called with ["menu", "ncurses", "-name", "YaST2", "-icon", "yast"]
y2log: 2021-02-26 17:42:40 y2base called with ["sw_single", "ncurses", "-name", "YaST2", "-icon", "yast"]
y2log: 2021-02-26 17:42:52 y2base called with ["repositories", "ncurses", "-name", "YaST2", "-icon", "yast"]
y2log: 2021-02-26 17:48:14 y2base called with ["repositories", "ncurses", "-name", "YaST2", "-icon", "yast"]
y2log: 2021-02-26 17:51:45 y2base called with ["repositories", "ncurses", "-name", "YaST2", "-icon", "yast"]
y2log: 2021-02-26 17:58:07 y2base called with ["repositories", "ncurses", "-name", "YaST2", "-icon", "yast"]
y2log: 2021-02-26 18:06:22 y2base called with ["dasd", "ncurses", "-name", "YaST2", "-icon", "yast"]
y2log: 2021-02-27 00:07:52 y2base called with ["dasd", "ncurses", "-name", "YaST2", "-icon", "yast"]
y2log: 2021-02-27 00:11:57 y2base called with ["dasd", "ncurses", "-name", "YaST2", "-icon", "yast"]
This removes the files created by y2log-split, i.e. all files that match
y2-[0-9][0-9]*.log.
Caution: This does not ask for confirmation, it just does it.
The scripts in this section are intended for simplifying y2logs.
This is a wrapper script to strip off common cruft in y2logs, to simplify the
line header, and to downgrade misleading Cheetah error log lines from <3> to
<1>.
All operations are done in-place, so make sure to copy the original file if you want to keep the untouched version.
This script can be used commonly for y2logs with or without debug logging.
Usage:
y2log-clean y2-??-*.log
This strips off irregular lines from one or more y2log files in-place. It leaves only those that match the standard log line pattern:
2024-02-30 14:07:42 <0> myhost(4711) [Ruby]
This is a remedy against all that cruft from random places that fills up the y2logs like this:
// global string String::Quote (string)
// global string String::UnQuote (string)
-------------------------------- accept ----------------------------
// filename: "/usr/share/YaST2/data/languages/language_af_ZA.ycp"
textdomain "languages_db";
return $["af_ZA":["Afrikaans", "Afrikaans", ".UTF-8", "", _("Afrikaans")], "keyboard":"english-us"];
It is hopeless that those who thought it would be a good idea to just dump that stuff into the y2log will ever clean up their mess; that hasn't happened over the last 15-20 years, so it won't happen in the next 15-20 years.
This y2log-strip-irregular script gets rid of it.
This is included in the more general y2log-clean script.
This strips off the date part of the log line headers, the hostname and the PID, so it considerably shortens the line header:
From
2024-02-30 14:07:42 <1> myhost(4711) [Ruby] foobar.rb:42: Hello world!
2024-02-30 14:07:42 <1> myhost(4711) [Ruby] foobar.rb:43: ...
2024-02-30 14:07:42 <1> myhost(4711) [Ruby] foobar.rb:44: ...
To
14:07:42 <1> [Ruby] foobar.rb:42: Hello world!
14:07:42 <1> [Ruby] foobar.rb:43: ...
14:07:42 <1> [Ruby] foobar.rb:44: ...
This is included in the more general y2log-clean script.
The rationale is that those parts are always the same anyway for each log line in all normal cases, so we can save some screen space; and, more importantly, give Bugzilla less opportunities to butcher any pasted content, like it loves to do.
This does only the date stripping part of y2log-strip-date-hostname:
From
2024-02-30 14:07:42 <1> myhost(4711) [Ruby] foobar.rb:42: Hello world!
2024-02-30 14:07:42 <1> myhost(4711) [Ruby] foobar.rb:43: ...
2024-02-30 14:07:42 <1> myhost(4711) [Ruby] foobar.rb:44: ...
To
14:07:42 <1> myhost(4711) [Ruby] foobar.rb:42: Hello world!
14:07:42 <1> myhost(4711) [Ruby] foobar.rb:43: ...
14:07:42 <1> myhost(4711) [Ruby] foobar.rb:44: ...
This might be useful if the hostname actually changes, or if multiple YaST processes are logging into the same file at the same time.
This does only the hostname/PID stripping part of y2log-strip-date-hostname:
From
2024-02-30 14:07:42 <1> myhost(4711) [Ruby] foobar.rb:42: Hello world!
2024-02-30 14:07:42 <1> myhost(4711) [Ruby] foobar.rb:43: ...
2024-02-30 14:07:42 <1> myhost(4711) [Ruby] foobar.rb:44: ...
To
2024-02-30 14:07:42 <1> [Ruby] foobar.rb:42: Hello world!
2024-02-30 14:07:42 <1> [Ruby] foobar.rb:43: ...
2024-02-30 14:07:42 <1> [Ruby] foobar.rb:44: ...
This might be useful if the date actually changes in the same y2log, or if multiple y2log files stretching over multiple dates are involved.
This strips common completely useless messages that only obscur important ones:
[Pkg] modules/PackageCallbacks.rb:9999 Pkg Builtin called: Callback...
[libstorage] SystemCmd.cc(doExecute):999 stopwatch ...
[libstorage] SystemCmd.cc(execute):999 timestamp ...
[libstorage] SystemCmd.cc(step_fork_and_exec):999 child_pid:9999
[libstorage] SystemCmd.cc(SystemCmd):999 constructor SystemCmd(...)
[libstorage] SystemCmd.cc(add_line):713 line stdout[99] 'loop99'
[agent-ini] IniParser.cc(getTimeStamp):999 Unable to stat '/etc/sysctl.conf'
[agent-ini] IniParser.cc(getTimeStamp):999 Unable to stat '/etc/install.inf'
This also shortens the boilerplate stdout output for libstorage and cheetah shell commands:
2024-04-16 17:36:56 <1> myhost(9999) [libstorage] SystemCmd.cc(add_line):713 line stdout[0] 'foo'
2024-04-16 17:36:56 <1> myhost(9999) [libstorage] SystemCmd.cc(add_line):713 line stdout[1] 'bar'
2024-04-16 17:36:56 <1> myhost(9999) [libstorage] SystemCmd.cc(add_line):713 line stdout[2] 'baz'
to
2024-04-16 17:36:56 <1> myhost(9999) 'foo'
2024-04-16 17:36:56 <1> myhost(9999) 'bar'
2024-04-16 17:36:56 <1> myhost(9999) 'baz'
2024-04-16 15:03:48 <1> myhost(9999) [Ruby] lib/cheetah.rb(log_stream_line):211 Standard output: foo
2024-04-16 15:03:48 <1> myhost(9999) [Ruby] lib/cheetah.rb(log_stream_line):211 Standard output: bar
2024-04-16 15:03:48 <1> myhost(9999) [Ruby] lib/cheetah.rb(log_stream_line):211 Standard output: baz
to
2024-04-16 15:03:48 <1> myhost(9999) foo
2024-04-16 15:03:48 <1> myhost(9999) bar
2024-04-16 15:03:48 <1> myhost(9999) baz
The context is still clear from the line immediately preceding the block of output which contains the executed command.
This cleans up misleading error log lines written by Cheetah which just logs all regular stdout of commands that it invokes as errors. Because reasons. And despite the bug report about this that was filed years ago.
This makes searching for real errors a royal PITA because a simple dracut run
will easily flood the y2log with some hundred error log lines, drowning out all
real errors.
This script simply downgrades all <3> (error) log lines to <1>
(milestone) if cheetah is found in the same line. This operation is
in-place in the file.
Usage:
y2log-clean-cheetah y2log
This is included in the more general y2log-clean script.
This strips off debug lines from one or more y2log files in-place; it removes
lines with log level <0> like these:
2024-02-30 14:07:42 <0> myhost(4711) [Ruby]
14:07:42 <0> myhost(4711) [Ruby]
14:07:42 <0> [Ruby]
14:07:42 <0> myhost(4711) [Ruby]
Sometimes it's useful to have debug logging, but mostly it's just additional noise that drowns out useful information.
Since this operates in-place, it is advised to copy the original file with debug logging away to a safe place before using this script:
cp y2log y2log-debug
y2log-strip-debug y2log
It is advised to do this together with the other scripts here:
y2log-clean y2log
cp y2log y2log-debug
y2log-strip-debug y2log
Now there are two cleaned-up y2logs: y2log without debug logging and
y2log-debug with.
This extracts an XML AutoYaST profile from an y2log and formats it in a human-readable way, with \n translated to newlines etc. Typically, this needs to be an y2log with debug logging.
Usage:
y2log-get-xml-profile y2log-debug >profile.xml
At the time of this writing, this uses the first XML profile that it finds. This might change in the future.
This searches one or more y2log files for XML AutoYaST profiles. It cuts off the output after a reasonable amount of characters to avoid flooding the terminal window.
% y2log-grep-xml-profile y2log-debug
02:25:14 ScriptingAgent.cc(executeSubagentCommand):592 opt: "<?xml version=\"1.0\"?>\n<!DOCTYPE
profile>\n<profile xmlns=\"http://www.suse.com/1.0/yast2ns\"
xmlns:config=\"http://www.suse.com/1.0/configns\">\n <bootloader t=\"map\">\n <global t=\"map\">\n
<append>showopts</append>\n <t
02:25:14 ../../libscr/src/include/scr/Y2AgentComponent.h(evaluate):95 evaluate (`Write (.string,
"/tmp/YaST2-06833-60VX5P/base_profile.xml", "<?xml version=\"1.0\"?>\n<!DOCTYPE profile>\n<profile
xmlns=\"http://www.suse.com/1.0/yast2ns\" xmlns:config=\"http://www.suse.com/1.0/configns\">\n
<bootloa
02:25:14 ../../libscr/src/include/scr/Y2AgentComponent.h(evaluate):100 Going to evaluate `Write
(.string, "/tmp/YaST2-06833-60VX5P/base_profile.xml", "<?xml version=\"1.0\"?>\n<!DOCTYPE
profile>\n<profile xmlns=\"http://www.suse.com/1.0/yast2ns\"
xmlns:config=\"http://www.suse.com/1.0/configns\">\n
...
...
...
21:26:15 ../../libscr/src/include/scr/Y2AgentComponent.h(evaluate):100 Going to evaluate `Write
(.string, "/tmp/YaST2-06833-60VX5P/autoinst.xml", "<?xml version=\"1.0\"?>\n<!DOCTYPE
profile>\n<profile xmlns=\"http://www.suse.com/1.0/yast2ns\"
xmlns:config=\"http://www.suse.com/1.0/configns\">\n <bo
21:26:15 ../../libscr/src/include/scr/Y2AgentComponent.h(evaluate):121 After code evaluation: `Write
(.string, "/tmp/YaST2-06833-60VX5P/autoinst.xml", "<?xml version=\"1.0\"?>\n<!DOCTYPE
profile>\n<profile xmlns=\"http://www.suse.com/1.0/yast2ns\"
xmlns:config=\"http://www.suse.com/1.0/configns\">\n
*** Number of profiles: 24
This extracts the external commands executed with the libstorage SystemCmd class to get a good
impression of storage operations:
y2log-grep-systemcmd
By default, this uses y2log-full.log (the result of y2log-merge), and if there is no file with
that name, y2log. You can also specify a file on the command line.
Since libstorage executes a lot of udevadm settle commands that are normally not very relevant, it
is usually a good idea to filter those out:
% y2log-grep-systemcmd | grep -v 'udevadm settle'
02:25:13 /usr/bin/uname -m
02:25:13 /usr/bin/test -d /sys/firmware/efi/efivars
02:25:13 /usr/bin/getconf PAGESIZE
02:25:13 /bin/ls -1 --sort=none /sys/block
02:25:13 /usr/bin/stat --format %f /dev/sda
02:25:13 /usr/bin/udevadm info /dev/sda
02:25:14 /usr/bin/uname -m
02:25:14 /usr/bin/test -d /sys/firmware/efi/efivars
02:25:14 /usr/bin/getconf PAGESIZE
21:26:15 /usr/bin/uname -m
21:26:15 /usr/bin/test -d /sys/firmware/efi/efivars
21:26:15 /usr/bin/getconf PAGESIZE
21:30:00 /sbin/multipath -d -v 2
21:30:00 /sbin/mdadm --assemble --scan
21:30:00 /sbin/dmraid --activate yes --no_partitions
21:30:00 /sbin/lvs --reportformat json --config log { command_names = 0 prefix = "" } --units b --nosuffix --all --options lv_name,lv_uuid,vg_name,vg_uuid,lv_role,lv_attr,lv_size,origin_size,segtype,stripes,stripe_size,chunk_size,pool_lv,pool_lv_uuid,origin,origin_uuid,data_lv,data_lv_uuid,metadata_lv,metadata_lv_uuid
21:30:01 /sbin/blkid -c /dev/null
21:30:01 /usr/bin/uname -m
21:30:01 /usr/bin/test -d /sys/firmware/efi/efivars
21:30:01 /usr/bin/getconf PAGESIZE
21:30:01 /bin/ls -1 --sort=none /sys/block
21:30:01 /usr/bin/stat --format %f /dev/sda
21:30:01 /usr/bin/udevadm info /dev/sda
21:30:01 /usr/bin/lsscsi --version
21:30:01 /usr/bin/lsscsi --transport
21:30:01 /sbin/multipath -d -v 2 -ll
21:30:01 /sbin/dmraid --sets=active -ccc
21:30:01 /sbin/blkid -c /dev/null
21:30:01 /usr/sbin/parted --version
21:30:01 /usr/sbin/parted --script --machine /dev/sda unit s print