Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

sysdump causing grml-hwinfo to hang #3

Closed
jkirk opened this issue Nov 14, 2020 · 0 comments
Closed

sysdump causing grml-hwinfo to hang #3

jkirk opened this issue Nov 14, 2020 · 0 comments

Comments

@jkirk
Copy link
Contributor

jkirk commented Nov 14, 2020

Similar to issue #1 and grml/grml#55 sysdump causes grml-info to hang. This time on a Desktop PC (MSI X99S GAMING 7 (MS-7885) + Intel Core i7-5820K + 16GB).

Currently no time to debug longer, but running sysdump | less (as root) ends up here:

[...]
can't open /sys/devices/virtual/thermal/cooling_device5/stats/reset: 'Permission denied'
can't open /sys/devices/virtual/thermal/cooling_device6/stats/reset: 'Permission denied'
can't open /sys/devices/virtual/thermal/cooling_device7/stats/reset: 'Permission denied'
can't open /sys/devices/virtual/thermal/cooling_device8/stats/reset: 'Permission denied'
can't open /sys/devices/virtual/thermal/cooling_device9/stats/reset: 'Permission denied'
can't open /sys/firmware/efi/vars/del_var: 'Permission denied'
can't open /sys/firmware/efi/vars/new_var: 'Permission denied'
can't open /sys/fs/ext4/dm-0/trigger_fs_error: 'Permission denied'

Pressing CTRL-C once at that point continues sysdump:

/sys:
/sys/block:
/sys/block/dm-0 -> ../devices/virtual/block/dm-0
/sys/block/loop0 -> ../devices/virtual/block/loop0
/sys/block/loop1 -> ../devices/virtual/block/loop1
/sys/block/loop10 -> ../devices/virtual/block/loop10
/sys/block/loop11 -> ../devices/virtual/block/loop11
/sys/block/loop12 -> ../devices/virtual/block/loop12
/sys/block/loop13 -> ../devices/virtual/block/loop13
/sys/block/loop14 -> ../devices/virtual/block/loop14
/sys/block/loop15 -> ../devices/virtual/block/loop15
/sys/block/loop16 -> ../devices/virtual/block/loop16
/sys/block/loop17 -> ../devices/virtual/block/loop17
/sys/block/loop18 -> ../devices/virtual/block/loop18
[...]
/sys/kernel/tracing/per_cpu:
/sys/kernel/tracing/per_cpu/cpu0:
/sys/kernel/tracing/per_cpu/cpu0/buffer_size_kb = '0'
/sys/kernel/tracing/per_cpu/cpu0/snapshot = '# tracer: nop
#
#
# * Snapshot is freed *
#
# Snapshot commands:
# echo 0 > snapshot : Invalid for per_cpu snapshot file.
# echo 1 > snapshot : Allocates snapshot buffer, if not already allocated.
#                      Takes a snapshot of the main buffer for this cpu.
# echo 2 > snapshot : Clears this cpu's snapshot buffer (but does not allocate)
#                      (Doesn't have to be '2' works with any number that
#                       is not a '0' or '1')'

And seems to finish fine then. Hope this gives a clue where to look for the problem.
If you need more infos, please let me know.
jkirk added a commit to jkirk/grml-hwinfo that referenced this issue Nov 14, 2020
@jkirk
[WIP] sysdump: ignore some files in /sys/kernel/tracing to avoid hang
89c5ee1 
Identified the following files in /sys/kernel/tracing which cause hang:

* /sys/kernel/tracing/trace_pipe
* /sys/kernel/tracing/per_cpu/cpu[0-9]+/trace_pipe
* /sys/kernel/tracing/per_cpu/cpu[0-9]+/trace_pipe_raw
* /sys/kernel/tracing/per_cpu/cpu[0-9]+/snapshot_raw

To make sysdump more robust for future blocking files,
ignoring files which contain 'pipe' seems obvious.
Instead ignoring the files explictly like this:

  if ($file =~ "/sys/kernel/tracing/per_cpu/cpu[0-9]+/trace_pipe*") {
  if ($file =~ "/sys/kernel/tracing/trace_pipe") {

Moved print $file to the beginning of the block
to make it easier to debug, in case other files cause blocking.

Closes: grml#3
jkirk added a commit to jkirk/grml-hwinfo that referenced this issue Nov 14, 2020
@jkirk
sysdump: ignore some files in /sys/kernel/tracing to avoid hang
0b2f902 
Identified the following files in /sys/kernel/tracing which cause hang:

* /sys/kernel/tracing/trace_pipe
* /sys/kernel/tracing/per_cpu/cpu[0-9]+/trace_pipe
* /sys/kernel/tracing/per_cpu/cpu[0-9]+/trace_pipe_raw
* /sys/kernel/tracing/per_cpu/cpu[0-9]+/snapshot_raw

To make sysdump more robust for future blocking files,
ignoring files which contain 'pipe' seems obvious.
Instead ignoring the files explictly like this:

  if ($file =~ "/sys/kernel/tracing/per_cpu/cpu[0-9]+/trace_pipe*") {
  if ($file =~ "/sys/kernel/tracing/trace_pipe") {

Moved print $file to the beginning of the block
to make it easier to debug, in case other files cause blocking.

Closes: grml#3
@jkirk jkirk mentioned this issue Nov 14, 2020
Closed
jkirk added a commit to jkirk/grml-hwinfo that referenced this issue Jan 27, 2021
@jkirk
sysdump: ignore some files in /sys/kernel/tracing to avoid hang
3e584aa 
Identified the following files in /sys/kernel/tracing which cause hang:

* /sys/kernel/tracing/trace_pipe
* /sys/kernel/tracing/per_cpu/cpu[0-9]+/trace_pipe
* /sys/kernel/tracing/per_cpu/cpu[0-9]+/trace_pipe_raw
* /sys/kernel/tracing/per_cpu/cpu[0-9]+/snapshot_raw

Instead of ignoring the files explicitly like this:

  if ($file =~ "/sys/kernel/tracing/per_cpu/cpu[0-9]+/trace_pipe*") {
  if ($file =~ "/sys/kernel/tracing/trace_pipe") {

and to make sysdump more robust for future blocking files,
it seemed more obvious to ignore all files which contain '*_pipe*'.

Also moved print $file to the beginning of the block
to make it easier to debug, in case other files cause blocking.

Closes: grml#3
jkirk added a commit to jkirk/grml-hwinfo that referenced this issue Feb 12, 2021
@jkirk
sysdump: ignore some files in /sys/kernel/tracing to avoid hang
852bc3b 
Identified the following files in /sys/kernel/tracing which cause hang:

* /sys/kernel/tracing/trace_pipe
* /sys/kernel/tracing/per_cpu/cpu[0-9]+/trace_pipe
* /sys/kernel/tracing/per_cpu/cpu[0-9]+/trace_pipe_raw
* /sys/kernel/tracing/per_cpu/cpu[0-9]+/snapshot_raw

Instead of ignoring the files explicitly like this:

  if ($file =~ "/sys/kernel/tracing/per_cpu/cpu[0-9]+/trace_pipe*") {
  if ($file =~ "/sys/kernel/tracing/trace_pipe") {

and to make sysdump more robust for future blocking files,
it seemed more obvious to ignore all files which contain '*_pipe*'.

Also moved print $file to the beginning of the block
to make it easier to debug, in case other files cause blocking.

Closes: grml#3
@mika mika closed this as completed in 3d53667 Mar 5, 2021
mika added a commit that referenced this issue Mar 5, 2021
@mika
Provide information to user if sysdump might be hanging
9193fd5 
Sometimes new files show up in the kernel and sysdump
might be hanging while reading them. To make this more
visible to the user and how to track this down, provide
information, if not running with -q/-quiet.

Related to #3
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

sysdump: ignore some files in /sys/kernel/tracing to avoid hang jkirk/grml-hwinfo
1 participant
@jkirk