Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fail dmesg check if it contains Call Trace or segfault #2583

Merged
merged 1 commit into from
Jan 24, 2024

Conversation

guoguojenna
Copy link
Contributor

00:00:33 fail /rdma/pyverbs-tests-2 (on default-0) [1/1]
00:00:00 pass dmesg (before-test check)
00:00:02 pass avc (after-test check)
00:00:01 fail dmesg (after-test check)

When users notice dmesg fail, they can go to check the dmesg output to see if it's a bug.

Fixes #2563.

Pull Request Checklist

  • implement the feature

@happz
Copy link
Collaborator

happz commented Dec 23, 2023

@guoguojenna thanks for the patch. I for one like the idea, but I'm afraid that the check would have to be smarter to avoid spoiling all tests. Without taking timestamps into account, one segfault - even a segfault that happened right after the boot, way before tmt connected to the guest for the first time - would mark all following tests as failed. I'm afraid dmesg check would have to approach it in the way AVC check does, i.e. checking dmesg output since test's start_time, see https://github.com/teemtee/tmt/blob/main/tmt/checks/avc.py#L137C4-L137C4. I also found https://unix.stackexchange.com/questions/729243/how-to-show-dmesg-logs-within-a-certain-timeframe, but sed is probably not going to help as much as in that example. Maybe we could save /proc/uptime and check only newer lines, and avoid converting into human-readable date & time completely.

tmt/checks/dmesg.py Outdated Show resolved Hide resolved
@happz
Copy link
Collaborator

happz commented Jan 4, 2024

@guoguojenna thanks for the patch. I for one like the idea, but I'm afraid that the check would have to be smarter to avoid spoiling all tests. Without taking timestamps into account, one segfault - even a segfault that happened right after the boot, way before tmt connected to the guest for the first time - would mark all following tests as failed. I'm afraid dmesg check would have to approach it in the way AVC check does, i.e. checking dmesg output since test's start_time, see https://github.com/teemtee/tmt/blob/main/tmt/checks/avc.py#L137C4-L137C4. I also found https://unix.stackexchange.com/questions/729243/how-to-show-dmesg-logs-within-a-certain-timeframe, but sed is probably not going to help as much as in that example. Maybe we could save /proc/uptime and check only newer lines, and avoid converting into human-readable date & time completely.

Actually, dmesg -c should clean the buffer. Running dmesg -c > some-log.txt before the test should empty the buffer, and running dmesg > another-log.txt after the test would provide the pile of text to check for issues.

Copy link
Contributor Author

@guoguojenna guoguojenna left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ran the test again after this code change, and can fail the dmesg as expected.

    00:00:37 fail /rdma/pyverbs-tests-2 (on default-0) [1/2]
    00:00:01     pass dmesg (before-test check)
    00:00:02     pass avc (after-test check)
    00:00:02     fail dmesg (after-test check)

tmt/checks/dmesg.py Outdated Show resolved Hide resolved
thrix
thrix previously requested changes Jan 22, 2024
tmt/checks/dmesg.py Show resolved Hide resolved
@thrix thrix self-requested a review January 22, 2024 22:23
@thrix thrix self-assigned this Jan 23, 2024
@thrix thrix requested review from happz and juk January 23, 2024 08:51
@thrix
Copy link
Collaborator

thrix commented Jan 23, 2024

@juk @psss @happz this seems to be ready for review

@thrix thrix added this to the 1.31 milestone Jan 23, 2024
@juk
Copy link
Collaborator

juk commented Jan 23, 2024

@juk @psss @happz this seems to be ready for review

Looks good to me, thank you guoguojenna !

Copy link
Collaborator

@psss psss left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, thanks for implementing this. Proposing to extend the test coverage in 35dd22c.

@psss psss dismissed thrix’s stale review January 24, 2024 14:15

To be covered in a separate patch.

@psss psss added the area | check Test check implementation label Jan 24, 2024
Add test checking that segfault is correctly reported.

Co-authored-by: Petr Šplíchal <psplicha@redhat.com>
@psss psss changed the title Fail dmesg check if it contains "Call Trace"/"segfault" Fail dmesg check if it contains Call Trace or segfault Jan 24, 2024
@psss psss assigned psss and unassigned thrix Jan 24, 2024
@psss
Copy link
Collaborator

psss commented Jan 24, 2024

/packit test

@psss
Copy link
Collaborator

psss commented Jan 24, 2024

/packit test -i full

@psss psss merged commit 324ee07 into teemtee:main Jan 24, 2024
18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area | check Test check implementation
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Fail the dmesg check if it contains messages like "Call Trace", "segfault" and etc.
6 participants