Skip to content

virt_mshv_vtl: Trace guest crash message #1395

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 2 commits into from
May 23, 2025
Merged

virt_mshv_vtl: Trace guest crash message #1395

merged 2 commits into from
May 23, 2025

Conversation

smalis-msft
Copy link
Contributor

Upgrade these traces to warnings to make them easier to spot, mark the guest crash message as explicitly confidential for CVMs.

@smalis-msft smalis-msft requested review from a team as code owners May 21, 2025 19:39
@smalis-msft smalis-msft mentioned this pull request May 21, 2025
Merged
@smalis-msft
Copy link
Contributor Author

Example output from a linux guest (windows doesn't seem to ever report a message?)

[2810.815961] virt_mshv_vtl::processor: WARN Guest has reported a system crash message "[ 8.160636] audit: type=1400 audit(1747938932.171:9): apparmor=<5c>"STATUS<5c>" operation=<5c>"profile_load<5c>" profile=<5c>"unconfined<5c>" name=<5c>"man_groff<5c>" pid=455 comm=<5c>"apparmor_parser<5c>"<5c>n[ 8.196500] audit: type=1400 audit(1747938932.207:10): apparmor=<5c>"STATUS<5c>" operation=<5c>"profile_load<5c>" profile=<5c>"unconfined<5c>" name=<5c>"tcpdump<5c>" pid=456 comm=<5c>"apparmor_parser<5c>"<5c>n[ 8.207524] audit: type=1400 audit(1747938932.218:11): apparmor=<5c>"STATUS<5c>" operation=<5c>"profile_load<5c>" profile=<5c>"unconfined<5c>" name=<5c>"/usr/lib/NetworkManager/nm-dhcp-client.action<5c>" pid=453 comm=<5c>"apparmor_parser<5c>"<5c>n[ 8.207534] audit: type=1400 audit(1747938932.218:12): apparmor=<5c>"STATUS<5c>" operation=<5c>"profile_load<5c>" profile=<5c>"unconfined<5c>" name=<5c>"/usr/lib/NetworkManager/nm-dhcp-helper<5c>" pid=453 comm=<5c>"apparmor_parser<5c>"<5c>n[ 8.207539] audit: type=1400 audit(1747938932.218:13): apparmor=<5c>"STATUS<5c>" operation=<5c>"profile_load<5c>" profi [2810.815973] virt_mshv_vtl::processor: le=<5c>"unconfined<5c>" name=<5c>"/usr/lib/connman/scripts/dhclient-script<5c>" pid=453 comm=<5c>"apparmor_parser<5c>"<5c>n[ 11.629764] loop6: detected capacity change from 0 to 8<5c>n[ 49.958427] hv_balloon: Max. dynamic memory size: 4096 MB<5c>n[ 2770.766437] systemd-journald[266]: File /var/log/journal/76e09c24cd0c40caba03532de353644b/user-1000.journal corrupted or uncleanly shut down, renaming and replacing.<5c>n[ 2806.065843] sysrq: Trigger a crash<5c>n[ 2806.065855] Kernel panic - not syncing: sysrq triggered crash<5c>n[ 2806.065861] CPU: 0 PID: 963 Comm: bash Not tainted 6.8.0-1017-azure #20~22.04.1-Ubuntu<5c>n[ 2806.065873] Hardware name: Microsoft Corporation Virtual Machine/Virtual Machine, BIOS Hyper-V UEFI Release v4.1 11/18/2024<5c>n[ 2806.065878] Call Trace:<5c>n[ 2806.065883] <TASK><5c>n[ 2806.065891] dump_stack_lvl+0x27/0x70<5c>n[ 2806.065909] dump_stack+0x10/0x20<5c>n[ 2806.065915] panic+0x351/0x390<5c>n[ 2806.065929] sysrq_handle_crash+0x15/0x20<5c>n[ 2806.065940] [2810.815981] virt_mshv_vtl::processor: __handle_sysrq+0xf2/0x290<5c>n[ 2806.065947] write_sysrq_trigger+0x5c/0x80<5c>n[ 2806.065955] proc_reg_write+0x5e/0xa0<5c>n[ 2806.065969] ? __cond_resched+0x1a/0x50<5c>n[ 2806.065982] vfs_write+0xef/0x410<5c>n[ 2806.065994] ? do_syscall_64+0x88/0x160<5c>n[ 2806.066002] ? syscall_exit_to_user_mode+0x81/0x220<5c>n[ 2806.066015] ? filp_flush+0x5d/0x90<5c>n[ 2806.066026] ksys_write+0x67/0xf0<5c>n[ 2806.066037] __x64_sys_write+0x19/0x20<5c>n[ 2806.066047] x64_sys_call+0x167d/0x20c0<5c>n[ 2806.066056] do_syscall_64+0x7c/0x160<5c>n[ 2806.066061] ? syscall_exit_to_user_mode+0x81/0x220<5c>n[ 2806.066072] ? do_syscall_64+0x88/0x160<5c>n[ 2806.066077] ? syscall_exit_to_user_mode+0x81/0x220<5c>n[ 2806.066087] ? do_syscall_64+0x88/0x160<5c>n[ 2806.066092] ? do_syscall_64+0x88/0x160<5c>n[ 2806.066097] ? __x64_sys_ioctl+0x95/0xd0<5c>n[ 2806.066105] ? wp_page_reuse+0x86/0xa0<5c>n[ 2806.066118] ? do_wp_page+0xcb/0x450<5c>n[ 2806.066125] ? filp_flush+0x5d/0x90<5c>n[ 2806.066135] ? syscall_e [2810.815989] virt_mshv_vtl::processor: xit_to_user_mode+0x81/0x220<5c>n[ 2806.066160] ? do_syscall_64+0x88/0x160<5c>n[ 2806.066167] ? __x64_sys_fcntl+0xa9/0xd0<5c>n[ 2806.066173] ? syscall_exit_to_user_mode+0x81/0x220<5c>n[ 2806.066183] ? do_syscall_64+0x88/0x160<5c>n[ 2806.066189] ? do_user_addr_fault+0x33d/0x640<5c>n[ 2806.066197] ? irqentry_exit_to_user_mode+0x7b/0x220<5c>n[ 2806.066208] ? irqentry_exit+0x1d/0x30<5c>n[ 2806.066218] ? exc_page_fault+0x80/0x160<5c>n[ 2806.066229] entry_SYSCALL_64_after_hwframe+0x78/0x80<5c>n[ 2806.066242] RIP: 0033:0x75195a114a37<5c>n[ 2806.066261] Code: 10 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b7 0f 1f 00 f3 0f 1e fa 64 8b 04 25 18 00 00 00 85 c0 75 10 b8 01 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 51 c3 48 83 ec 28 48 89 54 24 18 48 89 74 24<5c>n[ 2806.066270] RSP: 002b:00007ffff2007198 EFLAGS: 00000246 ORIG_RAX: 0000000000000001<5c>n[ 2806.066280] RAX: ffffffffffffffda RBX: 0000000000000002 RCX: 000075195a114a37<5c>n[ 2806.066286] RDX: 0000000000000002 RSI: 0000640 [2810.815997] virt_mshv_vtl::processor: 8c9333b80 RDI: 0000000000000001<5c>n[ 2806.066290] RBP: 00006408c9333b80 R08: 0000000000000000 R09: 00006408c9333b80<5c>n[ 2806.066294] R10: 000075195a219d00 R11: 0000000000000246 R12: 0000000000000002<5c>n[ 2806.066299] R13: 000075195a21a780 R14: 000075195a216600 R15: 000075195a215a00<5c>n[ 2806.066309] </TASK><5c>n[ 2806.067376] Kernel Offset: 0x31600000 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff)<5c>n"

chris-oo
chris-oo reviewed May 23, 2025
View reviewed changes
openhcl/virt_mshv_vtl/src/processor/mod.rs
let mut message = vec![0; message_size as usize];
match self.partition.gm[vtl].read_at(message_gpa, &mut message) {
Ok(()) => {
let message = String::from_utf8_lossy(&message).into_owned();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is into_owned required here? &str doesn't implement tracing::value?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cow doesn't implement tracing::value for whatever reason.

chris-oo
chris-oo approved these changes May 23, 2025
View reviewed changes
Copy link
Member

@chris-oo chris-oo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice, thanks for doing this.

@smalis-msft smalis-msft merged commit d6f9b0a into microsoft:main May 23, 2025
27 checks passed
@smalis-msft smalis-msft deleted the log-crash-msg branch May 23, 2025 17:20
@mebersol mebersol added backport_2505 Change should be backported to the release/2505 branch and removed backport_2505 Change should be backported to the release/2505 branch labels Jun 4, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@chris-oo chris-oo

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@smalis-msft @chris-oo @mebersol