Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

aarch64: rr crashes immediately upon replay -e #3568

Open
BenWibking opened this issue Aug 4, 2023 · 8 comments · May be fixed by #3569
Open

aarch64: rr crashes immediately upon replay -e #3568

BenWibking opened this issue Aug 4, 2023 · 8 comments · May be fixed by #3569

Comments

@BenWibking
Copy link

BenWibking commented Aug 4, 2023 •
edited
Loading

On Fedora 38 aarch64 (Apple Silicon), this can be reproduced with:

[benwibking@fedora ~]$ rr_install/bin/rr record echo "Hello, world"
rr: Saving execution to trace directory `/home/benwibking/.local/share/rr/echo-2'.
Hello, world
[benwibking@fedora ~]$ rr_install/bin/rr replay -e
[ERROR src/Registers.cc:299:maybe_print_reg_mismatch()] x[30] 0x41ffffb0998d40 != 0x7cffffb0998d40 (replaying vs. recorded)
[FATAL src/Registers.cc:409:compare_register_files()] 
 (task 24832 (rec:24825) at time 7)
 -> Assertion `!bail_error || match' failed to hold. Fatal register mismatch (ticks/rec:2748/2748)
Tail of trace dump:
=== Start rr backtrace:
rr_install/bin/rr(_ZN2rr9GdbServer15emergency_debugEPNS_4TaskE+0xe0)[0x48b320]
rr_install/bin/rr[0x49cc1c]
rr_install/bin/rr[0x49d81c]
rr_install/bin/rr(_ZN2rr9Registers22compare_register_filesEPNS_10ReplayTaskEPKcRKS0_S4_S6_NS_16MismatchBehaviorE+0x90)[0x518584]
rr_install/bin/rr(_ZN2rr10ReplayTask13validate_regsEj+0x164)[0x52dc04]
rr_install/bin/rr(_ZN2rr13ReplaySession13enter_syscallEPNS_10ReplayTaskERKNS0_15StepConstraintsE+0x2b4)[0x51f504]
rr_install/bin/rr(_ZN2rr13ReplaySession11replay_stepERKNS0_15StepConstraintsE+0x104)[0x522c04]
rr_install/bin/rr(_ZN2rr14ReplayTimeline19replay_step_forwardENS_10RunCommandE+0x84)[0x53b748]
rr_install/bin/rr(_ZN2rr9GdbServer12serve_replayERKNS0_15ConnectionFlagsE+0x70)[0x486e00]
rr_install/bin/rr[0x5193a4]
rr_install/bin/rr(_ZN2rr13ReplayCommand3runERSt6vectorINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESaIS7_EE+0x444)[0x519d64]
rr_install/bin/rr(main+0x168)[0x433998]
/lib64/libc.so.6(+0x30598)[0xffffb6730598]
/lib64/libc.so.6(__libc_start_main+0x9c)[0xffffb6730670]
rr_install/bin/rr(_start+0x30)[0x434f30]
=== End rr backtrace
Launch gdb with
  gdb '-l' '10000' '-ex' 'set sysroot /' '-ex' 'target extended-remote 127.0.0.1:24832' /home/benwibking/.local/share/rr/echo-2/mmap_clone_5_echo
@Keno
Copy link
Member

Keno commented Aug 5, 2023

Is pointer authentication enabled in your kernel? rr currently doesn't capture those keys, I don't think.

@BenWibking
Copy link
Author

Is pointer authentication enabled in your kernel? rr currently doesn't capture those keys, I don't think.

I'm using Fedora Asahi, so I'm not 100% sure, but it appears that pointer authentication has been in the upstream Fedora ARM64 kernel since Fedora 33 (https://fedoraproject.org/wiki/Changes/Aarch64_PointerAuthentication).

Have users on Apple Silicon built their own kernels to use rr? Or does only Fedora enable pointer authentication by default?

@Keno
Copy link
Member

Keno commented Aug 5, 2023

The apple silicon support was developed with an early version of the Asahi kernel, which I'm fairly sure didn't have PAC enabled. I don't know what Asahi Fedora remix is doing, but I wouldn't be surprised if that got enabled in the process.

@BenWibking
Copy link
Author

I'll rebuild the kernel and see it works.

@yuyichao
Copy link
Contributor

yuyichao commented Aug 5, 2023

I think this is probably not related to early vs later asahi but that arch Linux kernel doesn’t have this enabled vs fedora that does.

@Keno
Copy link
Member

Keno commented Aug 5, 2023

I don't think fixing this properly is that bad, we just need to dump the NT_ARM_PACA_KEYS/NT_ARM_PACG_KEYS regsets into the trace.

Keno added a commit that referenced this issue Aug 5, 2023
@Keno
aarch64: Save/Restore PAC keys
920add3 
Should fix #3568.
@Keno Keno linked a pull request Aug 5, 2023 that will close this issue
Open
@Keno
Copy link
Member

Keno commented Aug 5, 2023

Try #3569.

@BenWibking
Copy link
Author

It works!

@rocallahan rocallahan mentioned this issue Oct 25, 2023
Closed
@eggyal eggyal mentioned this issue Nov 24, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

aarch64: Save/Restore PAC keys rr-debugger/rr
3 participants
@yuyichao @Keno @BenWibking