Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

aarch64: rr crashes immediately upon replay -e #3568

Closed
BenWibking opened this issue Aug 4, 2023 · 8 comments
Closed

aarch64: rr crashes immediately upon replay -e #3568

BenWibking opened this issue Aug 4, 2023 · 8 comments

Comments

@BenWibking
Copy link

BenWibking commented Aug 4, 2023

On Fedora 38 aarch64 (Apple Silicon), this can be reproduced with:

[benwibking@fedora ~]$ rr_install/bin/rr record echo "Hello, world"
rr: Saving execution to trace directory `/home/benwibking/.local/share/rr/echo-2'.
Hello, world
[benwibking@fedora ~]$ rr_install/bin/rr replay -e
[ERROR src/Registers.cc:299:maybe_print_reg_mismatch()] x[30] 0x41ffffb0998d40 != 0x7cffffb0998d40 (replaying vs. recorded)
[FATAL src/Registers.cc:409:compare_register_files()] 
 (task 24832 (rec:24825) at time 7)
 -> Assertion `!bail_error || match' failed to hold. Fatal register mismatch (ticks/rec:2748/2748)
Tail of trace dump:
=== Start rr backtrace:
rr_install/bin/rr(_ZN2rr9GdbServer15emergency_debugEPNS_4TaskE+0xe0)[0x48b320]
rr_install/bin/rr[0x49cc1c]
rr_install/bin/rr[0x49d81c]
rr_install/bin/rr(_ZN2rr9Registers22compare_register_filesEPNS_10ReplayTaskEPKcRKS0_S4_S6_NS_16MismatchBehaviorE+0x90)[0x518584]
rr_install/bin/rr(_ZN2rr10ReplayTask13validate_regsEj+0x164)[0x52dc04]
rr_install/bin/rr(_ZN2rr13ReplaySession13enter_syscallEPNS_10ReplayTaskERKNS0_15StepConstraintsE+0x2b4)[0x51f504]
rr_install/bin/rr(_ZN2rr13ReplaySession11replay_stepERKNS0_15StepConstraintsE+0x104)[0x522c04]
rr_install/bin/rr(_ZN2rr14ReplayTimeline19replay_step_forwardENS_10RunCommandE+0x84)[0x53b748]
rr_install/bin/rr(_ZN2rr9GdbServer12serve_replayERKNS0_15ConnectionFlagsE+0x70)[0x486e00]
rr_install/bin/rr[0x5193a4]
rr_install/bin/rr(_ZN2rr13ReplayCommand3runERSt6vectorINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESaIS7_EE+0x444)[0x519d64]
rr_install/bin/rr(main+0x168)[0x433998]
/lib64/libc.so.6(+0x30598)[0xffffb6730598]
/lib64/libc.so.6(__libc_start_main+0x9c)[0xffffb6730670]
rr_install/bin/rr(_start+0x30)[0x434f30]
=== End rr backtrace
Launch gdb with
  gdb '-l' '10000' '-ex' 'set sysroot /' '-ex' 'target extended-remote 127.0.0.1:24832' /home/benwibking/.local/share/rr/echo-2/mmap_clone_5_echo
@Keno
Copy link
Member

Keno commented Aug 5, 2023

Is pointer authentication enabled in your kernel? rr currently doesn't capture those keys, I don't think.

@BenWibking
Copy link
Author

Is pointer authentication enabled in your kernel? rr currently doesn't capture those keys, I don't think.

I'm using Fedora Asahi, so I'm not 100% sure, but it appears that pointer authentication has been in the upstream Fedora ARM64 kernel since Fedora 33 (https://fedoraproject.org/wiki/Changes/Aarch64_PointerAuthentication).

Have users on Apple Silicon built their own kernels to use rr? Or does only Fedora enable pointer authentication by default?

@Keno
Copy link
Member

Keno commented Aug 5, 2023

The apple silicon support was developed with an early version of the Asahi kernel, which I'm fairly sure didn't have PAC enabled. I don't know what Asahi Fedora remix is doing, but I wouldn't be surprised if that got enabled in the process.

@BenWibking
Copy link
Author

I'll rebuild the kernel and see it works.

@yuyichao
Copy link
Contributor

yuyichao commented Aug 5, 2023

I think this is probably not related to early vs later asahi but that arch Linux kernel doesn’t have this enabled vs fedora that does.

@Keno
Copy link
Member

Keno commented Aug 5, 2023

I don't think fixing this properly is that bad, we just need to dump the NT_ARM_PACA_KEYS/NT_ARM_PACG_KEYS regsets into the trace.

Keno added a commit that referenced this issue Aug 5, 2023
@Keno
Copy link
Member

Keno commented Aug 5, 2023

Try #3569.

@BenWibking
Copy link
Author

It works!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants