Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

loongarch support #108

Closed
nickdesaulniers opened this issue Jun 23, 2023 · 11 comments
Closed

loongarch support #108

nickdesaulniers opened this issue Jun 23, 2023 · 11 comments
Labels
enhancement New feature or request good first issue Good for newcomers help wanted Extra attention is needed

Comments

@nickdesaulniers
Copy link
Member

cc @xen0n

@xen0n sent patches for loongarch enablement and mentioned qemu support.

@xen0n can you please share the qemu command line invocation you use for boot testing? That will help us wire up support for boot testing the kernel image.

Do you happen to know if buildroot supports loongarch?

@nickdesaulniers nickdesaulniers added enhancement New feature or request good first issue Good for newcomers help wanted Extra attention is needed labels Jun 23, 2023
@groeck
Copy link

groeck commented Jun 23, 2023 via email

@xen0n
Copy link
Member

xen0n commented Jun 23, 2023

Thanks for the prompt followup after the patch series!

My QEMU invocation FYI:

qemu-system-loongarch64 \
        -m 4G \
        -smp 4 \
        -bios ./QEMU_EFI_7.2.fd \
        -nographic \
        -net nic -net user \
        -hda ./rw.qcow2 \
        -kernel ./vmlinuz.efi -initrd ./initramfs-linux.img \
        -append 'root=UUID=fbaa5a22-85ff-4127-82fb-efb7eba03595 rw rootfstype=ext4 loglevel=7 console=ttyS0,115200'

This is a simplistic setup with @yetist's Arch Linux port. It could be useful but given we build our own sysroots I guess we only have to take the firmware from there.

The QEMU firmware can be built from 100% open-source code, and I've previously confirmed it to be working, so we could do that as well if anyone's doubting whether we can legally just grab that prebuilt firmware blob. Last time I asked they're working on integrating the firmware into the upstream QEMU build process too; it seems they've been busy doing other things but eventually this part should become as painless as other arches.

Now it's a bit late here in UTC+8, so I'd have to go AFK soon, but I guess the info should be enough to get something together. Thanks again for the heads-up!

@xen0n
Copy link
Member

xen0n commented Jun 23, 2023

Update: apparently the firmware is officially hosted at https://github.com/loongson/Firmware/tree/main/LoongArchVirtMachine -- maintained directly by the Loongson firmware team (who are very helpful and responsive). This could be more accessible (I'm not sure about connectivity to the Loong Arch Linux mirrors outside of China -- it's not globally mirrored yet) and having an issue tracker is definitely better.

@groeck
Copy link

groeck commented Jun 23, 2023 via email

@nathanchance
Copy link
Member

I have this tentatively wired up:

http://github.com/nathanchance/boot-utils/commits/loongarch

However, I see a crash when running /init when booting an LLVM=1 kernel:

$ ./boot-qemu.py -k $TMP_BUILD_FOLDER/linux-next

QEMU location: /usr/sbin

QEMU version: QEMU emulator version 8.0.2
$ timeout --foreground 3m stdbuf -eL -oL /usr/sbin/qemu-system-loongarch64 -display none -nodefaults -M virt -cpu la464 -bios $PWD/images/loongarch/edk2-loongarch64-code.fd -no-reboot -append console=ttyS0,115200 -kernel $KBUILD/arch/loongarch/boot/vmlinuz.efi -initrd $PWD/images/loongarch/rootfs.cpio -m 4G -smp 2 -serial mon:stdio
...
[    0.000000] Linux version 6.4.0-rc7-next-20230626+ (nathan@dev-arch.thelio-3990X) (ClangBuiltLinux clang version 17.0.0 (https://github.com/llvm/llvm-project 1a93ff55bc809487cd9f39ab6285df0a3019c8fc), ClangBuiltLinux LLD 17.0.0) #1 SMP PREEMPT Mon Jun 26 14:27:07 MST 2023
...
[    1.557056] Run /init as init process
[    1.575643] ------------[ cut here ]------------
[    1.579839] kernel BUG at arch/loongarch/kernel/traps.c:851!
[    1.580207] Oops - BUG[#1]:
[    1.580451] CPU: 0 PID: 1 Comm: init Not tainted 6.4.0-rc7-next-20230626+ #1
[    1.580738] Hardware name: QEMU QEMU Virtual Machine, BIOS unknown 2/2/2022
[    1.580962] pc 9000000000223e24 ra 9000000000223d90 tp 9000000100098000 sp 900000010009be90
[    1.581252] a0 0000000000000000 a1 0000000000402000 a2 0000000000400000 a3 0000000000000003
[    1.581489] a4 0000000000000000 a5 00007ffffc3901b8 a6 0000000000000004 a7 0000000000000030
[    1.581699] t0 90000000024d4850 t1 ffffffffffffffff t2 2424242424242424 t3 00007ffff3e93f48
[    1.581877] t4 7efefefefefefeff t5 810f070303010103 t6 4d4824160a4b570a t7 fffffffffefef000
[    1.582105] t8 ffffffffffffffff u0 900000000049cb94 s9 0000000000000000 s0 0000000000000000
[    1.582310] s1 00007ffff3ead2d0 s2 00007ffffbf8d8f0 s3 0000000000000000 s4 00007ffffbf8d8f0
[    1.582527] s5 0000555557d1febf s6 0000000000000000 s7 00007ffffbf8d8f0 s8 0000555557dfbdf0
[    1.582755]    ra: 9000000000223d90 init_restore_fp+0x50/0xe8
[    1.583280]   ERA: 9000000000223e24 init_restore_fp+0xe4/0xe8
[    1.583437]  CRMD: 000000b0 (PLV0 -IE -DA +PG DACF=CC DACM=CC -WE)
[    1.584064]  PRMD: 00000004 (PPLV0 +PIE -PWE)
[    1.584213]  EUEN: 00000001 (+FPE -SXE -ASXE -BTE)
[    1.584385]  ECFG: 00071c1c (LIE=2-4,10-12 VS=7)
[    1.584685] ESTAT: 000c0000 [BRK] (IS= ECode=12 EsubCode=0)
[    1.584871]  PRID: 0014c010 (Loongson-64bit, Loongson-3A5000)
[    1.585086] Process init (pid: 1, threadinfo=(____ptrval____), task=(____ptrval____))
[    1.585426] Stack : 900000010009bec0 90000000024d48a0 0000000000000000 00007ffff3ead228
[    1.585726]         00007ffffbf8de30 90000000033a1f24 0000000000000000 00007ffff3e93fc0
[    1.585927]         0000000000000000 00007ffffbf8d770 00007ffffbf8d7b0 0000000000000000
[    1.586147]         00007ffffbf8d998 0000000000000000 0000000000000000 00007ffffc3901b8
[    1.586340]         0000000000000004 0000000000000030 00007ffffbf8d7a0 00007ffff3e82b9c
[    1.586549]         2424242424242424 00007ffff3e93f48 7efefefefefefeff 810f070303010103
[    1.586761]         4d4824160a4b570a fffffffffefef000 ffffffffffffffff 0000000000000000
[    1.586978]         00007ffffbf8de30 00007ffff3ead228 00007ffff3ead2d0 00007ffffbf8d8f0
[    1.587198]         0000000000000000 00007ffffbf8d8f0 0000555557d1febf 0000000000000000
[    1.587406]         00007ffffbf8d8f0 0000555557dfbdf0 ffffffffffffff9c 00007ffff3e9ace4
[    1.587628]         ...
[    1.587727] Call Trace:
[    1.587831] [<9000000000223e24>] init_restore_fp+0xe4/0xe8
[    1.588206] [<90000000024d48a0>] do_fpu+0x50/0xa0
[    1.588353] [<90000000033a1f24>] exception_handlers+0x1f24/0x10000
[    1.588533] [<00007ffff3e9ace4>] 0x7ffff3e9ace4
[    1.588703]
[    1.588767] Code: 28c02061  02c04063  4c000020 <002a0001> 02ffc063  29c02061  29c00076  28c02044  14000405
[    1.589106]
[    1.590614] ---[ end trace 0000000000000000 ]---
[    1.590907] note: init[1] exited with preempt_count 1
[    1.591363] Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b
[    1.592013] ---[ end Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b ]---

$ git -C $CBL_SRC/linux-next log --oneline origin/master^..
4e3dc0ab3eab LoongArch: Use CLANG_FLAGS in vDSO instead of filtering out '--target='
226fd644a791 Makefile: Add loongarch target flag for Clang compilation
1a3f5dc5d49b LoongArch: Mark Clang LTO as working
e7e2acd4388f LoongArch: Tweak CFLAGS for Clang compatibility
72d195ec176e LoongArch: Simplify the invtlb wrappers
31be70e71c7d LoongArch: Make the CPUCFG and CSR ops simple aliases of compiler built-ins
b96f1a7e225a LoongArch: Prepare for assemblers with proper FCSR class support
d44d7d406ac0 LoongArch: extable: Also recognize ABI names of registers
32c459db8e9b LoongArch: Calculate various sizes in the linker script
60e7c4a25da6 Add linux-next specific files for 20230626

$ git -C $CBL_SRC/llvm-project log --oneline origin/main^..
1a93ff55bc80 D138135
7827beeebb59 [flang][hlfir] `hlfir.char_extremum` lowering

A kernel built using GCC 13.1.0 from kernel.org has no issues.

$ ./boot-qemu.py -k $TMP_BUILD_FOLDER/linux-next-gcc
...
[    1.572243] Run /init as init process
Starting syslogd: OK
Starting klogd: OK
Running sysctl: OK
Saving 256 bits of non-creditable seed for next boot
Starting network: OK
Linux version 6.4.0-rc7-next-20230626+ (nathan@dev-arch.thelio-3990X) (loongarch64-linux-gcc (GCC) 13.1.0, GNU ld (GNU Binutils) 2.40) #1 SMP PREEMPT Mon Jun 26 16:07:35 MST 2023
Stopping network: OK
Seeding 256 bits without crediting
Saving 256 bits of non-creditable seed for next boot
Stopping klogd: OK
Stopping syslogd: OK
umount: devtmpfs busy - remounted read-only
umount: can't unmount /: Invalid argument
The system is going down NOW!
Sent SIGTERM to all processes
Sent SIGKILL to all processes
Requesting system reboot
[    4.888685] ACPI: PM: Preparing to enter system sleep state S5
[    4.889428] reboot: Restarting system

Is this expected? I don't think I am missing a patch to LLVM, but I might be.

@xen0n
Copy link
Member

xen0n commented Jun 27, 2023

Thanks for the quick follow-up. @nathanchance : Would you please share the config used to build the kernel? @chenhuacai told me yesterday privately that he backed out the KASan patches as they apparently need some more polish, but that happened after next-20230626, so I'm not sure if that's why but I'll try reproducing the crash locally.

@xen0n
Copy link
Member

xen0n commented Jun 27, 2023

Thanks for the quick follow-up. @nathanchance : Would you please share the config used to build the kernel? @chenhuacai told me yesterday privately that he backed out the KASan patches as they apparently need some more polish, but that happened after next-20230626, so I'm not sure if that's why but I'll try reproducing the crash locally.

Oh I saw in the ClangBuiltLinux/linux#1787 issue it should be the defconfig. I'll try with a setup similar to yours shortly (previously I did all Clang builds natively).

@nathanchance
Copy link
Member

@xen0n If you have any issues reproducing, please let me know!

@xen0n
Copy link
Member

xen0n commented Jun 27, 2023

I've reproduced and debugged the issue: it's llvm/llvm-project#63549 and will get fixed by https://reviews.llvm.org/D153865.

nathanchance added a commit to nathanchance/boot-utils that referenced this issue Jun 27, 2023
LoongArch requires EFI firmware to boot, which is available as a
precompiled blob from Loongson's GitHub, so it is downloaded when
running 'boot-qemu.py -a loongarch' for the first time. There is no
update mechanism like for the rootfs images for a simple implementation.
If there comes a time where the firmware has to be updated for the
machine to boot properly, such a mechanism can be implemented then.

Other than that, the rest of the support is pretty straightforward. The
defaults for cores and memory are a little higher than other machines,
as that is what other folks are testing.

Closes: ClangBuiltLinux#108
Signed-off-by: Nathan Chancellor <nathan@kernel.org>
@xen0n
Copy link
Member

xen0n commented Jun 28, 2023

@nathanchance: FYI, next-20230628 now works with LLVM main branch (I tested llvm/llvm-project@193d3ac) + https://reviews.llvm.org/D138135 applied.

Regarding the QEMU options, it could be fine to continue running with -m 2G -smp 2 (i.e. the justification you put in the commit message is fair) but minimally just -m 1G would suffice and take slightly less resources. (The LoongArch virt machine needs at least 1G due to memory layout requirements.)

And thanks for your testing efforts!

@xen0n
Copy link
Member

xen0n commented Jun 28, 2023

Ah there's already #109. I re-posted the feedback there for better visibility.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request good first issue Good for newcomers help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

4 participants