Toolchain update (GCC 13.1.1, binutils 2.40) #226

vhotspur · 2023-07-25T14:03:39Z

I was unable to build our current toolchain on Fedora 37 so I decided to try to update it to something newer. This PR contains all changes required to make it work with newer GCC and binutils.

I would appreciate if someone would review the changes and test them (I have not yet tested it across all architectures).

A lot of changes in the code are related to better warnings from the compiler: some actually revealed true bugs that I tried to fix, some has to be suppressed.

Note that cpptest fails miserably but it seems to me that the same is happening on master branch too. Not sure why, it looks to me like if the vtable gets corrupted somehow.

__attribute__(packed) actually implies also align(1): thus the compiler can place the whole structure to arbitrary address. This commit fixes that by adding explicit align to some of these structures (where compiler produced a warning). Following reports explain the issues around packed attribute: * https://gcc.gnu.org/bugzilla/show_bug.cgi?id=93910 * https://gcc.gnu.org/bugzilla/show_bug.cgi?id=96293

jxsvoboda · 2023-07-25T19:14:38Z

Hi Vojta!

Thanks a lot for working on this!

Looks good to me overall. I added some small nits. I am not entirely sure I understand the changes to lexicographical_compare.

Thanks!
Jiri

le-jzr · 2023-08-04T20:56:23Z

I believe there's a way for libgcc to be built in multiple versions and GCC to automatically select the appropriate one.

Unfortunately other things are eating my time until the end of the month, but after that I'll be free to look into it again.

vhotspur · 2023-08-04T21:22:26Z

Jakub, thanks again! I pushed some more changes, mainly to allow testing with current toolchain (GCC 8) on top of my modifications plus some updates to the FPU configuration.

At the moment, I can build everything and boot of integratorcp and Raspberry profiles in QEMU works (on Raspberry it is terminal only in kernel console but I guess that is okay). I have no way to test any of the arm32 builds on real hardware, though.

But I admit that it is becoming more and more a mumbo jumbo of "let's try that and see what it does" for me :-).

Jirka, that would be great. Thank you.

jermar · 2023-08-05T05:58:08Z

Hi Vojta, I will be able to retest on the real hw again after Aug 14.

vhotspur · 2023-09-15T05:50:50Z

I have changed the arm32 configuration to use softfp instead.

I have tested (almost) all available emulators from tools/ew.py and:

arm64/virt fails with Synchronous Exception at 0x000000007C405FB8
arm32/raspberrypi boots in kernel console only (cannot switch to user terminal)
ia64/ski page-faults in VFS, the counter points into some futex code
sparc64/niagara not tested
riscv64 hangs on Booting the kernel...

Others look healthy.

jxsvoboda · 2023-09-29T07:00:51Z

I successfully tested on a real BeagleBoard xM. Looking good! See my report on helenos-devel for details.

jermar · 2023-09-30T15:51:18Z

Cannot reproduce the arm64/virt fail. Seems to work fine for me. arm32/raspberrypi behavior is expected. I can reproduce ia64/ski crash, but so far have no idea what is causing it. I had tested sparc64/niagara before and it worked fine with these changes. riscv64 I have no idea.

martin-decky · 2023-10-02T13:13:09Z

The riscv64 port is still unfinished, thus there's nothing beyond Booting the kernel... and that should not be considered a blocker for this PR.

vhotspur · 2023-10-02T14:01:14Z

I successfully tested on a real BeagleBoard xM. Looking good! See my report on helenos-devel for details.

Thank you!

Cannot reproduce the arm64/virt fail. Seems to work fine for me.

Good, thanks!

The riscv64 port is still unfinished, thus there's nothing beyond Booting the kernel... and that should not be considered a blocker for this PR.

OK, makes sense.

Thanks a lot to everyone for the testing!

So I assume that I can merge this and create ticket for ia64/ski failure as the only regression?

jxsvoboda · 2023-10-03T10:23:59Z

Sounds good.

jxsvoboda · 2023-10-03T12:30:20Z

I'd really like to thank you for doing this. I have an additional request, though. Since you've just gone through the process of porting our custom GCC and binutils targets to new version in HelenOS/gcc and Helenos/binutils-gdb, would you be able to document that process in HelenOS wiki? It probably requires some voodoo with git branches and it would be really nice if people didn't have to reinvent the wheel each time we need to do this...

vhotspur · 2023-10-12T18:17:40Z

About the toolchain upgrading.

The process is very similar for both GCC and binutils-gdb projects.

Add another Git remote with upstream development.
- git://sourceware.org/git/binutils-gdb.git
- git://gcc.gnu.org/git/gcc.git
Find a branch on the upstream with the right version.
- binutils-2_40-branch
- releases/gcc-13
Create a new branch starting on the last commit from branch from the previous step.
- binutils-2_40-helenos
- 13_1_1-helenos
Rebase changes in previous *-helenos branch onto the branch from the previous step.
- There will be conflicts: mostly in various *config* files as new targets are added or obsoleted ones are removed. Usually resolving them is straightforward.
Update the version numbers in toolchain.sh.

Pro-tip :-)

Set up the repositories in toolchain.sh to point to your local clones to speed-up the cloning.
- Or setup the clones in the downloads directory and comment-out all Git operations.

Not sure if I remembered all the big steps but this is at least rough outline. If you believe even this little is useful, I will copy it to the Wiki.

jermar

Thanks Vojta! I think we are good to merge this. Possible issue shall be addressed lazily.

le-jzr · 2023-10-21T16:26:47Z

Sorry about the long radio silence. I had a good long look at the GCC/binutils ports and everything looks good to me on that front. There's a couple things in here that I want to take another look tomorrow, like the packed structs that seem misused to me. A good chunk of commits here can (and should) be applied to master regardless of toolchain. The bug fixes and such. Mind if I cherry-pick and push those while taking a look at the rest?

le-jzr · 2023-10-22T18:08:53Z

Merged most of it with little changes, the notable ones being that I folded together some of the revisions of prior commits, and I removed the offending packed attribute rather than layering another on top (590cb6d).

That leaves two explicit casts between enums that I feel like should have a more elegant solution, which I'm trying to figure out.
And 7433234 which I don't understand the reason for. The commit message says it's "unused", but the code is very much used, so if it's unused in some other way I'd like a better explanation of the reason.

le-jzr · 2023-10-22T22:04:34Z

Did a bit of cleanup in the toolchain ports, and created new updated branches with binutils 2.41, GDB 13.2 and GCC 13.2.

martin-decky · 2023-10-23T17:14:36Z

BTW: Since GDB supports multi-arch now and most distributions actually distribute a multi-arch build of GDB, would it make sense to remove GDB from our toolchain script and thus save some time and disk space? Or does anyone see a major benefit of keeping our own GDB builds?

I vaguely remember discussing this already somewhere, but I am unable to find a written record of it. Thus let's reiterate here. Maybe we can build the GDB optionally.

le-jzr · 2023-10-23T22:32:08Z

Yeah, I don't really see the point of building custom GDB. I've been using packaged multiarch whenever I needed it and haven't seen any issue. Although admittedly my use of GDB is rare and rather surface-level.

BTW, since we're talking about debuggers, I'm currently working on using DWARF data for stack traces. I also plan to look at making a rudimentary integrated debugger for the kernel, using the same. Basic stuff like inspecting stack frames and printing out variables after a panic.

martin-decky · 2023-10-24T13:32:30Z

The DWARF support and the debugger plans are great news! Looking forward to it. And to the removal of the fragile map file parsing :)

If we only need symbol information, we might also use CTF which is supposed to be simpler to parse than DWARF at run time.

le-jzr · 2023-10-24T14:10:42Z

If we only need symbol information, we might also use CTF which is supposed to be simpler to parse than DWARF at run time.

Well, for just symbol names, we could also use the raw .symtab and .strtab sections. But I do like the idea of having proper filename:line:column output, so I'm aiming for that. :)

le-jzr · 2023-10-24T14:14:15Z

And 7433234 which I don't understand the reason for. The commit message says it's "unused", but the code is very much used, so if it's unused in some other way I'd like a better explanation of the reason.

Ooooh, I get it now. It's just broken syntax and never affected the compilation. I guess that explains my past inability to get libatomic and libstdc++ to build. 🤦

le-jzr · 2023-10-24T19:50:23Z

Done. Updated toolchain.sh in master to binutils 2.41, GDB 13.2 and GCC 13.2. Everything seems to build fine on my end, and don't see any issues running emulators so far, so hopefully there's no massive breakage in that extra minor version.
Whoever has access to ci.helenos.org should update it, including the old Meson there.

I was hoping that after the merge commit above, github will recognize the pull request as merged. Seems it's not that clever, so the hard way it is.

Thanks @vhotspur for all the work you did!

vhotspur added 28 commits July 14, 2023 14:41

char8_t behaves differently in C and C++

2b5628c

Silence compiler when accessing low memory addresses

851b842

Add __CONSTEXPR for headers shared by C and C++

b030416

C++: mutex::init should be constexpr

c4c90c8

Bypass compiler false-alarm

10e56b7

Kernel: do not check segment access rights

f1e7599

C++ lib: missing header

9fc58e3

-Wimplicit-function-declaration is only for C

eac3398

C++ lib: prevent use after free (hackish)

e652d8c

C++: expand loop if types are different

fcbf101

USB: fix function signature

6ad7f2f

Forcing enum types

2e7fc4d

Silence when NULL printed with %s

3dfb152

Force correct type

7a5702f

Use keyboard, not mouse enum

2fe799a

Silence compiler in a special test

fa992e5

Add missing malloc check

d3d267b

Prevent double-free

e7dc3a4

Ensure buffer is really big enough

417e1f6

Silence compiler warning in test

c1a7a65

Remove memory leak

474fd65

Silence compiler in test

f5fc6a8

C++ strings: missing space for null terminator

1339fd9

C++: resolve ambiguous naming

1f6bf85

C++: silence misleading indentation warnings

27bfbed

Toolchain: drop unused code

7433234

Toolchain: update to binutils 2.40, GCC 13.1.1

0557618

vhotspur changed the title ~~Toolchain update~~ Toolchain update (GCC 13.1.1, binutils 2.40) Jul 25, 2023

arm32: compile kernel with general registers only

3785b39

arm32: kernel does not care about FPU

dd0e909

arm32: use softfp configuration

315d487

jermar self-assigned this Oct 12, 2023

jermar approved these changes Oct 12, 2023

View reviewed changes

Merge branch 'master' into ticket

1c6c3e1

Merge branch 'master' into ticket

350ec74

le-jzr closed this Oct 24, 2023

le-jzr reopened this Oct 24, 2023

le-jzr merged commit 85ec7d4 into master Oct 24, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Toolchain update (GCC 13.1.1, binutils 2.40) #226

Toolchain update (GCC 13.1.1, binutils 2.40) #226

vhotspur commented Jul 25, 2023 •

edited

jxsvoboda commented Jul 25, 2023

le-jzr commented Aug 4, 2023

vhotspur commented Aug 4, 2023

jermar commented Aug 5, 2023 •

edited

vhotspur commented Sep 15, 2023

jxsvoboda commented Sep 29, 2023

jermar commented Sep 30, 2023

martin-decky commented Oct 2, 2023

vhotspur commented Oct 2, 2023

jxsvoboda commented Oct 3, 2023

jxsvoboda commented Oct 3, 2023

vhotspur commented Oct 12, 2023

jermar left a comment

le-jzr commented Oct 21, 2023

le-jzr commented Oct 22, 2023

le-jzr commented Oct 22, 2023 •

edited

martin-decky commented Oct 23, 2023

le-jzr commented Oct 23, 2023

martin-decky commented Oct 24, 2023

le-jzr commented Oct 24, 2023

le-jzr commented Oct 24, 2023

le-jzr commented Oct 24, 2023

Toolchain update (GCC 13.1.1, binutils 2.40) #226

Toolchain update (GCC 13.1.1, binutils 2.40) #226

Conversation

vhotspur commented Jul 25, 2023 • edited

jxsvoboda commented Jul 25, 2023

le-jzr commented Aug 4, 2023

vhotspur commented Aug 4, 2023

jermar commented Aug 5, 2023 • edited

vhotspur commented Sep 15, 2023

jxsvoboda commented Sep 29, 2023

jermar commented Sep 30, 2023

martin-decky commented Oct 2, 2023

vhotspur commented Oct 2, 2023

jxsvoboda commented Oct 3, 2023

jxsvoboda commented Oct 3, 2023

vhotspur commented Oct 12, 2023

jermar left a comment

Choose a reason for hiding this comment

le-jzr commented Oct 21, 2023

le-jzr commented Oct 22, 2023

le-jzr commented Oct 22, 2023 • edited

martin-decky commented Oct 23, 2023

le-jzr commented Oct 23, 2023

martin-decky commented Oct 24, 2023

le-jzr commented Oct 24, 2023

le-jzr commented Oct 24, 2023

le-jzr commented Oct 24, 2023

vhotspur commented Jul 25, 2023 •

edited

jermar commented Aug 5, 2023 •

edited

le-jzr commented Oct 22, 2023 •

edited