-
-
Notifications
You must be signed in to change notification settings - Fork 451
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Possibility of a sane linker script language? #4
Comments
Personally I would like
|
I have to think very carefully how to satisfy users' need that is currently (more or less) satisfied with linker scripts. The current GNU-style linker script is too complicated, and I have no plan to support it as-is. Being said that, I still have to support a subset of it, as (at least on Linux) /usr/lib/x86_64-linux-gnu/libc.so is not a shared object file but actually an ASCII text containing a linker script. I have a few random ideas:
|
@rui314 I saw this thread a few months ago and I have a use case for you and would be interested in your opinion on this. The linker script in question is here: https://github.com/wendigojaeger/ZigGBA/blob/291f66ff0fc70dc9e5ed4a33b8425e798e14357d/GBA/gba.ld This linker script is able to take object files with sections for the header and code and link them to create a valid GameboyAdvance ROM file. I thought it was pretty clever when I saw it, however, I can't help but ask whether it would be simpler to implement this using a custom tool that just merges the files together. As far as I can tell the script isn't doing any relocation, but then again I don't quite understand everything the script is actually doing. It does appear to be doing some things with different sections, but again, it's hard to tell without spending enough time learning how GNU's linker script language works. If I understood your comment above, it sounds like you would recommend axing this linker script and just writing a custom script or tool to combine the sections together. Is that right? P.S. this script doesn't fail if the |
Assert failure: /usr/include/c++/11.1.0/span:276: std::span::reference std::span<ElfSym<X86_64>, 18446744073709551615> ::operator[](std::span::size_type) const [_Type = ElfSym<X86_64>, _Extent = 18446744073709551615]: Assertion '__idx < size()' failed. GDB stacktrace: Thread 2.4 "mold" received signal SIGABRT, Aborted. [Switching to Thread 0x7fabc3d3f640 (LWP 17804)] 0x00007fabc496fd22 in raise () from /usr/lib/libc.so.6 (gdb) bt #0 0x00007fabc496fd22 in raise () from /usr/lib/libc.so.6 #1 0x00007fabc4959862 in abort () from /usr/lib/libc.so.6 rui314#2 0x000055a9685eb018 in std::__replacement_assert (...) at /usr/include/c++/11.1.0/x86_64-pc-linux-gnu/bits/c++config.h:504 rui314#3 0x000055a968614c47 in std::span<ElfSym<X86_64>, ...>::operator[] (...) at /usr/include/c++/11.1.0/span:276 rui314#4 ObjectFile<X86_64>::get_shndx (this=<...>, esym=...) at ./mold.h:2145 rui314#5 ObjectFile<X86_64>::get_section (this=<...>, esym=...) at ./mold.h:2154 Signed-off-by: Nehal J Wani <nehaljw.kkd1@gmail.com>
Assert failure: /usr/include/c++/11.1.0/span:276: std::span::reference std::span<ElfSym<X86_64>, 18446744073709551615> ::operator[](std::span::size_type) const [_Type = ElfSym<X86_64>, _Extent = 18446744073709551615]: Assertion '__idx < size()' failed. GDB stacktrace: Thread 2.4 "mold" received signal SIGABRT, Aborted. [Switching to Thread 0x7fabc3d3f640 (LWP 17804)] 0x00007fabc496fd22 in raise () from /usr/lib/libc.so.6 (gdb) bt #0 0x00007fabc496fd22 in raise () from /usr/lib/libc.so.6 #1 0x00007fabc4959862 in abort () from /usr/lib/libc.so.6 #2 0x000055a9685eb018 in std::__replacement_assert (...) at /usr/include/c++/11.1.0/x86_64-pc-linux-gnu/bits/c++config.h:504 #3 0x000055a968614c47 in std::span<ElfSym<X86_64>, ...>::operator[] (...) at /usr/include/c++/11.1.0/span:276 #4 ObjectFile<X86_64>::get_shndx (this=<...>, esym=...) at ./mold.h:2145 #5 ObjectFile<X86_64>::get_section (this=<...>, esym=...) at ./mold.h:2154 Signed-off-by: Nehal J Wani <nehaljw.kkd1@gmail.com>
It's not possible with the script as written since the (I do GBA stuff in Rust and wrote my own linker script which I uploaded as a gist with some explanatory comments. It provides some flexibility as to where code and data get put since the GBA has multiple blocks of work RAM with different properties. This might be useful to you as another data point for "interesting things you can do with linker scripts".) |
mold: error: undefined symbol: __atomic_fetch_add_8 >>> referenced by compress.cc >>> out/compress.o:(void tbb::detail::d1::fold_tree<tbb::detail::d1::tree_node>(tbb::detail::d1::node*, tbb::detail::d1::execution_data const&))>>> referenced by arch-arm64.cc >>> out/elf/arch-arm64.o:(tbb::detail::d2::for_each_root_task_base<__gnu_cxx::__normal_iterator<mold::elf::InputSection<mold::elf::ARM64>**, std::span<mold::elf::InputSection<mold::elf::ARM64>*, 4294967295u> >, mold::elf::create_range_extension_thunks(mold::elf::Context<mold::elf::ARM64>&, mold::elf::OutputSection<mold::elf::ARM64>&)::{lambda(mold::elf::InputSection<mold::elf::ARM64>*)rui314#2}, mold::elf::InputSection<mold::elf::ARM64>*>::cancel(tbb::detail::d1::execution_data&))>>> referenced by arch-arm64.cc >>> out/elf/arch-arm64.o:(tbb::detail::d2::for_each_root_task_base<__gnu_cxx::__normal_iterator<mold::elf::InputSection<mold::elf::ARM64>**, std::span<mold::elf::InputSection<mold::elf::ARM64>*, 4294967295u> >, mold::elf::create_range_extension_thunks(mold::elf::Context<mold::elf::ARM64>&, mold::elf::OutputSection<mold::elf::ARM64>&)::{lambda(mold::elf::InputSection<mold::elf::ARM64>*)rui314#4}, mold::elf::InputSection<mold::elf::ARM64>*>::cancel(tbb::detail::d1::execution_data&))>>> referenced 971 more times mold: error: undefined symbol: __atomic_load_8 >>> referenced by main.cc >>> out/elf/main.o:(mold::elf::Context<mold::elf::X86_64>::~Context())>>> referenced by output-chunks.cc >>> out/macho/output-chunks.o:(mold::macho::ExportEncoder::finish())>>> referenced by main.cc >>> out/elf/main.o:(mold::elf::Context<mold::elf::I386>::~Context())>>> referenced 3 more times
mold: error: undefined symbol: __atomic_fetch_add_8 >>> referenced by compress.cc >>> out/compress.o:(void tbb::detail::d1::fold_tree<tbb::detail::d1::tree_node>(tbb::detail::d1::node*, tbb::detail::d1::execution_data const&))>>> referenced by arch-arm64.cc >>> out/elf/arch-arm64.o:(tbb::detail::d2::for_each_root_task_base<__gnu_cxx::__normal_iterator<mold::elf::InputSection<mold::elf::ARM64>**, std::span<mold::elf::InputSection<mold::elf::ARM64>*, 4294967295u> >, mold::elf::create_range_extension_thunks(mold::elf::Context<mold::elf::ARM64>&, mold::elf::OutputSection<mold::elf::ARM64>&)::{lambda(mold::elf::InputSection<mold::elf::ARM64>*)rui314#2}, mold::elf::InputSection<mold::elf::ARM64>*>::cancel(tbb::detail::d1::execution_data&))>>> referenced by arch-arm64.cc >>> out/elf/arch-arm64.o:(tbb::detail::d2::for_each_root_task_base<__gnu_cxx::__normal_iterator<mold::elf::InputSection<mold::elf::ARM64>**, std::span<mold::elf::InputSection<mold::elf::ARM64>*, 4294967295u> >, mold::elf::create_range_extension_thunks(mold::elf::Context<mold::elf::ARM64>&, mold::elf::OutputSection<mold::elf::ARM64>&)::{lambda(mold::elf::InputSection<mold::elf::ARM64>*)rui314#4}, mold::elf::InputSection<mold::elf::ARM64>*>::cancel(tbb::detail::d1::execution_data&))>>> referenced 971 more times mold: error: undefined symbol: __atomic_load_8 >>> referenced by main.cc >>> out/elf/main.o:(mold::elf::Context<mold::elf::X86_64>::~Context())>>> referenced by output-chunks.cc >>> out/macho/output-chunks.o:(mold::macho::ExportEncoder::finish())>>> referenced by main.cc >>> out/elf/main.o:(mold::elf::Context<mold::elf::I386>::~Context())>>> referenced 3 more times Signed-off-by: Jan Palus <jpalus@fastmail.com>
Is there any update on this situation? Is a linker script language possible/planned? |
I implemented an experimental feature as a replacement for the linker script language. It's not documented, but you can see how to use in this test file: https://github.com/rui314/mold/blob/main/test/elf/section-order.sh The The problem here is that I don't know if the feature is towards the right direction, so I cannot make it an official feature of the linker yet. |
@rui314 I just played around with this to see if we can use this for systemd-boot. For various reasons we have to build EFI binaries as ELF and then convert to PE (cross-building is a pain, and not all arches can be built natively to PE). It would basically replace https://github.com/systemd/systemd/blob/main/tools/elf2efi.lds. From what I can tell, what we want can mostly be done with: But applying the section alignment is rather repetitive. Would be nice if we could use wildcards for section names But we still need a means to merge sections so that we can put BSS into DATA. Both, gnu-efi and edk2 state that there are firmwares that don't handle BSS correctly. Though, they also merge got etc. into it too, which we also do. Maybe Also, since there are several sections that we are going to discard anyways, it would be nice if we could move them to the end of our custom order so that the final vma address layout is more compact. Would be easy and generic if section names could be defined with wildcards, which you could put at the end of section-order. This way one could even do more specific layouts like |
@medhefgo Thank you for trying out that feature! As to the repeated As to turning bss into data, we could simply add a new command line option to tell the linker to turn all bss sections into data. As to /DISCARD/, debug sections doesn't consume VMA but instead they just consume the disk space. You can trim them off with |
Placing .bss into data segment is as easy as this is: https://gist.github.com/rui314/1922c5f8b177b459be82aab601ffde9d |
We already set max and common page size to 4K, and manpage says separate-loadable-segments is the default. But that doesn't help anyways as PE doesn't have the concept of loadable segments. PE sections are the equivalent to ELF segments and must be aligned to page boundaries. So I also wouldn't mind something like
I suppose that would work. But I would prefer to merge sections as there is no way of telling if some borked up firmware can handle multiple rw sections. As it is now, EFI binaries with edk2/gnu-efi only have one rw data section. And microsoft does the same thing with their bootloader.
While converting the ELF file, we already skip any sections we don't care for. Moving any sections we don't care for is mostly my OCD wanting to have a compact VMA space. Though, it would reduce runtime cost as the PE memory is allocated in one big allocation by EFI loaders and afaik, there is no virtual address space at this point (not that these few KB make a real difference, though). |
@medhefgo In mold, we do not provide a way to control the layout of segments, and you can only control the layout of sections. mold automatically creates the minimum number of segments that make sense and covers all the given sections. I think I like this design because it is very straightforward. On the other hand, GNU ld allows users to specify both segment and section layout using linker script, and they can conflict with each other (i.e. you can easily specify impossible layout with the script.) I guess when edk2/gnu-efi copies file contents from an ELF file to an PE file, it works based on sections? I think it should work based on segments instead, becasue that's how the Unix linker works. I.e. instead of copying |
That's fine by me, and I would agree that linker scripts are an abomination.
Well, it has to be section based copy as PE does not have a notion of segments. Copying like this would have to cram all sections that are part of a segment together into one PE section, no? But those sections need to be accessible on their own right. In particular Right now bfd produces this with our linker script:
Which gets converted to this PE layout:
So if we copy by segment, we'd use segment 0 as .text, 1 as .rodata, 2 as .data. But then other EFI consumers would lose the ability to inspect the EFI binary and access .sdmagic, .osrel and .sbat as they would be crammed into one .data section. |
So I looked into this a little more and sure, one can use section-order+allocate-bss to get a segment list that is similar to the PE sections layout we want. But that still requires that special sections like .sbat etc are split to their own segments or we cannot make them addressable as specific PE sections. As I see it, that would still require a means to force certain sections to get their separate segments. Ultimately, this would force the conversion script to understand both sections and segments so that we only copy segments/sections we care for and not copy things like .dnymic/.dynstr which are meaningless once we converted to PE. This all seems far more complicated than just walking through the section table itself and far less error prone. |
Wrt section merging: How about a means to rename sections instead? So if one were to pass |
@rui314 You might be qualified to answer this. I'm considering dropping the linker scripts altogether and do the things it does as part of the post-processing tool. The binaries are static-pies and only contain relative relocations (there are no DSOs). Now given that, is it legal to apply extra extra offsets to the output sections to satisfy the alignment requirements, possibly even doing negative offsets to reorder sections (which would then also allows merging sections). This all assumes we then apply the appropriate offsets when converting the relocations. Would this break some assumption that the linker/compiler makes? (Also, if this can only be properly done by linking with |
@medhefgo What do you mean by negative offsets? Section/segment offsets are addresses, so they are naturally non-negative. Are you asking if you can overlay sections at the same address? |
Any offsets would only be done so that no sections would overlap and any original section alignment would still be satisfied:
To, for example, re-order these you'd apply
(This is talking about the output VMAs.) |
Now that I think of it, re-arranging things during post-processing would break debugging (even if it's never used in practice) as we have to use the ELF binary for symbolization and therefore its must have identical memory layout... |
For anyone interested: I went with something similar to the suggested copying of ELF segments. I simply copy+concat any desired ELF section while adding padding to observe page and section alignments and splitting according to page permissions. It gives me the same result as copying by segments while making it easy to strip unwanted sections such as ELF dynamic linking stuff which has no use after conversion. Since I cannot guaranteed whether special info sections like Now we can easily support any linker that has static pie and |
The actual reason I stumbled across mold was, somehow, searching for information about linker scripts.
(Specifically, I was searching in the context of there seemingly not existing anything for generating a "default" linker script, for, e.g., whatever the default configuration of a linker is Edit: https://stackoverflow.com/questions/28688484/actual-default-linker-script-and-settings-gcc-uses).
I currently have no real understanding of linker scripts, so sorry if this goes in the wrong direction.
There seems to be very little material on dealing with linker scripts and I imagine part of it is that it's hard to deal with linkers and linker scripts? Maybe it doesn't need to be that way. Old *nix tools seem to have a pathology of hard to use domain specific languages? - such as xkeyboard config files which are also powerful but massively underdocumented?
I saw you mentioned in your readme that you don't intend to implement ld scripts, nevertheless there is a linker_script.cc?
In any case for my actual proposal; a new linker implementation with the potential to - pardon my language - disrupt the linker space, seems like a good opportunity to try to introduce some better scripting?
I naively see a few possibilities here
I hope this is not incompatible with your speed demon dreams?
(1) would provide a full featured language out of the box, meaning you don't have to implement any general purpose semantics yourself. With all the advantages that come with it, and without the issues of home-rolling another language. (?)
For (2):
(a) I say from personal experience that Nix/NixOS for example has quite a lot of success using a purely functional langugage for declaratively describing builds - another variant of "generating files". The approach used uses the DSL to generate a low level configuration which is then executed by the build evaluation semantics (for example, the Nixpkgs library has functionality for generating certain bash scripts, which then are run by the builder sandboxes). It may be possible to "precompile/preevaluate" scripts to help with possible speed issues. (I don't know how things would benchmark.)
(b) SMT-LIB appears to use the SExpr approach for example: http://smtlib.cs.uiowa.edu/papers/smt-lib-reference-v2.6-r2017-07-18.pdf
The human issue mentioned could later be solved by using some manner of frontend, if deemed necessary, but it's "perfectly" writable by hand.
This would also(, as a significant point?, ) allow other tools to interface better with the linker if flexibility is necessary.
The text was updated successfully, but these errors were encountered: