Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

UEFI? #349

Open
johalun opened this issue Aug 28, 2017 · 79 comments
Open

UEFI? #349

johalun opened this issue Aug 28, 2017 · 79 comments
Projects

Comments

@johalun
Copy link

@johalun johalun commented Aug 28, 2017

Hi! Thanks for this great blog series. It's really awesome and helping me a lot in learning how an OS works.

Do you have any plans of adding support for UEFI boot? I assume the assembly would be much simpler since UEFI setup alot of that for you, including preparing a graphical framebuffer to draw on.

As for the UEFI boot loader, I assume grub-efi can be used to load a kernel from a fat/ext/ufs partition. Or, perhaps the kernel can be put on the ESP and loaded directly by the UEFI BIOS.

If I find the time, maybe I'll play around with it and do a PR. Just wanted to make sure we're not doing double work..

@le-jzr
Copy link

@le-jzr le-jzr commented Aug 28, 2017

Booting on UEFI via grub doesn't really give you anything special. But as it happens, I'm working on a pure UEFI kernel, beginnings of which I'll upload to GitHub soon. (If you wanna try on your own, I found it best to duplicate gnu-efi build process. Their README was very helpful.)

@le-jzr
Copy link

@le-jzr le-jzr commented Aug 28, 2017

Sooo, I uploaded what I have now. https://github.com/le-jzr/sisyphos-kernel-uefi-x86_64

It's not much, but it builds, runs and prints stuff. Everything else is work in progress.

@johalun
Copy link
Author

@johalun johalun commented Aug 28, 2017

Will try it out! Thanks for sharing :)

@phil-opp
Copy link
Owner

@phil-opp phil-opp commented Aug 29, 2017

Cool! I'm planning a second edition of the tutorial with an own bootloader, since grub causes problems on many architectures. Ideally, we would support both UEFI and legacy BIOS and provide tutorials for both systems. So thanks a lot for sharing your prototype @le-jzr!

I try to create a clean branch for the second edition in the next days. I think I will focus on the legacy bootloader first, but I'll be happy to merge any UEFI related PR.

@gil0mendes
Copy link
Contributor

@gil0mendes gil0mendes commented Aug 30, 2017

@phil-opp Writing a bootloader for this series would be a great improvement 👏🏻

In times I wrote my own bootloader, but is in C and I want to convert it to Rust, but for now, I'm without time for that. :(

@toothbrush7777777
Copy link
Contributor

@toothbrush7777777 toothbrush7777777 commented Sep 19, 2017

I have been working on a UEFI kernel written in Rust, although it could equally be used to load a kernel and there hasn't been any need for any x86 Assembly except for halting the processor.

If you like, I can upload my code and share a link when the code is more presentable (I'm learning Rust and come from a VB6/VB.NET/JavaScript background).

@phil-opp
Copy link
Owner

@phil-opp phil-opp commented Sep 20, 2017

@toothbrush7777777 That would be awesome!

@toothbrush7777777
Copy link
Contributor

@toothbrush7777777 toothbrush7777777 commented Mar 14, 2018

@phil-opp @johalun I have uploaded my working example UEFI application to https://github.com/toothbrush7777777/uefi-app-x64.

Sorry for the really long delay. I've been very busy with work and travels. 😭

@phil-opp
Copy link
Owner

@phil-opp phil-opp commented Mar 17, 2018

@toothbrush7777777 That's awesome! I just tried it shortly and it compiled without problems. The code looks really short, that's great! I didn't manage to run it yet, but I will try again tomorrow.

@toothbrush7777777
Copy link
Contributor

@toothbrush7777777 toothbrush7777777 commented Mar 17, 2018

@phil-opp I’m glad you like it. The most complicated part is the code for converting to UTF-16 and printing in chunks. I haven’t finished the project yet, but it at least prints and exits correctly.

@toothbrush7777777
Copy link
Contributor

@toothbrush7777777 toothbrush7777777 commented Mar 20, 2018

@phil-opp The code doesn’t require LLVM anymore! I merged a pull request from @kkk669 to use the lld-link distributed with recent toolchain installs of Rust nightly.

@phil-opp
Copy link
Owner

@phil-opp phil-opp commented Apr 13, 2018

Hey @toothbrush7777777, sorry that I haven't replied for so long! I was busy with the BIOS bootloader and the bootimage tool.

I'm now thinking about the best way to intregrate UEFI booting into the blog and/or bootimage. I think we have the following options:

  • Build a plain UEFI app, maybe with an library that provides abstractions for common functions (loading a file, retrieving the memory map, etc.). The users have to set up and mount the disk image on their own.
  • Add UEFI support to bootimage, so that the disk image is automatically created from the kernel executable. This would be more easy for the user, but it might be lots of work (creating the file system etc.).
  • Add UEFI support to bootimage (see previous point) and create an "UEFI bootloader" with the same behavior and interface as the BIOS bootloader. This means that the user continues to have an ELF executable with GNU calling convention, a nice memory map abstraction, and already a correct page table mapping for their kernel (UEFI only does identity mapping AFAIK).

Option 3 might fit best into the blog and allows the user to start with interesting things (e.g. input and output), but it would mean a lot work for us and probably too much magic for some people (albeit this is solvable by writing additional blog posts about the lower abstraction layers).

What do you think?

@toothbrush7777777
Copy link
Contributor

@toothbrush7777777 toothbrush7777777 commented Apr 13, 2018

@phil-opp Option 3 sounds good. Setting up the disk and file-system is a lot of work, though. Someone would need to write libraries to create/edit MBR and GPT partitions, and FAT32 file-systems.
This would be similar to how the bootimage tool already works.

I won't be able to help much for a few months, though. I'm very busy with work right now.

@reynoldsbd
Copy link

@reynoldsbd reynoldsbd commented Apr 14, 2018

Hello! I have been working on a UEFI abstraction crate for Rust. It follows the UEFI spec closely and could be used to implement any of the three options listed. It also contains an example Makefile that shows how to build and run a UEFI-based application under QEMU.

My next goal for this crate is to add support for file I/O. Loading a file from disk in a UEFI environment is probably not so difficult. Section 13 of the UEFI spec describes the file system protocols that must be implemented by compliant UEFI firmware. These make it possible to do file I/O directly, without needing to implement any block device or file system drivers.

A UEFI-based version of bootimage's bootloader would just need to load the user's ELF file into memory from the EFI system partition, call UEFI's ExitBootServices routine, and then jump to the user's entry point.

@reynoldsbd
Copy link

@reynoldsbd reynoldsbd commented Apr 29, 2018

I think I have a working implementation here, but there is some bad news. Evidently, VGA is not supported when booting under UEFI. This thread has some further context.

In practical terms, this means that although the kernel can be loaded and executed under a UEFI system, it won't be able to access VGA hardware to output text. At best, the VGA driver will just be writing bytes to an unimportant memory region.

If you still wish to support UEFI, I see two options:

  1. Instead of using VGA to output text from the OS, use serial port I/O. This should work regardless of whether the system is booted under BIOS or UEFI (although I have not yet tested it), so you won't need to diverge any blog posts. The downside, of course, is that you'd need to completely replace the "VGA Text Mode" post with an analogous post about serial I/O, as well as modify the "minimal kernel" post to output "Hello world!" to serial.
  2. Write an "efifb" driver for outputting text from the kernel. This could either be as an alternative to the "VGA Text Mode" blog post, or as a "magic" support crate that users just import and use. In either case, such a driver would be much more complex than a VGA text-mode driver, because it would need to render fonts directly into a linear frame buffer. To accomplish this, we might want to reference another implementation, such as OpenBSD's efifb driver.

All things considered, I think option 1 involves less work, and I'd be happy to assist with the changes.

@phil-opp
Copy link
Owner

@phil-opp phil-opp commented Apr 29, 2018

@reynoldsbd Thanks a lot for your work! Sorry for replying so late.

Evidently, VGA is not supported when booting under UEFI.

That's a pity. Serial I/O is a good alternative, but only for emulation. Most real systems don't have a serial port anymore, so writing to a serial ports wouldn't have any observable effect. I haven't had the time to read up on UEFI, but isn't there some way to output some text on the EFI console?

I'm unsure how we should integrate UEFI into the blog, but I definitively plan to do it. It seems like the "support both BIOS and UEFI transparently from bootimage" approach doesn't quite work out. Making the blog UEFI-first would be cool, but too radical in my opinion for various reasons (there are still people with old BIOS systems, I don't have much experience with UEFI yet, emulation of UEFI is more difficult, cross-platform tooling does not exist yet, etc).

So I think the best way forward is to provide optional, non-transparent UEFI support: The bootimage tool gets support for building UEFI disk images from Windows executables and we add a post that explains how to transform the BIOS kernel into a hybrid BIOS/UEFI kernel. In this post the user learns how to use conditional compilation based on the target triple to select either the VGA text buffer or the EFI console for screen output. What do you think? I'm open to alternative ideas of course.

@reynoldsbd
Copy link

@reynoldsbd reynoldsbd commented Apr 29, 2018

UEFI does support console I/O while "boot services" are active. The catch is that the bootloader/OS must disable those boot services (by calling ExitBootServices) before it is allowed to take full control of the system (i.e. manage page tables, setup interrupts, or directly access any hardware). After that, there doesn't seem to be any alternative for getting text to the screen except by manually rendering it.

I agree, it sounds like there's no great way to transparently support both boot methods. I think we would need the following:

  • a post about adding support for UEFI boot via conditional compilation (probably via a feature flag, as the target triple would likely remain unchanged), with the caveat that screen output will not work
  • bootimage support for building a UEFI-bootable image
  • an LFB-based console output driver (no small feat...) and a corresponding post? (possibly an advanced topic)

I still think there may be value in adding a serial-based console output driver, if for no other reason than to make UEFI more accessible to a beginner (as the LFB-based driver is likely to be quite complex).

@IsaacWoods
Copy link
Contributor

@IsaacWoods IsaacWoods commented Apr 29, 2018

I've found serial invaluable for debugging even with a working VGA driver (mainly because you aren't restricted to 25 lines), so that could be a valuable post (it also isn't too tricky, if a little bit archaic).

On the LFB-based driver, we could always write a "black-box" crate, sort of like how x86_64 is used without being implemented in the blog, but is well documented. As a beginner to OSdev / Rust, writing one from scratch might seem a bit disconcerting.

@toothbrush7777777
Copy link
Contributor

@toothbrush7777777 toothbrush7777777 commented Apr 29, 2018

@reynoldsbd Then why don’t you set up the VGA buffer before (or after, with VBE) calling ExitBootServices? I seem to remember it being possible with one of the UEFI protocols.

@reynoldsbd
Copy link

@reynoldsbd reynoldsbd commented Apr 30, 2018

@toothbrush7777777 there is a "Graphics Output Protocol" (GOP) that provides access to linear frame buffers, but these buffers correspond to pixels, not characters. AFAICT there is just no such thing as VGA or VBE when booting under UEFI, or at least none that is guaranteed to be available by the specification :/

@le-jzr
Copy link

@le-jzr le-jzr commented Apr 30, 2018

Although not as simple as letting UEFI draw the glyphs, writing text to a linear framebuffer is still fairly simple, as long as you stick to a bitmap font. Hardcoding the ASCII set into an array of small bitmaps is a good start (for 8x8 glyphs, you can easily represent each glyph as u64). For larger things, one could use the GNU Unifont if one can swallow the GPL licensing.

@GabrielMajeri
Copy link

@GabrielMajeri GabrielMajeri commented Sep 9, 2018

@phil-opp I'm the author of (yet another?) Rust UEFI crate (uefi-rs). I don't claim to be an expert, but it's proven useful already in some people's hobby kernels. I'm interested in collaborating with you to get UEFI support for this awesome guide.

Plan

My idea for adding UEFI support would be something like this: we just replace the _start function with an uefi_start function (which will take in a UEFI image Handle and a reference to the SystemTable), and completly throw away the bootloader. Yes, really, people don't even have to worry what a boot loader is. People have two options of running the generated .EFI applications / kernels:

  • put them on any USB stick and run them on real computers (boot the file from an UEFI shell, or move the file to /efi/boot/bootx64.efi to boot it automatically)
  • boot them into QEMU with OVMF directly (you can tell QEMU to boot from a local directory as-if it were a FAT drive).

UEFI starts the binary in 32-bit/64-bit mode with flat paging, and GDTs are all set up. People can get a basic kernel working from Step 1, so they can play around with whatever they like.

VGA and text

As for text output, there's no need to manage the VGA text buffer anymore. The way I recommend users of my crate to do text output is to use log, and I have a simple logger which prints to UEFI's standard output.

There are many advantages of this approach: you can easily use info!() / error!() anywhere to print stuff with the right informational level, you can also modify the logger to also pipe stuff to something like a serial port, etc.

There is a massive advantage of using UEFI when it comes to graphics. VGA is legacy and hard to program, while UEFI's Graphics Output Protocol is quite high-level and is easy to use (see an example in the crate's tests, which draws some rectangles to the framebuffer).

Issue with UEFI

UEFI makes (newbie) programmers lazy. Since UEFI exposes APIs for accessing some simple filesystems, connecting to networks, has abstractions for accessing PCI devices, etc., some people might decide to use UEFI functions instead of writing their own drivers.

While it might seem advantageous for a beginner to use UEFI's functions, they won't learn anything unless they write their own drivers.

I'm hoping to get your opinion on this issue.

@GabrielMajeri
Copy link

@GabrielMajeri GabrielMajeri commented Sep 10, 2018

Here's how it looks like now, with QEMU + OVMF for UEFI support:
image

You can see my progress in porting here. As you can see, not many changes, besides removing VGA.

@vlad9486
Copy link

@vlad9486 vlad9486 commented Sep 14, 2018

Feel free to borrow code or entire crate uefi.

@GabrielMajeri IMHO, some people want to use UEFI as a standardized loader and perform "exit boot services" once booting is done. Osdev is about writing the own OS. Using UEFI API is not much different from using existing OS like Linux.

@GabrielMajeri
Copy link

@GabrielMajeri GabrielMajeri commented Sep 14, 2018

@vlad9486

Using UEFI API is not much different from using existing OS like Linux.

I specifically said I don't recommend people use UEFI APIs besides the basics.

In a computer with UEFI support and no BIOS, without using UEFI GOP or UEFI's stdout, you have no means of outputing text. You can't use VGA: you would have to write a full graphics driver for every GPU you want to support. That means thousands of lines of code to set up a framebuffer for each GPU manufacturer.
As for serial output, you can use that with QEMU, but good luck using it on a modern computer.

IMHO, some people want to use UEFI as a standardized loader and perform "exit boot services" once booting is done.

you can call ExitBootServices and keep using some of the existing protocols for standard output.
Even Linux uses UEFI's GOP if you lack a driver for your GPU.

@toothbrush7777777
Copy link
Contributor

@toothbrush7777777 toothbrush7777777 commented Sep 14, 2018

@GabrielMajeri I didn’t know you could continue using GOP services after ExitBootServices. Do you know which other protocols can be used after ExitBootServices?

@GabrielMajeri
Copy link

@GabrielMajeri GabrielMajeri commented Sep 14, 2018

@toothbrush7777777 The UEFI spec says that protocols which also work at runtime are marked so in their documentation.

AFAIK, the only protocols which keep working at runtime are:

  • GOP, but you're not allowed to change the video mode anymore, just to use the existing framebuffer you set up.

  • the network-related protocols, although that was just a recommendation, not a hard requirement.

Besides the GOP, most services transitioned to being boot time only, to discourage (ab)using them while the OS is running.

@vlad9486
Copy link

@vlad9486 vlad9486 commented Sep 14, 2018

@GabrielMajeri

I specifically said I don't recommend people use UEFI APIs besides the basics.

Ok, sorry for misunderstanding, I just reply on "Issue" section.

you have no means of outputing text

Yeah, it is a problem... It is the shame that the gpu vendors could not produce standard for framebuffer access.

@phil-opp
Copy link
Owner

@phil-opp phil-opp commented Oct 4, 2018

@GabrielMajeri Sorry for the delay, I was on vacation for the past weeks.

I haven't had the time to try it yet, but it sounds really great! The diff is much smaller than I imagined. I would be more than happy to collaborate with you to bring first-class uefi support to the blog! Thanks so much for your work!

One thing that's important to me: The guide should continue to work natively on Windows, macOS, and Linux. Is this possible with UEFI (including booting in QEMU)?

Also, I'm not 100% percent sure if we should really omit the bootloader. I like the idea of not depending on it, but the flat identity mapping done by UEFI has disadvantages to the bootloader mapping (e.g. no guard page below the stack, no support for higher half kernel). Also, we still need the bootloader for BIOS booting (which we still want to support) and I don't want to drop support for BIOS-booting.

In case we decide to keep the bootloader, we would need to add UEFI support to it. This includes adding an UEFI entry point, reading out the memory map, loading the kernel ELF file, and doing the kernel remapping. Most of these steps are already implemented, so it should be relatively little work.

In case we decide to not depend on the bootloader for UEFI, we would need to convert the bootloader to some kind of UEFI compatibility layer, so that a BIOS boot looks like an UEFI boot to the OS. This would include loading .exe binaries and providing the UEFI functions (how do they work by the way? through interrupts?). I'm not sure if this is possible, but it would be more work.

What do you think?

@IsaacWoods
Copy link
Contributor

@IsaacWoods IsaacWoods commented Apr 16, 2020

Instead, it provides a EFI_SIMPLE_TEXT_OUPUT_PROTOCOL that can be used to print text to the screen. Unfortunately, this protocol is only available until ExitBootServices is called, so we can't use it in our kernel. (@IsaacWoods Am I understanding this correctly?)

This is correct. Unless the firmware decides otherwise, protocols are not backed by runtime-allocated memory and so can disappear at any point after you call ExitBootServices. They specifically designed EFI_GRAPHICS_OUTPUT_PROTOCOL (GOP) with our use case in mind (not having real graphics drivers) and it's designed to supply enough information that you can write to the physical framebuffer after UEFI's involvement stops - GOP is only needed to set the framebuffer up, and not to write to it, whereas EFI_SIMPLE_TEXT_OUTPUT_PROTOCOL is involved in the actual output as well.

@64
Copy link

@64 64 commented Apr 16, 2020

the legacy PIC is not supported with UEFI, so we need to migrate the tutorial to the APIC

I haven’t touched OSDev for a while, but my next plan was to flesh out the APIC crate, which I own (the crate is almost completely empty). I would be happy to give people maintainer access to the crate if it helps.

@rpjohnst
Copy link

@rpjohnst rpjohnst commented Apr 16, 2020

Another option for text output, which is supported by both UEFI and BIOS, is to write to a serial port. I've used this quite a bit in the past to avoid setting up a full pixel-based framebuffer text system. It takes very little code to set up, and QEMU can forward it directly to the host console.

@phil-opp
Copy link
Owner

@phil-opp phil-opp commented Apr 17, 2020

@IsaacWoods Thanks a lot for the explanation!

@phil-opp
Copy link
Owner

@phil-opp phil-opp commented Apr 17, 2020

@64 That would be great, thanks a lot! Let me know if I can do anything to help.

@phil-opp
Copy link
Owner

@phil-opp phil-opp commented Apr 17, 2020

@rpjohnst Yes, the serial port is a good option too. I personally like screen output more because it really feels like you're creating a completely new system, while serial output feels a bit like running a normal userspace program. Also, most computers today don't have an actual serial port anymore, so serial output is not accessible when running on real hardware.

@toothbrush7777777
Copy link
Contributor

@toothbrush7777777 toothbrush7777777 commented Apr 17, 2020

@rpjohnst Yes, the serial port is a good option too. I personally like screen output more because it really feels like you're creating a completely new system, while serial output feels a bit like running a normal userspace program. Also, most computers today don't have an actual serial port anymore, so serial output is not accessible when running on real hardware.

Well, not quite. Some modern PCs do still come with serial ports. A lot of ASRock's motherboards come with internal serial port connectors.
I agree that writing to a frame-buffer does seem like creating a new system much more than writing to a serial port, though I found serial ports to be quite useful for debugging even with other VMMs like VirtualBox and VMware.

@phil-opp
Copy link
Owner

@phil-opp phil-opp commented Apr 17, 2020

Interesting, I didn't know that. Still, there are many PCs and laptops without a serial port today. I fully agree that they're more useful than a framebuffer for debugging. Maybe the best approach would be to introduce both approaches (framebuffer and serial) at the beginning.

@ethindp
Copy link

@ethindp ethindp commented Apr 17, 2020

I don't know if this helps, but there's an MSR that allows you to get the APIC memory address without reading ACPI tables. According to Volumes 3 and 4 of the Intel SDMs, this MSR is 1Bh. Section 10.4.4 notes that bit 11 is the enable APIC bit (or as they call it, the "APIC Global Enable" flag). The manual notes that "This flag is available in the Pentium 4, Intel Xeon, and P6 family processors. It is not guaranteed to be available or available at the same location in future Intel 64 or IA-32 processors." Additionally, it specifies that the default address (bits 35:12) is FEE00000H. Is this a reliable method of acquiring the APIC address?

@toothbrush7777777
Copy link
Contributor

@toothbrush7777777 toothbrush7777777 commented Apr 19, 2020

I don't know if this helps, but there's an MSR that allows you to get the APIC memory address without reading ACPI tables. According to Volumes 3 and 4 of the Intel SDMs, this MSR is 1Bh. Section 10.4.4 notes that bit 11 is the enable APIC bit (or as they call it, the "APIC Global Enable" flag). The manual notes that "This flag is available in the Pentium 4, Intel Xeon, and P6 family processors. It is not guaranteed to be available or available at the same location in future Intel 64 or IA-32 processors." Additionally, it specifies that the default address (bits 35:12) is FEE00000H. Is this a reliable method of acquiring the APIC address?

For those processors, probably. But I guess that would still depend on the firmware.

@ghost
Copy link

@ghost ghost commented May 22, 2020

@ethindp MSRs generally should be avoided when possible for portability purposes, unless they've been around for a long time and are pretty much universal for relevant hardware (e.g. that one MSR that can be used for enabling syscall support).

@phil-opp I would suggest emulating UEFI on BIOS machines and also using UEFI on machines that support it, to drastically increase portability. Simulating UEFI in the BIOS can be done with some bootloaders people made IIRC, and Rust can interface with them if they exist and if it's done the right way because it all becomes machine code at some point.

When simulating UEFI in the BIOS, make sure that the version of UEFI that the bootloader simulates is the same as the version of UEFI being used for UEFI machines. If the versions are different, this would cause unexpected behavior.

As to making edits to the tutorials, I would suggest a third edition, as the differences between BIOS and UEFI are simply too great.

It's also worth noting that the BIOS is not going to be an exposable interface forever. Intel said they would remove it from future processors starting this year in favor of UEFI, though I'm not sure if they changed their plans since they said it.

@phil-opp
Copy link
Owner

@phil-opp phil-opp commented May 24, 2020

@LiamTheProgrammer Emulating UEFI on BIOS machines sounds like a good solution, provided that this is not too much work. Otherwise, we could try to create some sort of common interface that abstracts over the BIOS/UEFI details.

I'm not really keen to create a third edition since this would be a lot of work for me and churn for readers. Instead, I think it's probably better to rewrite the individual posts that are affected by the change (the post about printing to screen and setting up hardware interrupts). We would probably need to make some adjustments to the other posts (e.g. updating the QEMU screenshots), but otherwise they should stay valid.

@ghost
Copy link

@ghost ghost commented Jun 4, 2020

@phil-opp Creating a common interface would be extremely hard. I think BIOS emulation shouldn't be too hard, but I could be wrong. It'd still be easier than developing a common interface, though, because for it to be fully-featured it would probably have to manually implement tons of useful UEFI functions from a UEFI specification (there are many of those) for the BIOS, which would eliminate the point anyway.

And yeah, you're right about the third edition thing. I shouldn't have been inconsiderate. :/

@IsaacWoods
Copy link
Contributor

@IsaacWoods IsaacWoods commented Jun 4, 2020

@LiamTheProgrammer I'm not sure creating a common interface at a high level would be too challenging - I'm imagining things like load_file, create_memory_map and stuff being called from a common layer, which would then be implemented by both the UEFI and BIOS layers independently (along with an entry point).

It definitely sounds far far easier than emulating BIOS on UEFI without platform support (which is a large project itself, and not useful for our purposes). I'm not sure it's even possible as a normal UEFI application, as UEFI has control over the IDT and mode, so how would you go about allowing real-mode-style accesses and the BIOS software interrupts? There are also quirks that robust BIOS bootloaders have to work around, like some BIOSs not being able to load segments over 64k boundaries and such, that would add pointless overhead when we can just rely on UEFI's much better file protocols.

@phil-opp phil-opp moved this from Other Planned Changes to In Progress in Roadmap Jun 15, 2020
@phil-opp
Copy link
Owner

@phil-opp phil-opp commented Jul 24, 2020

I just pushed the first prototype of the uefi bootloader here: https://github.com/rust-osdev/bootloader/tree/uefi

Use cargo uefi-build for building and cargo uefi-run for starting it in QEMU (requires OVMF). Like the BIOS bootloader, you need to set the path to your kernel binary in the KERNEL environment variable and the path to the Cargo.toml of your kernel in the KERNEL_MANIFEST_PATH environment variable when building.

The implementation currently only sets up a new page tables for mapping the kernel ELF file and then passes control to the kernel. No boot information is passed yet and no additional mappings (e.g. framebuffer, physical memory) are done yet.

@toothbrush7777777
Copy link
Contributor

@toothbrush7777777 toothbrush7777777 commented Jul 24, 2020

@phil-opp What is cargo uefi-build?

@phil-opp
Copy link
Owner

@phil-opp phil-opp commented Jul 24, 2020

The commands are cargo build-uefi/cargo run-uefi, sorry. Both are just aliases defined in the .cargo/config.toml so that you don't have to remember all required cargo arguments.

@toothbrush7777777
Copy link
Contributor

@toothbrush7777777 toothbrush7777777 commented Jul 24, 2020

@phil-opp OK, that makes sense now.

@RMuskovets
Copy link

@RMuskovets RMuskovets commented Aug 12, 2020

Any news?

@RMuskovets
Copy link

@RMuskovets RMuskovets commented Aug 12, 2020

@IsaacWoods looks like Clover bootloader does the opposite: emulating UEFI on BIOS systems

@phil-opp
Copy link
Owner

@phil-opp phil-opp commented Aug 20, 2020

Still working on it when I have time. Today I started implementing a new boot information struct including the memory map.

I also thought about redesigning the interface between the bootloader and bootimage crates. My basic idea is to move as much work as possible to the bootloader, so that bootimage can become a library that can be used from a local custom runner executable. The advantage of this would be that you don't have to install bootimage anymore. One requirement for this would be that cargo gains a way to reset a configured default target for a subdirectory. I opened a PR for this at rust-lang/cargo#8638, let's see what the cargo maintainers think about it.

@ethindp
Copy link

@ethindp ethindp commented Aug 20, 2020

@phil-opp That would be pretty neat. What do you need help with regarding the bootloader right now? I'm not ultra-well-versed in UEFI, but I'd love to help! Also, for parsing executables, instead of using xmas-elf, try goblin.

@phil-opp
Copy link
Owner

@phil-opp phil-opp commented Aug 23, 2020

Thanks for offering your help! I made some good progress over the last few days, but it's still an early prototype with lots of moving parts so I think it's difficult to collaborate at this stage. I will let you know when I finished the first prototype version, then there will probably be more opportunity to help.

Also, for parsing executables, instead of using xmas-elf, try goblin.

I worked with goblin before and it works quite well too, but I found xmas-elf more useful for no_std projects. If I remember correctly, many of goblin's abstractions are not available without the standard and alloc libraries.

@phil-opp
Copy link
Owner

@phil-opp phil-opp commented Aug 26, 2020

Copying my status update from our gitter chat:

As a short update from my side, I got the UEFI bootloader working and integrated with the BIOS bootloader. The code is available here: https://github.com/rust-osdev/bootloader/tree/uefi

I added a new builder binary that you can use to build both the BIOS and UEFI variants of the bootloader. It's aliased to cargo builder in the .cargo/config.toml. The basic usage is:

cargo builder --kernel-binary /path/to/your/kernel/binary --kernel-manifest /path/to/your/Cargo.toml --firmware uefi

For the BIOS variant, the output of this is the disk image, so you no longer need bootimage for that. For the UEFI variant, I currently only create the .efi file, but I plan to create the fat32 image at some point too. Using OVMF, you can run the .efi file directly in QEMU.

In order to fully get rid of bootimage, I created two helper crates : bootloader_locator for locating a bootloader dependency in the dependency tree and runner_utils for detecting whether an executable is a test binary and providing a run_with_timeout function that can be used for running tests.

Using these crates, it's relatively easy to write to write an own create_disk_image builder function for our kernel: https://github.com/phil-opp/blog_os/blob/db307d89bc9e52bc0ab5eb7faf28f0d6d6637b93/disk_image/src/lib.rs . Creating an own runner executable is also possible in a few lines: https://github.com/phil-opp/blog_os/blob/db307d89bc9e52bc0ab5eb7faf28f0d6d6637b93/disk_image/src/bin/runner.rs .

I hope that letting the users create the builder and runner executables themselves instead of relying on bootimage will make the build process less magic. Another advantage is that no globally-installed tool is needed anymore, so that all project dependencies are compiled directly with the project. You no longer have to keep the installed bootimage version in sync when you upgrade the bootloader crate to a new version. Also, working with fork of the bootloader is now easier too, since you directly talk to its build interface instead of requiring bootimage as a mediator.

Unfortunately, there is also a disadvantage: Since .cargo/config.toml values apply to all subdirectories, it is no longer possible to set a default target or automatically enable the -Zbuild-std parameter right now because these setting would also apply for the builder executable of the kernel, which is stored in a subfolder. To fix this, I proposed a --ignore-local-config flag for cargo in rust-lang/cargo#8643, which would allow us to keep the config values but ignore them for the builder/runner executables. (Let me know if you can think of an easier solution to this.)

@phil-opp
Copy link
Owner

@phil-opp phil-opp commented Sep 21, 2020

I finally got keyboard interrupts working with the IOAPIC, which means that the UEFI prototype now has all the features of the existing BIOS implementation 🎉.

As mentioned above, I had to switch from the VGA text buffer to a pixel based framebuffer because the former is not supported on UEFI. To keep things consistent, I also changed the BIOS implementation in the same way. The second difference is that we're no longer using the legacy PIC for hardware interrupts, since it is not supported either. Instead, we set up the local APIC and IOAPIC to do things properly.

The result looks like this for UEFI:

image

For BIOS systems, it should be exactly the same:

image

(There are only some differences in the default values of some registers.)

This means that we don't need to do anything special for either UEFI or BIOS from the kernel side. The new boot info structure provides a common interface that works with both firmware variants.

The next step is to merge and publish the new bootloader implementation (after some polishing). Then I'll update and restructure the blog for the new implementation, including rewriting some posts ("VGA Text Buffer" and "Hardware Interrupts"). Since the IOAPIC requires allocations, I plan to move the new "Hardware Interrupt" post after the memory management posts.

@phil-opp
Copy link
Owner

@phil-opp phil-opp commented Sep 21, 2020

@64 I created a small apic crate for accessing the most important registers of the local APIC and the IOAPIC. You mentioned above that you also wanted to create an apic crate. Do you want to build your own or would you consider collaborating on this under the rust-osdev organization, e.g. starting from my prototype? I don't have my code online yet, put I plan to push it in the next few days. If you're interested in collaborating, I could ping you after pushing it.

@phil-opp
Copy link
Owner

@phil-opp phil-opp commented Sep 21, 2020

@64 I pushed my implementation to https://github.com/phil-opp/apic.

@64
Copy link

@64 64 commented Sep 21, 2020

@phil-opp I’m happy to collaborate - putting it on the rust-osdev organisation sounds like a good idea.

I’ve given you owner access to the apic crate. Feel free to nuke what’s there and replace it with your prototype (my code is just a stub atm).

@phil-opp
Copy link
Owner

@phil-opp phil-opp commented Sep 21, 2020

Awesome, thanks a lot! I moved my repo to https://github.com/rust-osdev/apic and added you to new apic team with access to that repo. Feel free to extend or change my implementation if you like, it is just a first prototype.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Roadmap
In Progress
Linked pull requests

Successfully merging a pull request may close this issue.

None yet