Get deathstar.com running in qemu

[Theldus]

I'm not getting to run 'hello.com' provided in examples/ folder on bare-metal, If I run on Qemu (qemu-system-i386) I get:

αcτµαlly pδrταblε εxεcµταblε
error: S

on qemu-system-x86_64, the machine seems to restart in a loop (maybe tripple fault?)

My environment:
GCC 9.3, Binutils 2.33, Qemu 2.12.0, Bash 4.4.5, Linux v4.4.38, x86_64

Steps to reproduce:

git clone https://github.com/jart/cosmopolitan && cd cosmopolitan/
make -j8 -O o//examples/hello.com
o//examples/hello.com # Runs fine on Linux*
qemu-system-i386 o//examples/hello.com

* If I try to run on Qemu without running on Linux first, Qemu (both -system-i386 and -system-x86_64) get 'frozen' on:
Boot from Hard Disk....

Am I missing something?

[jart]

You're using qemu-system-i386 but cosmpolitan needs x86_64. Please be warned the bootloader has undergone some recent changes that haven't been tested with qemu yet. If you could help us fix any bugs in the bare metal process then I would adore you, since it's so challenging, but so rewarding at the same time.

Intel CPU bugs like Spectre and Meltdown have significantly increased the performance overhead of having an operating system. If we can find a way to democratize ring0 privileges by making each executable its own unikernel, then we can gain back the performance edge that was lost. Unikernels can move memory without copying it. They can do i/o without the 1us syscall cost. It feels like the next logical evolution for cloud computing environments, where diskless servers that just need network is the norm.

[Theldus]

You're using qemu-system-i386 but cosmpolitan needs x86_64.

As I said before, I tried with both qemu-system-i386 and qemu-system-x86_64, but in the latter Qemu just hangs in
Booting from Hard Disk....

Please be warned the bootloader has undergone some recent changes that haven't been tested with qemu yet. If you could help us fix any bugs in the bare metal process then I would adore you, since it's so challenging, but so rewarding at the same time.

Make sense, and sure, I have to study better how cosmopolitan works under the hood and how it integrates with bare metal, but as I find something that I can contribute, I will for sure.

Intel CPU bugs like Spectre and Meltdown have significantly increased the performance overhead of having an operating system. If we can find a way to democratize ring0 privileges by making each executable its own unikernel, then we can gain back the performance edge that was lost. Unikernels can move memory without copying it. They can do i/o without the 1us syscall cost. It feels like the next logical evolution for cloud computing environments, where diskless servers that just need network is the norm.

I completely agree with you, it would be amazing to see cosmopolitan moving in that direction in the future.

[Theldus]

Hi @jart,
After analyzing the cosmopolitan boot process, I identified at least three points that prevent 'hello.com' from working as expected:

1. In __map_image, the XD bit is set for the data segment; it turns out that according to the Intel SDM V3A § 4.5, Table 4-20:

63 (XD) - If IA32_EFER.NXE = 1, execute-disable (if 1, instruction fetches are not allowed from the 4-KByte page controlled by this entry; see Section 4.6); otherwise, reserved (must be 0).

Hence, the NXE bit of the EFER register must be set if XD is used. Otherwise, a page fault is triggered, with the RSVD error bit set.

2. The pcread (ape.S) function incorrectly reads the sectors of the disk: the cylinder counter register (cx) is incremented before the head counter (dh), thus, when the first sectors are read (usually 18 in 1.44MiB floppy), pcread jumps to the reading of the 36 sector, instead of the 19 (CHS (1, 0, 1) instead of (0, 1, 1)); that loads an inconsistent image from disk to the memory.

3. When 1) and 2) are fixed, the code runs fine until _start, which during the execution of ischardev tries to execute the syscall fstat as if it were from a Linux environment, which is not the case. The 'call trace' looks like this:

#0  systemfive.linux
#1  __fstat$sysv
#2  fstat$sysv
#3  ischardev
#4  _init_g_stdout2
#5  _construct
#6  _spawn
#7  _executive
#8  _start

As the code tries to perform a syscall, a triple fault occurs. I believe that systemfive.linux should not be invoked from bare metal, but I also don't know what should be done instead.

---

Anyway, I have the fixes for 1) and 2), and I can send a PR separately for each of the two if you wish. As for 3), I'm still trying to understand.

[jart]

@Theldus deathstar.com now boots in qemu, thanks to you! https://justine.lol/cosmopolitan/cosmo-metal-qemu.png

make -j12 o//tool/viz/deathstar.com
qemu-system-x86_64 -serial stdio -fda o//tool/viz/deathstar.com  # boot in qemu
o//tool/viz/deathstar.com  # run on local computer from userspace

I'm very impressed by how quickly and perfectly you diagnosed the issue. The project should be fully caught up with all the claims I've made. New binaries are now published to https://justine.lol/. I hope I'll get the chance to review more issues and/or pull requests from you in the future.

[Theldus]

My pleasure, I am very happy to be able to contribute and also to see cosmopolitan finally* working on baremetal, now I can really play with it, and of course, whenever I find something I will try to contribute.

* In fact there is still a tiny issue when trying to run deathstar.com or even hello.com: in your patch that corrects the CHS arithmetic, you compare the number of heads with the sector counter (cx) instead of head counter (dh), I believe something like:

diff --git a/ape/ape.S b/ape/ape.S
index 6dd6e08e..c07ee98f 100644
--- a/ape/ape.S
+++ b/ape/ape.S
@@ -348,8 +348,8 @@ pcread:	push	%ax
 	jbe	2f
 	mov	$1,%al
 	inc	%dh			# ++head
-	cmp	XLM(DRIVE_LAST_HEAD),%cx
-	jb	2f
+	cmp	XLM(DRIVE_LAST_HEAD),%dh
+	jbe	2f
 	xor	%dh,%dh
 	inc	%cx			# ++cylinder
 2:	ret
@@ -1571,4 +1571,4 @@ __data_start:
 	.type	__ubsan_types_end,@object

Fixing this, everything works like a charm. Thanks for your time on this issue, I really appreciate it.

[jart]

Thanks again @Theldus. The change is now integrated. I confirmed it's still working for me, in both Blinkenlights and QEMU.