Kernel HLE

Kernel HLE (High Level Emulation) can be used to avoid the need for an original Xbox kernel or BIOS. This would not only make it easier to users to get started, but also allow developers to gain an insight on what's currently happening inside the virtual Xbox.

Furthermore, this could be used to speed up the emulation.

How much HLE is necessary to emulate simple Xbox apps?

We'd have to emulate an XBE loader, threading emulation, and at least some exported kernel functions. Fortunately, some of the emulated kernel functions can be re-used in the XBE loader. There are about 64 commonly used kernel imports. Even the most basic software would require these. I'll group the 64 most basic ones by type, and then explain their complexity. A fully fledged game seems to have about ~120 imports (Usually more crypto functions/constants and more threading). The 4627 kernel contains 366 exports in total.

Constants / Variables

Kernel import                       : 0x80000142 (@322, XboxHardwareInfo)
Kernel import                       : 0x800000A4 (@164, LaunchDataPage)

Very easy to emulate

File I/O and hard disk drive

Kernel import                       : 0x80000028 (@40, HalDiskCachePartitionCount)
Kernel import                       : 0x800000BB (@187, NtClose)
Kernel import                       : 0x800000DA (@218, NtQueryVolumeInformationFile)
Kernel import                       : 0x800000CA (@202, NtOpenFile)
Kernel import                       : 0x800000DB (@219, NtReadFile)
Kernel import                       : 0x800000EC (@236, NtWriteFile)
Kernel import                       : 0x800000D3 (@211, NtQueryInformationFile)
Kernel import                       : 0x800000BE (@190, NtCreateFile)
Kernel import                       : 0x800000D7 (@215, NtQuerySymbolicLinkObject)
Kernel import                       : 0x800000CB (@203, NtOpenSymbolicLinkObject)
Kernel import                       : 0x80000043 (@67, IoCreateSymbolicLink)
Kernel import                       : 0x800000E2 (@226, NtSetInformationFile)

Easy to emulate

Memory handling

Kernel import                       : 0x800000D9 (@217, NtQueryVirtualMemory)
Kernel import                       : 0x800000C7 (@199, NtFreeVirtualMemory)
Kernel import                       : 0x800000B8 (@184, NtAllocateVirtualMemory)
Kernel import                       : 0x800000B4 (@180, MmQueryAllocationSize)
Kernel import                       : 0x800000A6 (@166, MmAllocateContiguousMemoryEx)
Kernel import                       : 0x800000A8 (@168, MmClaimGpuInstanceMemory)
Kernel import                       : 0x800000B2 (@178, MmPersistContiguousMemory)
Kernel import                       : 0x800000A5 (@165, MmAllocateContiguousMemory)
Kernel import                       : 0x800000AB (@171, MmFreeContiguousMemory)

Hard to emulate

AV handling

Kernel import                       : 0x80000004 (@4, AvSetSavedDataAddress)
Kernel import                       : 0x80000001 (@1, AvGetSavedDataAddress)
Kernel import                       : 0x80000002 (@2, AvSendTVEncoderOption)
Kernel import                       : 0x80000003 (@3, AvSetDisplayMode)

Easy to emulate

libc-funcs

Kernel import                       : 0x80000117 (@279, RtlEqualString)
Kernel import                       : 0x80000121 (@289, RtlInitAnsiString)
Kernel import                       : 0x8000010D (@269, RtlCompareMemoryUlong)

Easy to emulate

Date/time and timers

Kernel import                       : 0x80000080 (@128, KeQuerySystemTime)
Kernel import                       : 0x80000095 (@149, KeSetTimer)
Kernel import                       : 0x80000071 (@113, KeInitializeTimerEx)

Hard to emulate, except for KeQuerySystemTime

XBE Handling

Kernel import                       : 0x80000147 (@327, XeLoadSection)
Kernel import                       : 0x80000148 (@328, XeUnloadSection)

Easy to emulate, but lots of work and many places for Game specific bugs

HAL

Kernel import                       : 0x80000018 (@24, ExQueryNonVolatileSetting)
Kernel import                       : 0x80000168 (@360, HalInitiateShutdown)
Kernel import                       : 0x80000031 (@49, HalReturnToFirmware)
Kernel import                       : 0x8000002F (@47, HalRegisterShutdownNotification)
Kernel import                       : 0x8000002E (@46, HalReadWritePCISpace)

Easy to emulate

Threading

Kernel import                       : 0x80000102 (@258, PsTerminateSystemThread)
Kernel import                       : 0x800000FF (@255, PsCreateSystemThreadEx)
Kernel import                       : 0x80000123 (@291, RtlInitializeCriticalSection)
Kernel import                       : 0x80000126 (@294, RtlLeaveCriticalSection)
Kernel import                       : 0x80000115 (@277, RtlEnterCriticalSection)
Kernel import                       : 0x80000063 (@99, KeDelayExecutionThread)
Kernel import                       : 0x8000009F (@159, KeWaitForSingleObject)
Kernel import                       : 0x800000EA (@234, NtWaitForSingleObjectEx)
Kernel import                       : 0x800000E9 (@233, NtWaitForSingleObject)
Kernel import                       : 0x8000006B (@107, KeInitializeDpc)
Kernel import                       : 0x80000077 (@119, KeInsertQueueDpc)
Kernel import                       : 0x80000091 (@145, KeSetEvent)

Very hard to emulate, probably the deal breaker

Interrupt control

Kernel import                       : 0x80000062 (@98, KeConnectInterrupt)
Kernel import                       : 0x8000006D (@109, KeInitializeInterrupt)
Kernel import                       : 0x8000002C (@44, HalGetInterruptVector)
Kernel import                       : 0x80000064 (@100, KeDisconnectInterrupt)

Could be hard to emulate, but could also turn out to be easy

Exception Handling

Kernel import                       : 0x8000012D (@301, RtlNtStatusToDosError)
Kernel import                       : 0x80000138 (@312, RtlUnwind)
Kernel import                       : 0x8000012E (@302, RtlRaiseException)

Hard to emulate, but probably not necessary

Unused / Debug / Crash-Handling

Kernel import                       : 0x80000008 (@8, DbgPrint)
Kernel import                       : 0x8000005F (@95, KeBugCheck)

Not necessary, but easy.

Unknown

Kernel import                       : 0x800000C4 (@196, NtDeviceIoControlFile)
Kernel import                       : 0x800000C8 (@200, NtFsControlFile)
Kernel import                       : 0x800000C6 (@198, NtFlushBuffersFile)

Not sure yet what these do and too lazy to figure it out

Where to start

(First impression) It probably makes the most sense to start with the easy functions and check if those break games. The emulated functions must be isolated from the rest of the kernel. To catch other bugs, we should first hook the functions called in bad situations such as Dbg* or KeBugCheck. Then it should be easy to work on the libc style Rtl* functions. Also the crypto functions Xc* should be straightforward to implement [and are already part of the xqemu-tools EEPROM tool]. File I/O could also be an easy target. After that, we should probably work on the virtual EEPROM and the HAL IO-functions. We are probably in a good position then to start work on the memory functions. If that is also working we can start XBE Loading with Xe*. Finally, work on threading and interrupts could start.