Test every non tested opcode #5
Comments
|
Currently left(ignoring those still pending in the other issue(pushf(d)/popf(d)), as well as protected-mode instructions and related instruction for protected mode-specific functionality(0F beginning range instructions)):
Side note: opcode 82h is missing from the intel-opcodes list, which is an alias for opcode 80h( http://ref.x86asm.net/coder32.html#x82 ). |
|
Just created a little 'testsuite' that executes many of the basic instructions mentioned in my previous post(although it requires to be logged and verified manually by looking for register/memory reads/writes/changes): Although I've coded opcodes 00-3F by using raw binary statements to make sure the nasm assembler doesn't actually create opcodes outside of that range somehow(2-byte versions of them). The only test that actually verifies itself using assembly instructions is the ret imm testing, much in the way your stack tests run. Fixed some bugs in the basic *D opcodes for the 00-3F range being the wrong opcode byte. |
|
So, combining both testsuite results, that only leaves possible errors in:
** partially tested and verified in both testsuites |
|
Just found a little 'bug' in my emulator(which seems to be only half documented in the 80386 programmer's reference manual's far return instruction description). (E)SP is increased before popping SS:(E)SP during a stack switch to an outer privilege level(resulting CS.RPL>CPL), which is documented, but also after popping the SS:(E)SP(thus, increasing(popping) the caller's stack variables. The latter isn't documented within the 80386 programmer's reference manual as far as I can see. |
|
Having fixed the far return using the immediate on both stack locations(both on source and destination stacks), the Extended Memory Tester v3.0 now properly detects the video card and extended memory, continuing to test memory:D This app now runs properly: https://archive.org/details/msdos_TESTEXT3_shareware |
|
Hmmmm..... One basic device that can be used to test the ISNS/OUTS instructions is the IDE/ATA harddisk's buffer(Command E4 to read the buffer, Command E8 to write the buffer). First fill it with a pattern manually(512 bytes), then read it back and verify the pattern. That should be able to verify all in(s)/out(s) instructions. Edit: Although I know some of them work properly(8-bit plain, 8-bit string and 16-bit string) work already, since they're used during POST(8-bit plain), during disk reads(8-bit or 16-bit reads), disk writes(8-bit or 16-bit writes) and CD-ROM access(16-bit reads and writes). The only one of those still untested are string and normal 32-bit variants and normal 16/32-bit single input/output. |
|
I see you're busy on some more (protected mode) tests. Great! That now just leaves:
So mostly various kinds of move instructions and miscellaneous instructions. And of course all remaining protected-mode functionality itself(task switching etc.) is still left to test. I'm currently wondering if protected-mode functionality of UniPCemu still has bugs(concerning call gates and interrupts etc. pushing data on the stack with(out) stack switch) in various cases. |
|
I've found a pretty big bug on my emulator comparing its memory dump with Bochs' after the execution of the test suite: the Dirty bit of a PTE is not properly set on a write when the same page has been previously accessed by a read. |
|
I've just ran Bochs 2.6.9 and dumped the 640K memory. Then I did the same with UniPCemu's MMU functionality(running the debugger, press and hold circle and then tap square) when reaching the HLT at the end of the POST 0xFF(when reaching the HLT status). It reveals there's quite a lot wrong, apparently: So that means:
|
That's the IDT area, but it's not necessarily a problem with the IDT management, could be something that writes garbage at the wrong address. Possible. Maybe, or again you could be writing garbage to the wrong addresses (see below). Maybe. Can't tell right now what the segment at 0x2e000 is used for, but dword at 0x9F000 should be 0x50465046, for some reason you're missing the last byte 0x50 at 0x9F003, or you have overwritten it with 00. It seems like you have problems with the MMU. |
|
When debugging the direct memory writes by the Paging unit, I see:
So the Paging unit requests the memory mapping to actually write the correct values there at some point. Maybe something else is somehow overwriting it? Edit: Looking again at my own created logs(made using MinGW-w64 using the following script), from UniPCemu's ROM directory(which contains the Bochs dump as well as the ROMs): It copies the memory.dat to the ROM directory for processing, then edits it to become 640K large, finally executing the difference dump as per your documentation. I thought that the left was the Bochs dump, while the right value was my own. Looking at it again(as well as reading your documentation on the results of the gawk command again), it's reversed. So those three pages aren't supposed to be accessed(since bits 5(PTE&PDE)&6(PDE) are supposed to be cleared instead of set in the end of logging). But they're accessed anyways, which isn't supposed to happen? So those paged accesses on those locations are instructions addressing a wrong point in memory, or incorrect mapping somehow? |
|
PTE at 0x23e0 is for the page at 0xF8000. Its Accessed bit should be 0, not 1 (byte at 0x23e0 should be 0x07 not 0x27). It seems like you're fetching the NOPs that are present in that area of the test code. Or maybe your MMU is updating the wrong PTE. I also think the dword at 0x9F000 is interesting. See what is wrinting 00 at 0x9F003, overwriting the 0x50. |
|
About 9F003, I see 0x50 being written at 10:0000C455. The current testsuite lst used: |
|
Just found a bug that caused the highest written byte to memory to remain unlogged(it was storing size-1 in the memory usage variable, instead of size). It still needed to add 1 to the current address for detection of memory usage, which it didn't(used for logging purposes only). |
|
The first two are instruction fetches triggering the accessed being set during writeback of the Page tables. |
|
It's checking an instruction fetch operand there at 10:7FFB. It's checking a DWORD at that location. A part of said operand is at F8000. It's the byte after 0F85. The PDE is 000023E0. The PTE to write back is 0x000f8027. It's the final byte of line 15393 in my test386.lst file. |
This seems correct? IT IS |
|
You say that Paged memory at 0xF8000-FA000 shouldn't be accessed, but there's executable code there that's executing in my case(as well as being present in the test386.lst). The test386.lst says this about said instruction: It seems to be one of the arithmetic tests that's executed. The entire block of nasm code: That's bit_m.asm, row 85 being assembled. |
|
I'm getting confused by all the edits. |
|
I've recompiled the test386.asm source code(it uses the BOCHS version and 386-specific tests(POST E0) enabled). As I said in my last post, the strange page (which has a PTE/PDE located at paged address F8000 and onwards) is part of the E0 tests that are executing and are present in the test386.lst generated by the nasm compiler. This is the result of UniPCemu's memory dump(which is a direct dump of physical RAM, which is zero-padded to 640K) compared against the Bochs memory dump, as in your instructions at the test386.asm main code page(Readme). The second column IS confirmed to be UniPCemu's data in memory. The third column is Bochs' dump. |
|
BTW dword at 0x2E000 is currently used as scratch memory for arith-logic tests 0xEE. |
|
Just ran the testsuite on Bochs again. Ran the continue command to let it run until the permanent HLT. Then used Ctrl-C to break into the debugger. It's not a problem with UniPCemu there! Bochs errors out at the processor-specific arithmetic logic tests! It's at 0010:0000C002(the error jumped location) when it's processing those. So Bochs is having a problem with those tests(the E0 tests)! Edit: Having disabled those tests using the flag, I now see Bochs dumping all EE test results into it's debugger window. |
Are you absolutely 100% sure? Because if I compare your post with a Bochs dump I've done, the 2nd middle column seems exactly like Bochs' data. For example the correct value for dword 2E000 is 0xFFFFFF01. |
|
Well, I know for sure that the dumps of UniPCemu's Paging table locations contain those values in the dump and memory variable (0x27 and 0x67). So the second column is the UniPCemu memory and the third column is the Bochs memory. |
Yes, 0xE0 are the undefined behaviours tests. Bochs is not a faithful replica of the i80386. |
|
This is my new dump of the comparison of UniPCemu and Bochs after having disabled said test: Only a few bugs left! :D |
|
:D |
I can confirm |
|
OK. So that's correct behaviour. What about 32-bit SLDT/STR? |
Upper 16 bits are set to 0. |
|
So that's the same behaviour as UniPCemu has already implemented. The only thing that keeps me wondering is what the remainder of the bug in my instruction emulation and related emulation itself are. Still can't get NT 3.1 to boot from the hard disk(did manage to fix Ctrl+Alt+Del just now, which causes the error code of the BSOD within brackets (0x0000007B (< thisone >, 0x00000000, 0x00000000, 0x00000000)) to change, but it still fails to correctly boot the hard disk due to some weird driver problems(although all HDD timings should be OK according to PCem(based on my timing of ATAPI PACKET command to be 20us)) weirdly crashing the disk driver(eventually ntoskrnl crashing on itself after the write(and readback) of the ATAPI sector count register(actually ATAPI Interrupt Identification register, which is R/O). The primary drive(IDE HDD) seems to be detected just fine, but immediately after it tries to detect the ATAPI secondary drives(secondary master&slave), which it somehow doesn't like, causing the kernel to panic somehow(perhaps due to a CPU bug)? |
|
Just been reading http://www.rcollins.org/Productivity/DescriptorCache.html . It makes sense, but the tables at the bottom seems to have the descriptor S-bit(System) inversed? Also, it seems that the Executable bit isn't required for execution permissions on the code segment descriptor cache(code fetches using said descriptor)? |
|
Any idea if something happens besides the CPU reset line being raised and lowered on a triple fault? So what happens if IDTR.limit<3, an interrupt occurs(leading to a triple fault) and the A20 gate is disabled(forcing A20 to 0)? Will the CPU fetch from FFFFFFF0(A20 gate on) or FFEFFFF0(A20 gate off, probably leading to an #UD fault that might be used by software for CPU Identification(using EDX etc.)) after reset toggles back to 0(due to bus logic)? |
Yep, it's inverted. S=1 is code/data, S=0 is system
Code segment bit is checked in protected mode when CS is loaded. |
I've always thought A20 is enabled when the CPU is reset, because that's what every emu does. I've never questioned that behaviour. But this is telling another story: |
|
Yes, found that article as well. So that means that A20 still keeps it's state when resetting externally(using the 8042 or system port A). But what's explained nowhere is what happens when the CPU sends a shutdown signal on the bug(due to triple fault). Various documentation says the reset line is triggered, which makes sense. But what's happening to the A20 gate in that case isn't explained ANYWHERE. One simple way to verify it is as said earlier, LIDT in real mode with a limit of 0, then throw an interrupt. Maybe hook Interrupt 06h and make it set the top-left of the display character(direct VGA A0000&A0001 writes) then CLI HLT? If that triggers instead of the BIOS rebooting, you'd know that(at least for that motherboard it's ran on) A20 isn't affected by the shutdown? Such a program would be trivial with nasm: What happens when you run said program on your 386/486 machine? Does it print A or reboot? |
|
Any plans on testing the remainder of protected-mode instructions and functionality(far jmp/call in protected mode(call gates and normal segments), interrupts(both INT and IRET), far return)? I also see some untested control transfers(opcodes E9 and EB(near jumps), C2(near return))? |
I've tested your program on all my retro gear and the results are interesting: So it happears to depend on the specific system.
As the title of this issue implies, eventually yes, but real life issues take the precedence. |
|
I've just gave Windows 3.0,3.1 and WFW3.11 another go for real and protected(WFW in it's only 386 mode) modes. Both 3.0 run fine now(when in Windows apps). The only issue left there is an unresponsive MS-DOS program when ran from Windows(real(3.0) and protected(3.0&3.1) modes). In 3.0, typing "dir" a few times, after about 3-4 times "r" is printed at the MS-DOS 6.22 prompt. In 3.1 it's fully unresponsive. COMMAND.COM reaches the command line and input in both cases. 386 enhanced mode is broken for all 3.x distributions(thus WFW3.11 doesn't boot at all) and all result in a text-mode black screen(80x25) with blinking cursor at character at 0,0. Any ideas on possible causes? If even 3.0 in real mode goes wrong, does that mean a CPU bug or perhaps a 8042/Keyboard issue? Edit: Just improved the 8042 a bit to not change the buffer while translating the scancode set(only 1c and 9c in the buffer instead of 1c(status bit 0= 1), f0(status bit 0= 0), 9c(status bit 0= 1). Instead the f0 step doesn't happen so just 1c and 9c(both status 0=1. status 0=0 and value=1c in between when read(not changing to f0 incorrectly)). |
|
Just fixed a 'bug' in the 8042 that was clearing empty 8042 output buffers to become zeroed when checking for new input to fill the buffer with. But somehow, MS-DOS and Windows 3.x seem to require said value to still be there to run properly? Also, ever noticed that you can actually nest windows 3.0 real mode sessions within each other this way? Run windows (on a 80(1)86 or with the /r parameter), then from within Windows, run the MS-DOS prompt shortcut, after which you can once again type "win /r" to run a second Windows 3.0a session. This is proven to actually be nested, because when I looked at Windows' memory stats(using the Help->About option), less free memory was seen left in the nested Windows session compared to the one that was already loaded before executing the MS-DOS prompt shortcut. |
|
Just looking through the opcodes again(the normal opcodes seem to be fine. The 0F opcodes I'm still reviewing). Strangely enough, http://ref.x86asm.net/coder32.html#x0FB7 and http://ref.x86asm.net/coder32.html#x0FBF seems to imply that a 16-bit version of said opcodes does exist, as it mentions r16/32 in the op1 column? Edit: Just found and fixed a bug in the PUSH/POP SegReg instructions that caused it to increase/decrease virtual ESP(for protection checks) by 2 instead of 4 while verifying against 16-bit memory accesses. As well as fixing 16-bit SIDT/SGDT to properly fill the final byte with 0x00(386+) or 0xFF(286+). |
16-bit 0FB7 and 0FBF are both |
|
So, 0FBF is MOVSX Gv,Ew and 0FB7 is MOVZX Gv,Ew. Is that correct? The Gv being either a 32-bit register or 16-bit register(depending on the operand size) and Ew always being a 16-bit memory location or register? |
|
Just ran the Bochs comparision of memory again. Now I end up with the following in the compare file: Now the question, what is it? Hmmmm... They're all accessed(A-bit) bits of said descriptors? Aren't those set when said code/data descriptors are loaded into the processor(before parsing faults for their contents)? Is it the case that VERR and VERW don't 'touch' the descriptor, setting the Accessed-bit(bit0) of the selector to 1)? Edit: Changing the VERR and VERW instructions to not touch the descriptor(just load it, nothing more), those differences compared to Bochs disappear? Interestingly enough, Bochs instructions of LAR, LSL, VERR and VERW all have this behaviour(as far as I can see in the source code file). They don't load the access rights(set it's accessed bit)? |
Yes. |
The accessed bit is set when the descriptor is loaded into a segment register. LAR, LSL, VERR and VERW load the descriptor data from memory without altering any seg reg and the Intel docs don't mention any a-bit alteration for those instructions. So I think the Bochs behaviour is correct. |
|
OK. So my current implementation of MOVSX and MOVZX is now correct again. Also, Ev=r/m16/32 and Gv(which is used for the register part of opcode 0FBF/0FB7)=r16/32. The 'v' part of that means that it's a 16-bit register or memory or 32-bit register or memory, depending on the operand size(e.g. Ev means AX in 16-bit operand size, EAX in 32-bit operand size with R/M being 0 and MOD being 3, the same applies to Gv being EAX or AX when reg=0, depending on the operand size). So the 'v' suffix is a 32-bit memory location(r/m only) or register(either r/m or reg) or 16-bit one, depending on the operand size. You seem to think the 'v' is always 32-bit register or memory. But all documentation I can find implies it's actually a word or doubleword location in memory or register(which is determined by the operand size instead of being hardcoded for the instruction(as is the case with 'b', 'w' and 'd')). So, my current implementation according to the above logic is: The original crash to a black screen with a message on Windows 95 not being able to boot and please reinstall is now gone. Instead, the Windows 95 hangs the graphics animation that's booting(when not booted in Safe mode) instead. I see it infinitely #UD faulting on a ARPL instruction now? Edit: The Windows 95 bootlog.txt became very small now, only a few rows: So IOS.VXD is now inexplicably crashing? Edit: Reinstalling Windows 95 seems to fix that(see vogons thread https://www.vogons.org/viewtopic.php?f=9&t=65223&p=747883#p747883 ). In safe mode it still gets a BSOD on the "Initializing KERNEL" being the last thing that's logged? |
|
I notice something, looking through the arithmetic(EE) tests: you're not testing all memory-related cases of the tested opcodes. Also C* and D* shift/rotate instructions aren't tested for memory operands. |
|
Just tried the OS/2 warp 3 installer again. Now I noticed that it was throwing a #GP(0) fault at a MOV CR2,ECX instruction. Then, digging deeper, I noticed that it was faulting because it was checking the invalid condition of CR0 with PG set and PE cleared at the loading of ANY CR-register! Whoops. Then, fixing that, it continues on to the DISK1's boot screen(which finally appears now!) and seems to hang(the floppy disk indicator keeps being on, thus the drive motor isn't being stopped at anytime). |
|
I'm now looking into the Expand-Down segments. Some sources state that the base is actually 64KB or 4GB(thus no additional effect compared to Expand-Up segments, except the limit algorithmic) below the base address? So does that mean that if the Base address in the descriptor is set to e.g. 0x80000000 and a offset of FFFC(on a 64KB data descriptor with the Big bit set to 0), said dereference actually results in a linear address of 0x7FFFFFFC? |
|
One good thing to report now: Windows 3.11 is now properly booting in 386 enhanced mode! :D It just seems that one 3.x 386-enhanced mode application(using win32s) seems to crash within the kernel due to the kernel double faulting and triple faulting because the stack overflows(due to nested page faults) on address 0x80000FFC on what seems to be inside the page fault handler, faulting on a ADD instruction(either 8-bit or 32-bit) addressing some user-mode memory(just below 80000000), which fails due to said page not being in memory? Edit: Still can't get Linux to boot, since it tries to return using a near return to a high user-mode address(in the 7XXXXXXX range) while it's having a CS base of c0000000 and a limit of 3fffffff(thus mapping lower memory to high memory for easy transition to 32-bit split user and kernel modes transparently). Since said address breaks the CS limit, it throws a #GP(0) fault, which Linux obviously doesn't like. I do see that said return address on the stack is never written to, except by the boot extraction program(it's final instruction being a block move writing to said address(probably the extracted kernel image)) just before jumping to the kernel entry point. It's still in it's intial kernel software, as CR3 is stilll 101000(4KB(it's PDE) past the 1MB barrier). So the kernel itself that's used during the setup is crashing in that case(due to some strange unmodified memory location being RETN'ed to. |
|
Hmmmm... Running JEMM386 in MS-DOS seems to run fine in all settings(386, Pentium with and without VME). But when I try to run it in FreeDOS, it faults(exception 03h, 08h and 09h, as it dumps) on the Pentium and even triple faults on the 80386! Edit: The 80386 even triple faults on JEMM386 when ran in FreeDOS. Runs fine in MS-DOS 6.22, though. So perhaps a FreeDOS problem with my x86 emulation? |
|
Managed to fix some more bugs, including the PIC triggering incorrect INT 0 problems(providing interrupts when there were none). :S Now, with some more fixes, Windows 95 can (sometimes) boot in Normal mode until the Initializing the kernel(and at all other times a Windows Protection Error on the second ESDI_506.pdr(hard disk driver, probably secondary ATA controller?). Trying to boot the Basic Linux 3.5 from a floppy crashes with an IRET(D) trying to execute a task return to task 0x0001(descriptor type 02h), due to the task register still being NULL(0x0000, type=0x82)? That's odd, as TR is never loaded by LTR? No LTR is observed during boot and FLAGS' high 4 bits are 0x7? |
|
Just added IN/OUT instruction tests(which are actually fully automated checks, not requiring manual looking at it's results which use OUT and IN instructions to port 0 and 2(the DMA address ports) combined with the resetting of the flopflop(writing port 0xC) to the remainder testsuite. Still trying to get MS-DOS 6.22 setup disk 1 to boot. It keeps hanging when handling some interrupt or something like that, while detecting the system configuration(It literally says in it's cyan screen: "Please wait. Setup is checking your sytem configuration."). Somehow, the CPU has some bug causing it to not work properly there? |
|
Interesting! With the new RCR checks fixed, I get errors on UniPCemu's 32-bit RCR instruction, according to the reference! test386-EE-reference.zip |
A comment from VOGONS thread https://www.vogons.org/viewtopic.php?f=9&t=60095:
The text was updated successfully, but these errors were encountered: